Dynamic compensation of polygon and motor tolerances using galvanometer control curves

Abstract

A light detection and ranging system is provided. The system includes a galvanometer mirror; a multi-faceted light steering device; and a controller device, the controller device including one or more processors, a memory, and processor-executable instructions stored in the memory. The processor-executable instructions include instructions for receiving a first movement curve of a galvanometer mirror of a LiDAR scanning system; receiving calibration data of the multi-faceted light steering device of the LiDAR scanning system; generating a second movement curve of the galvanometer mirror based on the calibration data and the first movement curve; and providing one or more control signals to adjust the movement of the galvanometer mirror based on the second movement curve.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 177,289, filed on April 20, 2021, entitled “DYNAMIC COMPENSATION TO POLYGON AND MOTOR TOLERANCE USING GALVO CONTROL PROFILE” and U.S. Non-Provisional Patent Application No. 17 / 714,077, filed on April 5, 2022, entitled “DYNAMIC COMPENSATION TO POLYGON AND MOTOR TOLERANCE USING GALVO CONTROL PROFILE”. The contents of both applications are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] The present disclosure relates generally to optical scanning and, more particularly, to methods and systems for dynamically controlling the movement of a galvanometer mirror based on calibration data and real-time position feedback data of a multi-faceted light steering device. Background Art

[0004] Light detection and ranging (LiDAR) systems use light pulses to create images or point clouds of the external environment. Some typical LiDAR systems include a light source, a light transmitter, a light steering system, and a light detector. The light source generates a light beam that is directed in a specific direction by the light steering system when transmitted from the LiDAR system. When the transmitted light beam is scattered by an object, a portion of the scattered light returns to the LiDAR system as a return light pulse. The light detector detects the return light pulse. Using the difference between the time when the return light pulse is detected and the time when the corresponding light pulse in the light beam is transmitted, the LiDAR system can use the speed of light to determine the distance to the object. The light steering system can guide the light beam along different paths to allow the LiDAR system to scan the surrounding environment and produce an image or point cloud. The LiDAR system can also use technologies other than time of flight and scanning to measure the surrounding environment. Summary of the invention

[0005] The present disclosure provides methods and systems for compensating for manufacturing-induced non-uniformity and assembly errors in a multi-faceted light steering device (e.g., a polygonal mirror) by modifying a galvanometer mirror movement curve using calibration data of the light steering device. Also provided are methods and systems for compensating for positional inaccuracies of a light steering device by further modifying a galvanometer mirror movement curve using real-time position feedback data of the light steering device and / or real-time position feedback data of the galvanometer mirror. The methods and systems described herein facilitate generating improved scan lines (e.g., straight scan lines) in a LiDAR scan pattern. The improved LiDAR scan pattern improves the efficiency and accuracy of one or more subsequent processes (e.g., point cloud-based perception). The overall performance of the LiDAR system and the quality of the output point cloud may also be improved.

[0006] In one embodiment, the present disclosure provides a light detection and ranging system. The system includes a galvanometer mirror; a multi-faceted light steering device; and a controller device, the controller device including one or more processors, a memory, and processor-executable instructions stored in the memory. The processor-executable instructions include instructions for: receiving a first movement curve of a galvanometer mirror of a LiDAR scanning system; receiving calibration data of the multi-faceted light steering device of the LiDAR scanning system; generating a second movement curve of the galvanometer mirror based on the calibration data and the first movement curve; and providing one or more control signals to adjust the movement of the galvanometer mirror based on the second movement curve.

[0007] In one embodiment, the present disclosure provides a method for controlling a light detection and ranging (LiDAR) scanning system. The method is executed by one or more processors and memories and includes: receiving a first movement curve of a galvanometer mirror of the LiDAR scanning system; receiving calibration data of a multi-faceted light steering device of the LiDAR scanning system; generating a second movement curve of the galvanometer mirror based on the calibration data and the first movement curve; and providing one or more control signals to adjust the movement of the galvanometer mirror based on the second movement curve.

[0008] In one embodiment, the present disclosure provides a non-transitory computer-readable medium storing processor-executable instructions including instructions. The instructions, when executed by one or more processors of an electronic device, cause the electronic device to perform a process, the process comprising receiving a first movement curve of a galvanometer mirror of a LiDAR scanning system; receiving calibration data of a multi-faceted light steering device of the LiDAR scanning system; generating a second movement curve of the galvanometer mirror based on the calibration data and the first movement curve; and providing one or more control signals to adjust the movement of the galvanometer mirror based on the second movement curve.

[0009] In one embodiment, the present disclosure provides a controller device for a light detection and ranging (LiDAR) scanning system. The controller device includes one or more processors; a memory; and processor-executable instructions stored in the memory. The processor-executable instructions include instructions for: receiving a first movement curve of a galvanometer mirror of the LiDAR scanning system; receiving calibration data of a multi-faceted light steering device of the LiDAR scanning system; generating a second movement curve of the galvanometer mirror based on the calibration data and the first movement curve; and providing one or more control signals to adjust the movement of the galvanometer mirror based on the second movement curve.

[0010] In one embodiment, the present disclosure provides a vehicle including a light detection and ranging (LiDAR) scanning system. The system includes a galvanometer mirror; a multi-faceted light steering device; and a controller device, the controller device including one or more processors, a memory, and processor-executable instructions stored in the memory. The processor-executable instructions include instructions for: receiving a first movement curve of the galvanometer mirror of the LiDAR scanning system; receiving calibration data of the multi-faceted light steering device of the LiDAR scanning system; generating a second movement curve of the galvanometer mirror based on the calibration data and the first movement curve; and providing one or more control signals based on the second movement curve to adjust the movement of the galvanometer mirror. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present disclosure may be better understood by referring to the following figures described in conjunction with the drawings, wherein like components may be represented by like reference numerals.

[0012] Figure 1 One or more exemplary LiDAR systems disposed or included in a motor vehicle are shown.

[0013] Figure 2 is a block diagram illustrating the interaction between an exemplary LiDAR system and a number of other systems including a vehicle perception and planning system.

[0014] Figure 3 A block diagram illustrating an exemplary LiDAR system is shown.

[0015] Figure 4 is a block diagram illustrating an exemplary fiber-based laser source.

[0016] FIG. 5A to FIG. 5C An exemplary LiDAR system using pulse signals to measure the distance to objects disposed in a field of view (FOV) is shown.

[0017] Figure 6 is a block diagram illustrating an exemplary apparatus for implementing the systems, apparatus, and methods in various embodiments.

[0018] Fig. 7A A simplified LiDAR scanning system with a multi-faceted light redirecting device and a galvanometer mirror is shown in accordance with some embodiments.

[0019] Figure 7B Another example of a multi-faceted light redirecting device in accordance with some embodiments is shown.

[0020] Figure 7C is a side view of an example multi-faceted light redirecting device in accordance with some embodiments.

[0021] Figure 8 Example movement curves of a galvanometer mirror and corresponding LiDAR scan patterns are shown in accordance with some embodiments.

[0022] Fig. 9 An example LiDAR scan pattern is shown with a magnified view of a portion of the pattern in accordance with some embodiments.

[0023] Fig.10 is a block diagram illustrating the use of a galvanometer mirror movement profile to control the mirror position based on light steering device calibration data and real-time position feedback data.

[0024] Fig.11 An example movement curve of a galvanometer mirror and a magnified view of a portion of the curve are shown for controlling the mirror position in accordance with some embodiments.

[0025] Fig.12 An example LiDAR scan pattern and a magnified view of a portion of the pattern are shown in accordance with some embodiments.

[0026] Fig.13 is a flow chart illustrating a method for controlling a LiDAR scanning system. DETAILED DESCRIPTION

[0027] To provide a more thorough understanding of the present invention, the following description sets forth many specific details, such as specific configurations, parameters, examples, etc. However, it should be appreciated that these descriptions are not intended to limit the scope of the present invention, but are intended to provide a better description of exemplary embodiments.

[0028] Throughout the specification and claims, the following terms take the meanings explicitly associated herein, unless the context clearly dictates otherwise.

[0029] As used herein, the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may. Therefore, as described below, the various embodiments of the present disclosure can be easily combined without departing from the scope or spirit of the invention.

[0030] As used herein, the term "or" is the inclusive expression "or" and is equivalent to the term "and / or" unless the context clearly dictates otherwise.

[0031] The term "based on" is not exclusive and allows for being based on additional factors not described unless the context clearly dictates otherwise.

[0032] As used herein, unless the context dictates otherwise, the term "coupled to" is intended to include direct coupling (where two elements being coupled to each other are in contact with each other) and indirect coupling (where at least one additional element is located between the two elements). Thus, the terms "coupled to" and "coupled with" are used synonymously. In the context of a network environment where two or more components or devices are able to exchange data, the terms "coupled to" and "coupled with" are also used to mean "communicatively coupled with," possibly via one or more intermediate devices.

[0033] Although the following description uses the terms "first," "second," etc. to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first sensor may be referred to as a second sensor, and similarly, a second sensor may be referred to as a first sensor without departing from the scope of the various described examples. Both the first sensor and the second sensor may be sensors, and in some cases, may be separate and different sensors.

[0034] Furthermore, throughout the specification, the meanings of “a,” “an,” and “the” include plural expressions, and the meaning of “in” includes “in” and “on.”

[0035] Although some of the various embodiments presented herein constitute a single combination of inventive elements, it should be understood that the inventive subject matter is considered to include all possible combinations of disclosed elements. Thus, if one embodiment includes elements A, B, and C, and another embodiment includes elements B and D, the inventive subject matter is also considered to include other remaining combinations of A, B, C, or D, even if not explicitly discussed herein. In addition, the transitional term "comprising" means having parts or components, or those parts or components. As used herein, the transitional term "comprising" is inclusive or open-ended, and does not exclude additional, unlisted elements or method steps.

[0036] In the following disclosure, numerous references may be made to servers, services, interfaces, engines, modules, clients, peers, portals, platforms, or other systems formed by computing devices. It should be understood that the use of these terms is considered to represent one or more computing devices having at least one processor (e.g., ASIC, FPGA, PLD, DSP, x86, ARM, RISC-V, ColdFire, GPU, multi-core processor, etc.), at least one processor being configured to execute software instructions stored on a computer-readable, tangible, non-temporary medium (e.g., hard drive, solid-state drive, RAM, flash memory, ROM, etc.). For example, a server may include one or more computers that operate as a network server, database server, or other type of computer server in a manner that performs the described roles, responsibilities, or functions. It should also be understood that the disclosed computer-based algorithms, processes, methods, or other types of instruction sets may be implemented as a computer program product that includes a non-temporary, tangible computer-readable medium storing instructions that cause a processor to perform the disclosed steps. The various servers, systems, databases or interfaces may exchange data using standardized protocols or algorithms, possibly based on HTTP, HTTPS, AES, public-private key exchange, web service APIs, well-known financial transaction protocols or other electronic information exchange methods. The data exchange may be performed over a packet-switched network, a circuit-switched network, the Internet, a LAN, a WAN, a VPN or other types of networks.

[0037] As used in this specification and the claims that follow, when a system, engine, server, device, module, or other computing element is described as being configured to perform or execute functions on data in a memory, the meaning of "configured to" or "programmed to" is defined as one or more processors or cores of the computing element being programmed by a set of software instructions stored in the memory of the computing element to perform a set of functions on target data or data objects stored in the memory.

[0038] It should be noted that any language pointing to a computer should be understood to include any suitable combination of computing devices or network platforms, including servers, interfaces, systems, databases, agents, peers, engines, controllers, modules, or other types of computing devices that operate alone or together. It should be understood that the computing device includes a processor that is configured to execute software instructions stored on a tangible non-temporary computer-readable storage medium (e.g., a hard disk drive, FPGA, PLA, solid-state drive, RAM, flash memory, ROM, etc.). The software instructions configure or program the computing device to provide roles, responsibilities, or other functions as discussed below about the disclosed device. In addition, the disclosed technology can be implemented as a computer program product, which includes a non-temporary computer-readable medium storing software instructions, which enables the processor to perform the disclosed steps related to the implementation of a computer-based algorithm, process, method, or other instruction. In some embodiments, various servers, systems, databases, or interfaces use standardized protocols or algorithms to exchange data, possibly based on HTTP, HTTPS, AES, public-private key exchange, web service API, known financial transaction protocols, or other electronic information exchange methods. Data exchange between devices may be performed via a packet switching network, the Internet, a LAN, a WAN, a VPN or other types of packet switching networks, circuit switching networks, cell switching networks, or other types of networks.

[0039] The LiDAR system is a type of three-dimensional depth sensor. In the LiDAR scanning system, a rotatable polygonal mirror and a galvanometer mirror are used for a light steering mechanism to scan a laser to illuminate an object in the field of view (FOV) and direct the return light to the receiver of the LiDAR system. The polygonal mirror can be made of metal, polycarbonate and / or glass, for example. Metal-based polygonal mirrors are widely used due to their mature manufacturing processes. Typically, metal-based polygonal mirrors are processed by using computer numerical control (CNC) equipment with diamond turning. Diamond turning is turning using a tool with a diamond bit. The diamond turning process is used for machining precision components such as optical components. The diamond turning process has high accuracy, but the CNC process using diamond turning limits the output due to its long processing time.

[0040] For high-volume applications such as automotive sensors, metal-based polygonal mirrors cannot meet mass production requirements. Therefore, other types of polygonal mirrors (such as polycarbonate molded or glass molded polygonal mirrors) are more practical. Those molded polygonal mirrors can achieve very low costs and high yields. But in the mass production of molded polygonal mirrors, the uniformity of different facets of the polygonal mirror is difficult to control. The polygonal mirror has multiple reflective side surfaces or facets that reflect light. The polygonal mirror can be used to guide the transmitted light to the FOV and guide the return light to the receiver of the LiDAR system. Therefore, the uniformity of the reflective facets of the polygonal mirror may be important because the differences caused by the unevenness between the facets may degrade the accuracy of the LiDAR scanning pattern (for example, resulting in discontinuous scan lines).

[0041] In addition, the process of assembling the motor using the polygonal mirror may also introduce cumulative assembly errors. The motor is used to rotate the polygonal mirror. When the polygonal mirror rotates, the assembly error may affect the light reflecting ability of the different reflective surfaces of the polygonal mirror, and in turn degrade the accuracy of the LiDAR scanning pattern. For optical devices like LiDAR systems, deviations from the ideal scanning pattern (for example, caused by discontinuous scanning lines) affect the quality and accuracy of the resulting point cloud, which in turn degrades the subsequent perceived accuracy. Therefore, it is necessary to reduce or eliminate the non-uniformity differences and assembly errors between different facets of the polygonal mirror. The present disclosure provides methods and systems for compensating for non-uniformity differences and assembly errors caused by the manufacture of a light steering device (e.g., a polygonal mirror) by modifying the movement curve of a galvanometer mirror using calibration data and real-time position feedback data of the polygonal mirror.

[0042] Figure 1 One or more exemplary LiDAR systems 110 provided or included in a motor vehicle 100 are shown. The motor vehicle 100 may be a vehicle with any level of automation. For example, the motor vehicle 100 may be a partially automated vehicle, a highly automated vehicle, a fully automated vehicle, or an unmanned vehicle. A partially automated vehicle may perform some driving functions without human driver intervention. For example, a partially automated vehicle may perform blind spot monitoring, lane keeping and / or lane changing operations, automatic emergency braking, smart cruise control, and / or traffic following, etc. Certain operations of a partially automated vehicle may be limited to specific applications or driving scenarios (e.g., limited to highway driving). A highly automated vehicle may generally perform all operations of a partially automated vehicle, but with fewer restrictions. A highly automated vehicle may also detect its own limits when operating the vehicle and require the driver to take over control of the vehicle when necessary. A fully automated vehicle may perform all vehicle operations without driver intervention, but may also detect its own limits and require the driver to take over when necessary. An unmanned vehicle may operate on its own without any driver intervention.

[0043] In a typical configuration, the motor vehicle 100 includes one or more LiDAR systems 110 and 120A-F. Each of the LiDAR systems 110 and 120A-F can be a scanning-based LiDAR system and / or a non-scanning LiDAR system (e.g., flash LiDAR). A scanning-based LiDAR system scans one or more light beams in one or more directions (e.g., horizontal and vertical directions) to detect objects in a field of view (FOV). A non-scanning-based LiDAR system emits a laser to illuminate the FOV without scanning. For example, a flash LiDAR is a non-scanning-based LiDAR system. A flash LiDAR can emit a laser to illuminate the FOV simultaneously using a single light pulse or light shots.

[0044] LiDAR systems are often an essential sensor for at least partially automated vehicles. Figure 1 As shown, the motor vehicle 100 may include a single LiDAR system 110 (e.g., without LiDAR systems 120A-F) disposed at the highest position of the vehicle (e.g., on the roof). Placing the LiDAR system 110 on the roof facilitates 360-degree scanning around the vehicle 100. In some other embodiments, the motor vehicle 100 may include multiple LiDAR systems, including two or more of the systems 110 and / or 120A-F. Figure 1 As shown, in one embodiment, multiple LiDAR systems 110 and / or 120A-F are attached to the vehicle 100 at different locations of the vehicle. For example, LiDAR system 120A is attached to the vehicle 100 at the right front corner; LiDAR system 120B is attached to the vehicle 100 at the front center; LiDAR system 120C is attached to the vehicle 100 at the left front corner; LiDAR system 120D is attached to the vehicle 100 at the right side mirror; LiDAR system 120E is attached to the vehicle 100 at the left side mirror; and / or LiDAR system 120F is attached to the vehicle 100 at the rear center. In some embodiments, LiDAR systems 110 and 120A-F are independent LiDAR systems with their own laser sources, control electronics, transmitters, receivers, and / or guidance mechanisms. In other embodiments, some of the LiDAR systems 110 and 120A-F may share one or more components, thereby forming a distributed sensor system. In one example, optical fiber is used to deliver laser light from a centralized laser source to all LiDAR systems. It should be understood that one or more LiDAR systems may be distributed and attached to the vehicle in any desired manner, and Figure 1Only one embodiment is shown. As another example, LiDAR systems 120D and 120E can be attached to the B-pillars of vehicle 100 instead of rearview mirrors. As another example, LiDAR system 120B can be attached to the windshield of vehicle 100 instead of the front bumper.

[0045] Figure 2 2 is a block diagram 200 showing the interaction between a vehicle-mounted LiDAR system(s) 210 and a plurality of other systems including a vehicle perception and planning system 220. The LiDAR system(s) 210 may be mounted on or integrated into a vehicle. The LiDAR system(s) 210 include sensors that scan laser light into the surrounding environment to measure the distance, angle, and / or velocity of objects. Based on the scattered light returned to the LiDAR system(s) 210, it may generate sensor data (e.g., image data or 3D point cloud data) representing the perceived external environment.

[0046] The (multiple) LiDAR systems 210 may include one or more of a short-range LiDAR sensor, a mid-range LiDAR sensor, and a long-range LiDAR sensor. The short-range LiDAR sensor measures objects that are approximately 20 to 40 meters away from the LiDAR sensor. For example, the short-range LiDAR sensor can be used to monitor nearby moving objects (e.g., pedestrians crossing the road in a school zone), parking assistance applications, etc. The medium-range LiDAR sensor measures objects that are approximately 100 to 150 meters away from the LiDAR sensor. The medium-range LiDAR sensor can be used, for example, to monitor road intersections, assist in entering or leaving the fast lane, etc. The long-range LiDAR sensor measures objects located approximately 150 to 300 meters. Long-range LiDAR sensors are typically used when the vehicle is traveling at higher speeds (e.g., in the fast lane), so that the vehicle's control system may only have a few seconds (e.g., 6 to 8 seconds) to respond to any situation detected by the LiDAR sensor. As Figure 2 As shown, in one embodiment, LiDAR sensor data can be provided to vehicle perception and planning system 220 via communication path 213 for further processing and control of vehicle operation. Communication path 213 can be any wired or wireless communication link capable of transmitting data.

[0047] Still refer to Figure 2In some embodiments, other onboard sensors 230 are used to provide additional sensor data alone or in conjunction with the LiDAR system 210. The other onboard sensors 230 may include, for example, one or more cameras 232, one or more radars 234, one or more ultrasonic sensors 236, and / or other sensors 238. The camera(s) 232 may capture images and / or video of the vehicle's external environment. The camera(s) 232 may capture, for example, high-definition (HD) video with millions of pixels in each frame. The camera produces monochrome or color images and videos. For some situations, color information may be important in interpreting the data (e.g., interpreting images of traffic lights). Color information may not be available from other sensors such as LiDAR or radar sensors. The camera(s) 232 may include one or more of a narrow focal length camera, a wide focal length camera, a side camera, an infrared camera, a fisheye camera, and the like. The image and / or video data generated by the camera(s) 232 may also be provided to the vehicle perception and planning system 220 via the communication path 233 for further processing and control of vehicle operation. Communications path 233 may be any wired or wireless communications link capable of transmitting data.

[0048] Other onboard sensors 230 may also include radar sensors 234. Radar sensors 234 use radio waves to determine the distance, angle, and speed of an object. Radar sensors 234 generate electromagnetic waves in the radio or microwave spectrum. Electromagnetic waves are reflected by objects, and some of the reflected waves return to the radar sensor, providing information about the location and speed of the object. Radar sensors 234 may include one or more of short-range radars, medium-range radars, and long-range radars. Short-range radars measure objects about 0.1 to 30 meters from the radar. Short-range radars can be used to detect objects near the vehicle, such as other vehicles, buildings, walls, pedestrians, cyclists, etc. Short-range radars can be used to detect blind spots, assist in lane changes, provide rear-end collision warnings, assist in parking, provide emergency braking, etc. Medium-range radars measure objects about 30 to 80 meters from the radar. Long-range radars measure objects located about 80 to 200 meters away. Medium-range and / or long-range radars can be used, for example, for traffic tracking, adaptive cruise control, and / or highway automatic braking. Sensor data generated by radar sensor(s) 234 may also be provided to vehicle perception and planning system 220 via communication path 233 for further processing and control of vehicle operation.

[0049] (Multiple) other vehicle-mounted sensors 230 may also include (multiple) ultrasonic sensors 236. (Multiple) ultrasonic sensors 236 use acoustic waves or pulses to measure objects located outside the vehicle. The sound waves generated by (multiple) ultrasonic sensors 236 are transmitted to the surrounding environment. At least some of the transmitted waves are reflected by the objects and returned to (multiple) ultrasonic sensors 236. Based on the return signal, the distance of the object can be calculated. (Multiple) ultrasonic sensors 236 can be used, for example, to check blind spots, identify parking spots, provide lane change assistance in traffic, etc. The sensor data generated by (multiple) ultrasonic sensors 236 can also be provided to the vehicle perception and planning system 220 via the communication path 233 for further processing and control of vehicle operation.

[0050] In some embodiments, one or more other sensors 238 may be attached to the vehicle and may also generate sensor data. The other sensor(s) 238 may include, for example, a global positioning system (GPS), an inertial measurement unit (IMU), etc. The sensor data generated by the other sensor(s) 238 may also be provided to the vehicle perception and planning system 220 via the communication path 233 for further processing and control of vehicle operation. It should be understood that the communication path 233 may include one or more communication links to transmit data between the various sensors 230 and the vehicle perception and planning system 220.

[0051] In some embodiments, Figure 2 As shown, sensor data from other onboard sensor(s) 230 may be provided to the onboard LiDAR system(s) 210 via communication path 231. The LiDAR system(s) 210 may process the sensor data from the other onboard sensor(s) 230. For example, sensor data from camera(s) 232, radar sensor(s) 234, ultrasonic sensor(s) 236, and / or other sensor(s) 238 may be correlated or fused with the sensor data LiDAR system(s) 210, thereby at least partially offloading the sensor fusion process performed by the vehicle perception and planning system 220. It should be understood that other configurations for transmitting and processing sensor data from various sensors may also be implemented (e.g., data may be transmitted to a cloud service for processing, and the processing results may then be transmitted back to the vehicle perception and planning system 220).

[0052] Still refer to Figure 2In some embodiments, sensors on other vehicle(s) 250 are used to provide additional sensor data alone or in conjunction with LiDAR system(s) 210. For example, two or more nearby vehicles may have their own LiDAR sensor(s), camera(s), radar sensor(s), ultrasonic sensor(s), etc. Nearby vehicles may communicate with each other and share sensor data. Communication between vehicles is also referred to as V2V (vehicle-to-vehicle) communication. For example, Figure 2 As shown, sensor data generated by (multiple) other vehicles 250 can be communicated to the vehicle perception and planning system 220 and / or (multiple) on-board LiDAR system 210 via communication path 253 and / or communication path 251, respectively. Communication paths 253 and 251 can be any wired or wireless communication links capable of transmitting data.

[0053] Sharing sensor data helps to better perceive the environment outside the vehicle. For example, the first vehicle may not sense a pedestrian that is behind the second vehicle but is approaching the first vehicle. The second vehicle can share sensor data related to this pedestrian with the first vehicle so that the first vehicle can have additional reaction time to avoid colliding with the pedestrian. In some embodiments, similar to the data generated by (multiple) sensors 230, the data generated by sensors on (multiple) other vehicles 250 can be correlated or fused with the sensor data generated by (multiple) LiDAR systems 210, thereby at least partially offloading the sensor fusion process performed by the vehicle perception and planning system 220.

[0054] In some embodiments, the smart infrastructure system(s) 240 are used to provide sensor data alone or in conjunction with the LiDAR system(s) 210. Certain infrastructure may be configured to communicate with vehicles to convey information, and vice versa. Communication between vehicles and infrastructure is often referred to as V2I (vehicle-to-infrastructure) communication. For example, the smart infrastructure system(s) 240 may include a smart traffic light that can communicate its status to approaching vehicles with a message such as "turning yellow in 5 seconds." The smart infrastructure system 240 may also include its own LiDAR system installed near a road junction so that it can transmit traffic monitoring information to the vehicle. For example, a vehicle turning left at a road junction may not have sufficient sensing capabilities because some of its own sensors may be blocked by vehicles in the opposite direction. In this case, the sensors of the smart infrastructure system(s) 240 can provide useful and sometimes important data to the left-turning vehicle. Such data may include, for example, traffic conditions, information about objects in the direction the vehicle is turning, traffic light status and predictions, etc. Sensor data generated by the (multiple) smart infrastructure system 240 can be communicated to the vehicle perception and planning system 220 and / or the (multiple) on-board LiDAR system 210 via communication path 243 and / or communication path 241, respectively. Communication paths 243 and / or 241 may include any wired or wireless communication links capable of transmitting data. For example, sensor data from the (multiple) smart infrastructure system 240 can be transmitted to the (multiple) LiDAR system 210 and correlated or fused with sensor data generated by the (multiple) LiDAR system 210, thereby at least partially offloading the sensor fusion process performed by the vehicle perception and planning system 220. The V2V and V2I communications described above are examples of vehicle-to-X (V2X) communications, where "X" represents any other device, system, sensor, infrastructure, etc. that can share data with the vehicle.

[0055] Still refer to Figure 2, through various communication paths, the vehicle perception and planning system 220 receives sensor data from one or more of the (multiple) LiDAR systems 210, (multiple) other onboard sensors 230, (multiple) other vehicles 250, and / or (multiple) smart infrastructure systems 240. In some embodiments, different types of sensor data are associated and / or integrated by the sensor fusion subsystem 222. For example, the sensor fusion subsystem 222 can use multiple images or videos captured by multiple cameras set at different locations of the vehicle to generate a 360-degree model. The sensor fusion subsystem 222 obtains sensor data from different types of sensors and uses the combined data to more accurately perceive the environment. For example, the onboard camera 232 may not be able to capture a clear image because it is directly facing the sun or a light source (e.g., the headlights of another vehicle at night). The LiDAR system 210 may not be affected too much, and therefore the sensor fusion subsystem 222 can combine the sensor data provided by the camera 232 and the LiDAR system 210, and use the sensor data provided by the LiDAR system 210 to compensate for the unclear image captured by the camera 232. As another example, in rainy or foggy weather, radar sensor 234 may work better than camera 232 or LiDAR system 210. Therefore, sensor fusion subsystem 222 may use sensor data provided by radar sensor 234 to compensate for sensor data provided by camera 232 or LiDAR system 210.

[0056] In other examples, sensor data generated by other onboard sensors (multiple) 230 may have a lower resolution (e.g., radar sensor data), and therefore may need to be correlated and confirmed by the LiDAR system (multiple) 210, which typically has a higher resolution. For example, a manhole cover (also known as a manhole cover) may be detected by the radar sensor 234 as an object that the vehicle is approaching. Due to the low resolution nature of the radar sensor 234, the vehicle perception and planning system 220 may not be able to determine whether the object is an obstacle that the vehicle needs to avoid. Therefore, the high resolution sensor data generated by the LiDAR system (multiple) 210 can be used to correlate and confirm that the object is a manhole cover and will not cause damage to the vehicle.

[0057] The vehicle perception and planning system 220 also includes an object classifier 223. Using the raw sensor data and / or the associated / fused data provided by the sensor fusion subsystem 222, the object classifier 223 can detect and classify objects and assess the location of the objects. In some embodiments, the object classifier 233 can use machine learning-based techniques to detect and classify objects. Examples of machine learning-based techniques include utilizing algorithms such as region-based convolutional neural networks (R-CNN), fast R-CNN, faster R-CNN, histogram of oriented gradients (HOG), region-based fully convolutional networks (R-FCN), single shot detectors (SSD), spatial pyramid pooling (SPP-net), and / or You Only Look Once (Yolo).

[0058] The vehicle perception and planning system 220 also includes a road detection subsystem 224. The road detection subsystem 224 locates the road and identifies objects and / or markings on the road. For example, based on raw or fused sensor data provided by (multiple) radar sensors 234, (multiple) cameras 232, and / or (multiple) LiDAR systems 210, the road detection subsystem 224 can build a 3D model of the road based on machine learning techniques (e.g., a pattern recognition algorithm for identifying lanes). Using the 3D model of the road, the road detection subsystem 224 can identify objects (e.g., obstacles or debris on the road) and / or markings on the road (e.g., lane lines, turn signs, crosswalk signs, etc.).

[0059] The vehicle perception and planning system 220 also includes a positioning and vehicle attitude subsystem 225. Based on raw or fused sensor data, the positioning and vehicle attitude subsystem 225 can determine the position of the vehicle and the attitude of the vehicle. For example, using sensor data from (multiple) LiDAR systems 210, (multiple) cameras 232, and / or GPS data, the positioning and vehicle attitude subsystem 225 can determine the exact position of the vehicle on the road and the six degrees of freedom of the vehicle (e.g., whether the vehicle is moving forward or backward, up or down, left or right). In some embodiments, a high-definition (HD) map is used for vehicle positioning. The HD map can provide a very detailed three-dimensional computerized map that accurately locates the location of the vehicle. For example, using the HD map, the positioning and vehicle attitude subsystem 225 can accurately determine the current position of the vehicle (e.g., which lane of the road the vehicle is currently in, how close it is to the curb or sidewalk) and predict the future position of the vehicle.

[0060] The vehicle perception and planning system 220 also includes an obstacle predictor 226. The objects identified by the object classifier 223 can be stationary (e.g., light poles, road signs) or dynamic (e.g., moving pedestrians, bicycles, another car). For moving objects, predicting their moving paths or future positions is important for avoiding collisions. The obstacle predictor 226 can predict obstacle trajectories and / or warn the driver or vehicle planning subsystem 228 about potential collisions. For example, if there is a high probability that the trajectory of the obstacle intersects the current moving path of the vehicle, the obstacle predictor 226 can generate such a warning. The obstacle predictor 226 can use various techniques to make such predictions. These techniques include, for example, constant speed or acceleration models, constant rotation rate and speed / acceleration models, models based on Kalman filters and extended Kalman filters, models based on recursive neural networks (RNNs), models based on long short-term memory (LSTM) neural networks, encoder-decoder RNN models, etc.

[0061] Still refer to Figure 2 In some embodiments, the vehicle perception and planning system 220 also includes a vehicle planning subsystem 228. The vehicle planning subsystem 228 may include a route planner, a driving behavior planner, and a mobility planner. The route planner may plan the vehicle's route based on the vehicle's current position data, target position data, traffic information, and the like. The driving behavior planner uses the obstacle prediction results provided by the obstacle predictor 226 to adjust the timing and planned movement based on how other objects may move. The mobility planner determines the specific operations that the vehicle needs to follow. The planning results are then communicated to the vehicle control system 280 via the vehicle interface 270. Communications may be performed via communication paths 223 and 271, which may include any wired or wireless communication links that may transmit data.

[0062] The vehicle control system 280 controls the vehicle's steering mechanism, throttle, brakes, etc. to operate the vehicle according to the planned route and movement. The vehicle perception and planning system 220 may also include a user interface 260 that provides a user (e.g., a driver) with access to the vehicle control system 280, for example to manipulate or take over control of the vehicle when necessary. The user interface 260 can communicate with the vehicle perception and planning system 220, for example, to obtain and display raw or fused sensor data, identified objects, the vehicle's position / posture, etc. These displayed data can help the user better operate the vehicle. The user interface 260 can communicate with the vehicle perception and planning system 220 and / or the vehicle control system 280 via communication paths 221 and 261, respectively, and this communication path includes any wired or wireless communication link that can transmit data. It should be understood that Figure 2The various systems, sensors, communication links and interfaces in the can be configured in any desired manner and are not limited to Figure 2 The configuration shown in .

[0063] Figure 3 A block diagram of an exemplary LiDAR system 300 is shown. The LiDAR system 300 may be used to implement Figure 1 and Figure 2 1 . The LiDAR system 300 shown in FIG. 1 includes a laser source 310, a transmitter 320, an optical receiver and light detector 330, a steering system 340, and a control circuit system 350. These components are coupled together using communication paths 312, 314, 322, 332, 343, 352, and 362. These communication paths include communication links (wired or wireless, bidirectional or unidirectional) between various LiDAR system components, but do not have to be physical components themselves. Although the communication paths can be implemented by one or more wires, buses, or optical fibers, the communication paths can also be wireless channels or free space optical paths, so that there is no physical communication medium. For example, in one embodiment of the LiDAR system 300, the communication path 314 between the laser source 310 and the transmitter 320 can be implemented using one or more optical fibers. Communication paths 332 and 352 can represent optical paths implemented using free space optical components and / or optical fibers. And communication paths 312, 322, 342, and 362 can be implemented using one or more wires carrying electrical signals. The communication paths can also include one or more of the above-mentioned types of communication media (for example, they can include optical fibers and free space optical components, or include one or more optical fibers and one or more wires).

[0064] The LiDAR system 300 may also include Figure 3 Other components not depicted in the diagram may include power buses, power supplies, LED indicators, switches, etc. Additionally, there may be other communication connections between components, such as a direct connection between the light source 310 and the optical receiver and light detector 330 to provide a reference signal so that the time from the transmission of a light pulse to the detection of a return light pulse can be accurately measured.

[0065] The laser source 310 outputs a laser for illuminating an object in the field of view (FOV). The laser source 310 may be, for example, a semiconductor-based laser (e.g., a diode laser) and / or a fiber-based laser. A semiconductor-based laser may be, for example, an edge-emitting laser (EEL), a vertical cavity surface-emitting laser (VCSEL), etc. A fiber-based laser is a laser in which the active gain medium is an optical fiber doped with a rare earth element (such as erbium, ytterbium, neodymium, dysprosium, praseodymium, thulium, and / or holmium). In some embodiments, the fiber laser is based on a double-clad fiber, in which the gain medium forms the core of the optical fiber surrounded by two layers of cladding. Double-clad fiber allows the core to be pumped with a high-power beam, thereby making the laser source a high-power fiber laser source.

[0066] In some embodiments, the laser source 310 includes a master oscillator (also called a seed laser) and a power amplifier (MOPA). The power amplifier amplifies the output power of the seed laser. The power amplifier can be a fiber amplifier, a bulk amplifier, or a semiconductor optical amplifier. The seed laser can be a diode laser (e.g., a Fabry-Perot cavity laser, a distributed feedback laser), a solid-state bulk laser, or a tunable external cavity diode laser. In some embodiments, the laser source 310 can be an optically pumped microchip laser. A microchip laser is an alignment-free monolithic solid-state laser in which the laser crystal is directly in contact with the end mirror of the laser resonator. Microchip lasers are typically pumped with a laser diode (directly or using an optical fiber) to obtain the desired output power. Microchip lasers can be based on neodymium-doped yttrium aluminum garnet (Y3Al5O 12 ) laser crystal (ie, Nd:YAG), or neodymium-doped vanadate (ie, Nd:YVO4) laser crystal.

[0067] Figure 4 A block diagram of an exemplary fiber-based laser source 400 is shown having a seed laser and one or more pumps (eg, laser diodes) for pumping a desired output power. The fiber-based laser source 400 is Figure 3. In some embodiments, the fiber-based laser source 400 includes a seed laser 402 to generate initial optical pulses of one or more wavelengths (e.g., 1550 nm), which are provided to a wavelength division multiplexer (WDM) 404 via an optical fiber 403. The fiber-based laser source 400 also includes a pump 406 for providing laser power (e.g., of a different wavelength, such as 980 nm) to the WDM 404 via an optical fiber 405. The WDM 404 multiplexes the optical pulses provided by the seed laser 402 and the laser power provided by the pump 406 onto a single optical fiber 407. The output of the WDM 404 can then be provided to one or more preamplifiers 408 via the optical fiber 407. The preamplifier(s) 408 can be an optical amplifier(s) that amplifies the optical signal (e.g., having a gain of about 20 to 30 dB). In some embodiments, the preamplifier(s) 408 are low noise amplifiers. The preamplifier(s) 408 output to the combiner 410 via an optical fiber 409. Combiner 410 combines the output laser of preamplifier(s) 408 with the laser power provided by pump 412 via optical fiber 411. Combiner 410 can combine optical signals having the same wavelength or different wavelengths. An example of a combiner is WDM. Combiner 410 provides pulses to boost amplifier 414, which produces output optical pulses via optical fiber 410. Boost amplifier 414 provides further amplification of the optical signal. The output optical pulses can then be transmitted to transmitter 320 and / or steering mechanism 340 ( Figure 3 It should be understood that Figure 4 An exemplary configuration of a fiber-based laser source 400 is shown. Figure 4 One or more components shown and / or Figure 4 Laser source 400 can have many other configurations by using different combinations of other components not shown (e.g., other components such as power supplies, lenses, filters, beam splitters, combiners, etc.).

[0068] In some variations, the fiber gain curve of the optical fiber used in the fiber-based laser source 400 can control the fiber-based laser source 400 (e.g., via the control circuit system 350) to generate pulses of different amplitudes. The communication path 312 couples the fiber-based laser source 400 to the control circuit system 350 ( Figure 3350) so that the components of the fiber-based laser source 400 can be controlled by or communicate with the control circuit system 350. Optionally, the fiber-based laser source 400 can include its own dedicated controller. Compared to the control circuit system 350 directly communicating with the components of the fiber-based laser source 400, the dedicated controller of the fiber-based laser source 400 can communicate with the control circuit system 350 and control and / or communicate with the components of the fiber-based light source 400. The fiber-based light source 400 may also include other components not shown, such as one or more power connectors, power supplies, and / or power lines.

[0069] refer to Figure 3 , typical operating wavelengths of the laser source 310 include, for example, about 850nm, about 905nm, about 940nm, about 1064nm, and about 1550nm. The upper limit of the maximum available laser power is set by USFDA (U.S. Food and Drug Administration) rules. The optical power limit at the 1550nm wavelength is much higher than the optical power limits of the other aforementioned wavelengths. In addition, at 1550nm, the optical power loss in the optical fiber is low. These characteristics of the 1550nm wavelength make it more conducive to long-range LiDAR applications. The amount of optical power output from the laser source 310 can be characterized by its peak power, average power, and pulse energy. Peak power is the ratio of pulse energy to the width of the pulse (e.g., full width at half maximum or FWHM). Therefore, for a fixed amount of pulse energy, a smaller pulse width can provide a larger peak power. The pulse width can be in the range of nanoseconds or picoseconds. The average power is the product of the energy of the pulse and the pulse repetition rate (PRR). As described in more detail below, PRR represents the frequency of the pulsed laser. The PRR generally corresponds to the maximum range that the LiDAR system can measure. The laser source 310 can be configured to generate pulses at a high PRR to meet the desired number of data points in the point cloud generated by the LiDAR system. The laser source 310 can also be configured to generate pulses at a medium or low PRR to meet the desired maximum detection distance. Wall plug efficiency (WPE) is another factor in evaluating the total power consumption, which can be a key indicator for evaluating the efficiency of the laser. For example, Figure 1 As shown, multiple LiDAR systems may be attached to a vehicle, which may be an electric vehicle or otherwise have a limited fuel or battery power source. Therefore, high WPE and an intelligent way of using laser power are often important considerations when selecting and configuring a laser source 310 and / or designing a laser delivery system for a vehicle-mounted LiDAR application.

[0070] It should be understood that the above description provides non-limiting examples of laser source 310. Laser source 310 can be configured to include many other types of light sources (e.g., laser diodes, short cavity fiber lasers, solid-state lasers, and / or tunable external cavity diode lasers) configured to generate one or more optical signals at various wavelengths. In some examples, light source 310 includes an amplifier (e.g., a preamplifier and / or a boost amplifier), which can be a doped fiber amplifier, a solid-state bulk amplifier, and / or a semiconductor optical amplifier. The amplifier is configured to receive and amplify the optical signal with a desired gain.

[0071] Reference again Figure 3 , the LiDAR system 300 also includes a transmitter 320. The laser source 310 provides laser light (e.g., in the form of a laser beam) to the transmitter 320. The laser light provided by the laser source 310 can be an amplified laser with a predetermined or controlled wavelength, pulse repetition rate, and / or power level. The transmitter 320 receives the laser light from the laser source 310 and transmits the laser light to the steering mechanism 340 with a low divergence. In some embodiments, the transmitter 320 may include, for example, optical components (e.g., lenses, optical fibers, mirrors, etc.) for transmitting the laser beam to the field of view (FOV) directly or via the steering mechanism 340. Although Figure 3 The transport 320 and the steering mechanism 340 are illustrated as separate components, but in some embodiments, they may be combined or integrated into a system. The steering mechanism 340 is described in more detail below.

[0072] The laser beam provided by the laser source 310 may diverge as it travels to the transmitter 320. Therefore, the transmitter 320 typically includes a collimating lens that is configured to collect the diverging laser beam and produce a more parallel beam with reduced or minimal divergence. The collimated beam can then be further directed by various optical devices such as mirrors and lenses. The collimating lens can be, for example, a single plano-convex lens or a lens array. The collimating lens can be configured to achieve any desired characteristics, such as beam diameter, divergence, numerical aperture, focal length, etc. The beam spread ratio or beam quality factor (also known as M 2 The factor M is used to measure the quality of the laser beam. In many LiDAR applications, it is important to have good laser beam quality in the generated transmission laser beam. 2 The factor represents the degree of change of the beam relative to the ideal Gaussian beam. Therefore, M 2 The factor reflects how well a collimated laser beam can be focused on a small spot, or how well a divergent laser beam can be collimated. Thus, the laser source 310 and / or the transmitter 320 can be configured to meet, for example, scanning resolution requirements while maintaining a desired M 2 factor.

[0073] One or more of the light beams provided by the transmitter 320 are scanned to the FOV by the steering mechanism 340. The steering mechanism 340 scans the light beams in multiple dimensions (e.g., in both the horizontal and vertical dimensions) so that the LiDAR system 300 can map the environment by generating a 3D point cloud. The steering mechanism 340 will be described in more detail below. The laser light scanned to the FOV may be scattered or reflected by objects in the FOV. At least a portion of the scattered light or reflected light is returned to the LiDAR system 300. Figure 3 Also shown is an optical receiver and light detector 330 configured to receive the return light. The optical receiver and light detector 330 include an optical receiver configured to collect the return light from the FOV. The optical receiver may include optics (e.g., lenses, optical fibers, mirrors, etc.) for receiving, redirecting, focusing, amplifying and / or filtering the return light from the FOV. For example, the optical receiver typically includes a focusing lens (e.g., a single plano-convex lens or a lens group) to collect and / or focus the collected return light onto the light detector.

[0074] The photodetector detects the return light focused by the optical receiver and generates a current and / or voltage signal proportional to the incident intensity of the return light. Based on such current and / or voltage signals, the depth information of the object in the FOV can be derived. An exemplary method for deriving such depth information is based on direct TOF (time of flight), which will be described in more detail below. The photodetector can be characterized by its detection sensitivity, quantum efficiency, detector bandwidth, linearity, signal-to-noise ratio (SNR), overload resistance, interference resistance, etc. Based on the application, the photodetector can be configured or customized to have any desired characteristics. For example, the optical receiver and the photodetector 330 can be configured so that the photodetector has a large dynamic range while having good linearity. The linearity of the photodetector indicates the ability of the detector to maintain a linear relationship between the input optical signal power and the output of the detector. A detector with good linearity can maintain a linear relationship over a large dynamic input optical signal range.

[0075] To achieve the desired detector characteristics, the structure of the photodetector and / or the material system of the detector can be configured or customized. Various detector structures can be used for the photodetector. For example, the photodetector structure can be a PIN-based structure having an undoped intrinsic semiconductor region (i.e., an "I" region) between the p-type semiconductor and n-type semiconductor regions. Other photodetector structures include, for example, APD (avalanche photodiode)-based structures, PMT (photomultiplier tube)-based structures, SiPM (silicon photomultiplier tube)-based structures, SPAD (single photon avalanche diode)-based structures, and / or quantum wires. For the material system used in the photodetector, Si, InGaAs and / or Si / Ge-based materials can be used. It should be understood that many other detector structures and / or material systems can be used in the optical receiver and photodetector 330.

[0076] A photodetector (e.g., an APD-based detector) may have an internal gain so that the input signal is amplified when generating an output signal. However, due to the internal gain of the photodetector, noise may also be amplified. Common types of noise include signal shot noise, dark current shot noise, thermal noise, and amplifier noise (TIA). In some embodiments, the optical receiver and photodetector 330 may include a preamplifier as a low noise amplifier (LNA). In some embodiments, the preamplifier may also include a TIA transimpedance amplifier that converts the current signal into a voltage signal. For linear detector systems, the input equivalent noise or noise equivalent power (NEP) measures the sensitivity of the photodetector to weak signals. Therefore, they can be used as an indicator of overall system performance. For example, the NEP of the photodetector indicates the power of the weakest signal that can be detected, and therefore it in turn indicates the maximum range of the LiDAR system. It should be understood that various photodetector optimization techniques can be used to meet the requirements of the LiDAR system 300. Such optimization techniques may include selecting different detector structures, materials, and / or implementing signal processing techniques (e.g., filtering, noise reduction, amplification, etc.). For example, in addition to or instead of direct detection using a return signal (e.g., by using TOF), coherent detection can also be used for a photodetector. Coherent detection allows the amplitude and phase information of the received light to be detected by interfering with the received light using a local oscillator. Coherent detection can improve detection sensitivity and noise immunity.

[0077] Figure 3The LiDAR system 300 is also shown to include a steering mechanism 340. As described above, the steering mechanism 340 guides the light beam from the transmitter 320 to scan the FOV in multiple dimensions. The steering mechanism is called a raster mechanism or a scanning mechanism. Scanning the light beam in multiple directions (for example, in both the horizontal and vertical directions) helps the LiDAR system to map the environment by generating an image or a 3D point cloud. The steering mechanism can be based on mechanical scanning and / or solid-state scanning. Mechanical scanning uses a rotating mirror to steer the laser beam or physically rotates the LiDAR transmitter and receiver (collectively referred to as a transceiver) to scan the laser beam. Solid-state scanning guides the laser beam through various positions of the FOV without mechanically moving any macroscopic components, such as a transceiver. Solid-state scanning mechanisms include, for example, steering based on an optical phased array and steering based on flash LiDAR. In some embodiments, since the solid-state scanning mechanism does not physically move the macroscopic components, the steering performed by the solid-state scanning mechanism can be referred to as effective steering. A LiDAR system using solid-state scanning can also be referred to as non-mechanical scanning or simply as a non-scanning LiDAR system (a flash LiDAR system is an exemplary non-scanning LiDAR system).

[0078] The steering mechanism 340 can be used with a transceiver (e.g., a transmitter 320 and an optical receiver and light detector 330) to scan the FOV to generate an image or a 3D point cloud. As an example, to implement the steering mechanism 340, a two-dimensional mechanical scanner can be used with a single-point transceiver or multiple single-point transceivers. The single-point transceiver transmits a single beam or a small number of beams (e.g., 2 to 8 beams) to the steering mechanism. The two-dimensional mechanical steering mechanism includes, for example, (multiple) polygonal mirrors, (multiple) oscillating mirrors, (multiple) rotating prisms, (multiple) rotating tilt mirrors, or a combination thereof. In some embodiments, the steering mechanism 340 may include (multiple) non-mechanical steering mechanisms, such as (multiple) solid-state steering mechanisms. For example, the steering mechanism 340 can be based on the tuned wavelength of a laser combined with a refraction effect, and / or based on a reconfigurable grating / phase array. In some embodiments, the steering mechanism 340 can use a single scanning device to achieve two-dimensional scanning, or use a combination of two devices to achieve two-dimensional scanning.

[0079] As another example, to implement the steering mechanism 340, a one-dimensional mechanical scanner can be used with an array or a large number of single-point transceivers. Specifically, the transceiver array can be mounted on a rotating platform to achieve a 360-degree horizontal field of view. Optionally, a static transceiver array can be combined with a one-dimensional mechanical scanner. The one-dimensional mechanical scanner includes (multiple) polygonal mirrors, (multiple) oscillating mirrors, (multiple) rotating prisms, (multiple) rotating tilt mirrors for obtaining a forward horizontal field of view. A steering mechanism using a mechanical scanner can provide robustness and reliability in terms of mass production for automotive applications.

[0080] As another example, to implement the steering mechanism 340, a two-dimensional transceiver can be used to directly generate a scanned image or a 3D point cloud. In some embodiments, a stitching or micro-shifting method can be used to improve the resolution of the scanned image or the scanned field of view. For example, using a two-dimensional transceiver, a signal generated in one direction (e.g., horizontal direction) and a signal generated in another direction (e.g., vertical direction) can be integrated, interleaved and / or matched to generate a higher or full resolution image or 3D point cloud representing the scanned FOV.

[0081] Some embodiments of the steering mechanism 340 include one or more optical redirecting elements (e.g., mirrors or lenses) that redirect the return light signal along the receive path (e.g., by rotation, vibration, or guidance) to direct the return light signal to the optical receiver and light detector 330. The optical redirecting elements that direct the light signal along the transmit and receive paths can be the same components (e.g., shared), separate components (e.g., dedicated), and / or a combination of shared and separate components. This means that in some cases, the transmit and receive paths are distinct, although they may partially overlap (or, in some cases, substantially overlap).

[0082] Still refer to Figure 3 , the LiDAR system 300 also includes a control circuit system 350. The control circuit system 350 can be configured and / or programmed to control various parts of the LiDAR system 300 and / or perform signal processing. In a typical system, the control circuit system 350 can be configured and / or programmed to perform one or more control operations, including, for example, controlling the laser source 310 to obtain the desired laser pulse timing, repetition rate, and power; controlling the steering mechanism 340 (e.g., controlling speed, direction, and / or other parameters) to scan the FOV and maintain pixel registration / alignment; controlling the optical receiver and light detector 330 (e.g., controlling sensitivity, noise reduction, filtering, and / or other parameters) so that it is in an optimal state; and monitoring the overall system health / functional safety status.

[0083] The control circuit system 350 can also be configured and / or programmed to perform signal processing on the raw data generated by the optical receiver and light detector 330 to derive distance and reflectivity information, and to package and communicate the data with the vehicle perception and planning system 220 (in Figure 2 For example, control circuitry 350 determines the time taken from transmitting a light pulse to receiving a corresponding return light pulse; determines when a return light pulse is not received for a transmitted light pulse; determines the direction (e.g., horizontal and / or vertical information) of a transmitted / return light pulse; determines an estimated range in a particular direction; and / or determines any other type of data related to LiDAR system 300.

[0084] The LiDAR system 300 can be set in a vehicle, which can operate in a variety of different environments, including hot or cold weather, rough road conditions that may cause strong vibrations, high humidity or low humidity, dusty areas, etc. Therefore, in some embodiments, the optical and / or electronic components of the LiDAR system 300 (e.g., the optical receiver and light detector 330, the steering mechanism 340, and the optical devices in the transmitter 320) are arranged or configured in a manner to maintain long-term mechanical and optical stability. For example, the components in the LiDAR system 300 can be fixed and sealed so that they can operate in all conditions that the vehicle may encounter. For example, a moisture-proof coating and / or an airtight seal can be applied to the optical components of the optical receiver and light detector 330, the steering mechanism 340, and the transmitter 320 (as well as other components that are susceptible to moisture). As another example, (multiple) housings, (multiple) covers, and / or windows can be used in the LiDAR system 300 to provide desired properties, such as hardness, protection level (IP), self-cleaning ability, chemical resistance, and impact resistance. Furthermore, an efficient and economical method for assembling the LiDAR system 300 may be used to meet LiDAR operating requirements while keeping costs low.

[0085] It should be understood by those skilled in the art that Figure 3 The above description is for illustrative purposes only, and the LiDAR system may include other functional units, blocks, or segments, and may include variations or combinations of the above functional units, blocks, or segments. For example, the LiDAR system 300 may also include Figure 3 Other components not depicted in the diagram may include a power bus, power supplies, LED indicators, switches, etc. Additionally, there may be other connections between components, such as a direct connection between the light source 310 and the optical receiver and light detector 330 so that the light detector 330 can accurately measure the time from when a light pulse is transmitted from the light source 310 until the light detector 330 detects a return light pulse.

[0086] Figure 3The components shown in FIG. 1 are coupled together using communication paths 312, 314, 322, 332, 342, 352, and 362. These communication paths represent communications (bidirectional or unidirectional) between various LiDAR system components, but are not necessarily the physical components themselves. Although communication paths can be implemented by one or more wires, buses, or optical fibers, communication paths can also be wireless channels or open-air optical paths such that no physical communication medium is present. For example, in one exemplary LiDAR system, communication path 314 includes one or more optical fibers; communication path 352 represents an optical path; and communication paths 312, 322, 342, and 362 are all wires that carry electrical signals. Communication paths can also include more than one of the above-mentioned types of communication media (e.g., they can include optical fibers and optical paths, or one or more optical fibers and one or more wires).

[0087] As mentioned above, some LiDAR systems use the time of flight (TOF) of a light signal (e.g., a light pulse) to determine the distance to an object in the light path. Figure 5A , an exemplary LiDAR system 500 includes a laser light source (e.g., a fiber laser), a steering system (e.g., a system of one or more moving mirrors), and a light detector (e.g., a photon detector having one or more optical devices). The LiDAR system 500 can be implemented using, for example, the LiDAR system 300 described above. The LiDAR system 500 emits light pulses 502 along an optical path 504 determined by the steering system of the LiDAR system 500. In the depicted example, the light pulses 502 generated by the laser light source are short pulses of laser light. In addition, the signal steering system of the LiDAR system 500 is a pulsed signal steering system. However, it should be understood that the LiDAR system can operate by generating, transmitting, and detecting non-pulsed light signals, and using techniques other than time of flight to derive the distance to objects in the surrounding environment. For example, some LiDAR systems use frequency modulated continuous waves (i.e., FMCW). It should also be understood that any of the techniques described herein for time-of-flight-based systems using pulsed signals can also be applied to LiDAR systems that do not use one or both of these techniques.

[0088] Reference again Figure 5A(e.g., a time-of-flight LiDAR system using light pulses is shown), when light pulse 502 reaches object 506, light pulse 502 is scattered or reflected to generate return light pulse 508. Return light pulse 508 may return to system 500 along optical path 510. The time from when the transmitted light pulse 502 leaves LiDAR system 500 to when the return light pulse 508 returns to arrive at LiDAR system 500 may be measured (e.g., by a processor or other electronic device within the LiDAR system, such as control circuitry 350). This time of flight, combined with knowledge of the speed of light, may be used to determine the range / distance from LiDAR system 500 to the portion of object 506 where light pulse 502 was scattered or reflected.

[0089] like Figure 5B As depicted, LiDAR system 500 scans the external environment by directing a number of light pulses (e.g., by directing light pulses 502, 522, 526, 530 along light paths 504, 524, 528, 532, respectively). Figure 5C As depicted, the LiDAR system 500 receives return light pulses 508, 542, 548 (which correspond to the transmitted light pulses 502, 522, 530, respectively). The return light pulses 508, 542, and 548 are generated by scattering or reflecting the transmitted light pulses from one of the objects 506 and 514. The return light pulses 508, 542, and 548 can be returned to the LiDAR system 500 along optical paths 510, 544, and 546, respectively. Based on the direction of the transmitted light pulses (as determined by the LiDAR system 500) and the calculated distance from the LiDAR system 500 to the portion of the object that scattered or reflected the light pulses (e.g., the portion of objects 506 and 514), the external environment within the detectable range (e.g., including the field of view between the paths 504 and 532) can be accurately mapped or plotted (e.g., by generating a 3D point cloud or image).

[0090] If for a particular transmitted light pulse, a corresponding light pulse is not received, it can be determined that there is no object within the detectable range of the LiDAR system 500 (e.g., the object is beyond the maximum scanning distance of the LiDAR system 500). Figure 5B In the example, light pulse 526 may not have a corresponding return light pulse (e.g. Figure 5C 528 within the detectable range of the LiDAR system 500).

[0091] exist Figure 5B, light pulses 502, 522, 526, and 530 may be transmitted in any order, serially, in parallel, or based on other timing relative to each other. Additionally, although Figure 5B The transmitted light pulses are depicted as being directed in one dimension or plane (e.g., the plane of the paper), but the LiDAR system 500 may also direct the transmitted light pulses along other dimensions or planes. For example, the LiDAR system 500 may also direct the transmitted light pulses perpendicular to the plane of the paper. Figure 5B The transmitted light pulses are guided in the dimensions or planes of the dimensions or planes shown in , thereby forming a two-dimensional transmission of the light pulses. This two-dimensional transmission of the light pulses can be performed point by point, line by line, simultaneously, or in other ways. A point cloud or image (e.g., a single horizontal line) from a one-dimensional transmission of the light pulses can generate two-dimensional data (e.g., (1) data from the horizontal transmission direction and (2) the range or distance to the object). Similarly, a point cloud or image from a two-dimensional transmission of the light pulses can generate three-dimensional data (e.g., (1) data from the horizontal transmission direction, (2) data from the vertical transmission direction, and (3) the range or distance to the object). Typically, a LiDAR system that performs n-dimensional transmission of light pulses generates (n+1)-dimensional data. This is because the LiDAR system can measure the depth of an object or the range / distance to an object, which provides an additional dimension of data. Therefore, a 2D scan performed by a LiDAR system can generate a 3D point cloud for mapping the external environment of the LiDAR system.

[0092] The density of the point cloud refers to the number of measurements (data points) per area performed by the LiDAR system. The point cloud density is related to the LiDAR scan resolution. In general, at least for the region of interest (ROI), a greater point cloud density is desired, and therefore a higher resolution is desired. The point density in the point cloud or image generated by the LiDAR system is equal to the number of pulses divided by the field of view. In some embodiments, the field of view may be fixed. Therefore, in order to increase the density of points generated by a set of transmit-receive optical devices (or transceiver optical devices), the LiDAR system may need to generate pulses more frequently. In other words, a light source with a higher pulse repetition rate (PRR) is required. On the other hand, by generating and transmitting pulses more frequently, the farthest distance that the LiDAR system can detect may be limited. For example, if a return signal from a distant object is received after the system transmits the next pulse, the return signal may be detected in a different order from the order in which the corresponding signal was transmitted, so if the system cannot correctly associate the return signal with the transmitted signal, it will cause ambiguity.

[0093] To illustrate, consider an exemplary LiDAR system that can transmit laser pulses with a repetition rate between 500kHz and 1MHz. Based on the time it takes for the pulses to return to the LiDAR system, and to avoid confusion of the return pulses from continuous pulses in conventional LiDAR designs, the farthest distances that the LiDAR system can detect can be 300 meters and 150 meters for 500kHz and 1MHz, respectively. The point density of a LiDAR system with a repetition rate of 500kHz is half the point density of a LiDAR system with a repetition rate of 1MHz. Therefore, this example shows that if the system cannot properly correlate return signals that arrive out of order, then increasing the repetition rate from 500kHz to 1MHz (and therefore increasing the point density of the system) may reduce the detection range of the system. Various techniques are used to mitigate the tradeoff between higher PRR and limited detection range. For example, multiple wavelengths can be used to detect objects at different ranges. Optical and / or signal processing techniques are also used to correlate between the transmitted optical signal and the return optical signal.

[0094] The various systems, devices, and methods described herein may be implemented using digital circuitry, or using one or more computers using known computer processors, memory units, storage devices, computer software, and other components. Typically, a computer includes a processor for executing instructions and one or more memories for storing instructions and data. A computer may also include or be coupled to one or more mass storage devices, such as one or more disks, internal hard disks and removable disks, magnetic disks, optical disks, and the like.

[0095] The various systems, devices, and methods described herein can be implemented using computers operating in a client-server relationship. Typically, in such a system, a client computer is located remotely from a server computer and interacts via a network. The client-server relationship can be defined and controlled by computer programs running on the respective client and server computers. Examples of client computers can include desktop computers, workstations, portable computers, cellular smartphones, tablet computers, or other types of computing devices.

[0096] The various systems, apparatuses, and methods described herein may be implemented using a computer program product tangibly embodied in an information carrier (e.g., in a non-transitory machine-readable storage device) so as to be executed by a programmable processor; and the method procedures and steps described herein (including Fig.13One or more of the steps of the process may be implemented using one or more computer programs executable by such a processor. A computer program is a set of computer program instructions that can be used directly or indirectly in a computer to perform a specific activity or produce a specific result. A computer program may be written in any form of programming language, including compiled or interpreted languages, and it may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0097] A high-level block diagram of an exemplary apparatus that can be used to implement the systems, apparatus, and methods described herein is provided in Figure 6 6. The apparatus 600 includes a processor 610 operably coupled to a permanent storage device 620 and a main memory device 630. The processor 610 controls the overall operation of the apparatus 600 by executing computer program instructions that define these operations. The computer program instructions may be stored in the permanent storage device 620 or other computer-readable medium and loaded into the main memory device 630 when it is desired to execute the computer program instructions. For example, the processor 610 may be used to implement one or more components and systems described herein, such as the control circuit system 350 ( Figure 3 ), vehicle perception and planning system 220 ( Figure 2 ) and the vehicle control system 280 ( Figure 2 Therefore, Fig.13 One or more of the method steps may be defined by computer program instructions stored in the main memory device 630 and / or the permanent storage device 620 and controlled by the processor 610 executing the computer program instructions. For example, the computer program instructions may be implemented as a computer executable code programmed by a person skilled in the art to execute the method executed by the processor 610. Fig.13 Thus, by executing the computer program instructions, the processor 610 performs the Figure 3 To Figure 5 and Fig.13 The device 600 may further include one or more network interfaces 680 for communicating with other devices via a network. The device 600 may further include one or more input / output devices 690 that enable a user to interact with the device 600 (e.g., a display, keyboard, mouse, speaker, buttons, etc.).

[0098] The processor 610 may include general and special purpose microprocessors and may be the sole processor of the device 600 or one of multiple processors. The processor 610 may include one or more central processing units (CPUs) and one or more graphics processing units (GPUs), which may, for example, work independently and / or perform multitasking with one or more CPUs to accelerate processing, such as for various image processing applications described herein. The processor 610, the permanent storage device 620, and / or the main memory device 630 may include, or be supplemented or incorporated with, one or more application specific integrated circuits (ASICs) and / or one or more field programmable gate arrays (FPGAs).

[0099] The permanent storage device 620 and the main memory device 630 each include a tangible non-transitory computer-readable storage medium. The permanent storage device 620 and the main memory device 630 may each include a high-speed random access memory, such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a double data rate synchronous dynamic random access memory (DDRRAM), or other random access solid-state storage devices, and may include non-volatile memory, such as one or more disk storage devices, such as internal hard disks and removable disks, magnetic disk storage devices, optical disk storage devices, flash memory devices, semiconductor memory devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), digital versatile disk read-only memory (DVD-ROM) disks, or other non-volatile solid-state storage devices.

[0100] Input / output devices 690 may include peripheral devices such as printers, scanners, display screens, etc. For example, input / output devices 690 may include a display device (such as a cathode ray tube (CRT), plasma or liquid crystal display (LCD)) for displaying information to a user, a keyboard, and a pointing device (such as a mouse or trackball with which a user can provide input to apparatus 600).

[0101] Any or all of the functionality of the systems and devices discussed herein may be performed by the processor 610, and / or incorporated into a device or system, such as the LiDAR system 300. In addition, the LiDAR system 300 and / or device 600 may utilize one or more neural networks or other deep learning techniques performed by the processor 610 or other systems or devices discussed herein.

[0102] Those skilled in the art will recognize that the actual computer or computer system implementation may have other structures and may also include other components, and Figure 6 is a high-level representation of some of the components of such a computer for purposes of illustration.

[0103] Fig. 7A 7 shows a simplified LiDAR system 700 with a multi-faceted light redirecting device 702 and a galvanometer mirror 706 in accordance with some embodiments. Fig. 7A As shown, in addition to the device 702 and the mirror 706, the system 700 may also include a transmitter 712 and a focusing lens 710. The transmitter 712 includes a laser source capable of providing one or more transmitted laser beams 715. The transmitted laser beams 715 are directed to the mirror 706. The mirror 706 is controlled to oscillate around the axis 708 between two predetermined angular positions. The oscillation of the mirror 706 helps to scan the light along one dimension (e.g., the vertical dimension) of the FOV. The mirror 706 reflects the transmitted laser beam 715 to form a transmitted light beam 717, which is directed toward the multi-faceted light redirection device 702.

[0104] In some embodiments, multi-faceted light redirecting device 702 includes a top surface 722, a bottom surface (not shown), and a plurality of light-reflecting facets 724. Side surfaces 724 are disposed between the top and bottom surfaces of device 702 and are sometimes also referred to as facets. Fig. 7A , where the device has a polygonal top and bottom surface (e.g., a pentagonal shape, a hexagonal shape, an octagonal shape, etc.) and a plurality of rectangular shaped facets 724. In some embodiments, the facets 724 are reflective surfaces (e.g., mirrors), and thus the multi-faceted light redirecting device can be a multi-faceted mirror. Fig. 7A As shown, facet 724 reflects transmitted light beam 717 to form transmitted light beam 719, which illuminates the object in FOV 714. Light redirection device 702 is configured to rotate about axis 704. Therefore, each facet of light redirection device 702 reflects light in turn. In the present disclosure, oscillation means continuous back and forth movement in two opposite directions (e.g., clockwise and counterclockwise) within a predetermined angular range (e.g., 40 degrees, 80 degrees, etc.) in a periodic or non-periodic manner. Rotation means continuous movement in only one direction for at least 360 degrees.

[0105] In some embodiments, at any particular time, multiple transmitted light beams 717 are reflected by the same facet of the light redirecting device 702 to form multiple transmitted light beams 719. In some embodiments, multiple transmitted light beams 717 are reflected by different faces of the redirecting device 702. Regardless of whether a single facet or multiple facets are used to reflect the transmitted light beams, the uniformity of the facets affects various aspects of the light redirection capabilities of the light redirecting device 702. For example, differences in non-uniformity between the facets of the light redirecting device 702 may affect the reflection angle of the transmitted light beam 719, which in turn may affect the shape of the scan lines in the resulting LiDAR scan pattern. Differences in non-uniformity and their effects on the scan lines will be described in more detail below.

[0106] As the transmitted light beam 719 propagates to illuminate one or more objects in the FOV 714, at least a portion of the transmitted light beam 719 is reflected or scattered to form return light 729. The return light 729 is redirected (e.g., reflected) by the light steering device 702 to form return light 727, which is directed toward the mirror 706. The return light 727 is redirected (e.g., reflected) by the mirror 706 to form return light 725, which is collected by the condenser lens 710. The condenser lens 710 directs the collected return light to other components (e.g., a light detector) of the LiDAR system 700. Thus, in some embodiments, the light steering device 702 and the mirror 706 are used both to transmit a light beam to illuminate an object in the FOV and to receive the return light and redirect the return light to a receiver of the LiDAR system 700.

[0107] In some embodiments, the return light 729 is formed by multiple transmitted light beams 719 and is reflected by the same facet of the light redirecting device 702 at any particular time. In some embodiments, the return light 729 is reflected by different facets of the light redirecting device 702. Regardless of whether a single facet or multiple facets are used, the uniformity of the facets of the light redirecting device 702 affects the device's ability to receive and redirect return light. For example, differences in non-uniformity between the facets of the light redirecting device 702 may affect the angle of the redirected return light, which in turn may affect the shape of the scan lines in the resulting LiDAR scan pattern.

[0108] U.S. non-provisional patent application Ser. No. 16 / 682,774, filed on Nov. 14, 2018, and entitled “LIDAR SYSTEMS THAT USE A MULTI-FACETMIRROR,” describes the invention in greater detail. Fig. 7A The LiDAR system 700 and steering device 702 are shown in , the contents of which are incorporated herein by reference in their entirety.

[0109] In some embodiments, Fig. 7AThe facet 724 of the light redirecting device 702 shown in FIG. 7 has a tilt angle of 90 degrees. The tilt angle is the angle between the normal direction of the facet and the axis of rotation of the multi-faceted light redirecting device. Thus, for the facets of device 702, the tilt angle is between the direction perpendicular to the facet and the axis 704. Thus, in Fig. 7A In the embodiment shown in FIG. 7 , the tilt angle of facet 724 is 90 degrees. In some embodiments, the tilt angle may not be 90 degrees. Figure 7B Such a multi-faceted light redirecting device 752 is shown. The light redirecting device 752 may be a device having multiple facets 756 ( Figure 7B 752). The steering device 752 can rotate around the line 754. Each facet 756 of the steering device 752 has an inclination angle other than 90 degrees, thereby forming a wedge-shaped facet. Therefore, the cross-section of the light steering device can have a trapezoidal shape. Figure 7C A cross-sectional view of a multi-faceted light redirecting device 762 is shown. Similar to device 752, the facets of light redirecting device 762 have tilt angles other than 90 degrees. Figure 7C As shown, one such tilt angle 765 is illustrated as the angle between the rotation axis 764 and the normal direction 766 of the facet 767. The multi-faceted light redirecting device 764 can rotate about its rotation axis 764. Light redirecting devices with non-90 degree tilt angles are described in more detail in U.S. non-provisional patent application Ser. No. 16 / 242,567, filed on Jan. 8, 2019, and entitled “LIDAR DETECTION SYSTEMS AND METHODS THAT USE MULTI-PLANE MIRRORRS,” the contents of which are incorporated herein by reference in their entirety.

[0110] Figure 7B Also shown is a position encoder 758, which is configured to detect the angular position of the light redirecting device 764. The position encoder 758 can be, for example, a rotary encoder ( Figure 7B ) and / or Hall effect sensors. As described in more detail below, the position encoder can provide real-time position feedback data of the light steering device for further modifying the movement curve of the galvanometer mirror to compensate for position inaccuracies during operation. Similar position encoders can provide real-time position feedback data of the galvanometer mirror for similar purposes.

[0111] Figure 8 An example movement curve 810 of a galvanometer mirror and a corresponding LiDAR scan pattern 820 are shown according to some embodiments. The galvanometer mirror may be, for example, Fig. 7A810. As described above, the galvanometer mirror (e.g., mirror 706) can be configured to oscillate between two angular positions (e.g., -20 degrees to +20 degrees) to facilitate scanning light in one dimension of the FOV (e.g., the vertical dimension). Movement curve 810 includes an example trajectory 815 of the movement of the galvanometer mirror. Trajectory 815 shows the angular position and time relationship of the galvanometer mirror. The angular position of the galvanometer mirror can be predetermined at any specific time position. The galvanometer mirror can be controlled to move to a specific angular position at each specific time position. For example, at time position t0, the mirror is in a first angular position 812. The angular position of the mirror changes with time until the mirror moves to a second angular position 814. In some embodiments, the angular position changes in a linear manner with respect to time. The first and second angular positions can be the end positions of the oscillation. Therefore, when the galvanometer mirror moves to one of the first or second angular positions, the mirror changes to move in the opposite direction. Thus, at time position t1, the mirror moves to the second angular position 814 and then changes its oscillation direction. At time t2, the galvanometer mirror moves back to the first angular position 812 and then changes the oscillation direction again. The movement of the mirror then repeats, as Figure 8 In one embodiment, the movement curve 810 includes: Figure 8 The sawtooth curve shown. The galvanometer mirror can be configured to oscillate periodically or in any desired manner according to the movement curve. It should be understood that the movement curve 810 can have other shapes, such as a symmetrical triangular wave.

[0112] As described above, the oscillation of the galvanometer mirror facilitates scanning of the transmitted light beam in one dimension of the FOV (e.g., the vertical dimension). The multi-faceted light steering device (e.g., a polygonal mirror) rotates to facilitate scanning of the transmitted light beam in another dimension of the FOV (e.g., the horizontal dimension). The resulting LiDAR scan pattern may be, for example, Figure 8820 is shown in FIG. 820. Pattern 820 has a plurality of scan lines 822 distributed from top to bottom. Scan lines 822 are formed by the simultaneous movement of a multi-faceted light redirecting device and a galvanometer mirror. When the galvanometer mirror oscillates to one of its two end angular positions (e.g., the second angular position 814), it retrace or flies back to the other end angular position (e.g., the first angular position 812). The retrace is indicated by the retrace line 824 in pattern 820. Therefore, during the time that the galvanometer mirror moves from the first angular position 812 to the second angular position 814 in one stroke (e.g., from time t0 to t1), the multi-faceted light redirecting device (e.g., a multi-faceted mirror) rotates multiple times to generate a plurality of scan lines (e.g., by using one or more transmitted light beams). Each facet of the light redirecting device contributes to the generation of a portion of the scan line. In addition, because the multi-faceted light redirecting device and the galvanometer mirror move simultaneously (e.g., the light redirecting device rotates while the galvanometer mirror oscillates), the scan lines in pattern 820 generally have a non-zero slope (e.g., the scan lines slope downward from left to right). In other words, the left and right ends of the scan lines in pattern 820 may not be the same in the vertical dimension. The slope of the scan line reflects the movement of the galvanometer mirror scanning the light in the vertical dimension while the light redirecting device rotates to produce the scan line from left to right.

[0113] If all facets of the steering device are identical without any inhomogeneity differences, and if the assembly process is performed perfectly (e.g., without any movement of the motor shaft in the radial direction), the scan lines in the LiDAR scan pattern 820 should be straight lines, such as Figure 8 That is, the scan lines should not be deformed or deviate from their straight-line shape.

[0114] However, in mass production, all facets of the same light redirecting device cannot be produced identically to each other, and the assembly process is not always perfect. Therefore, for a particular light redirecting device, non-uniformity between multiple facets may occur. Fig. 9A multifaceted steering device 902 is shown, which includes facets 902A-D for reflecting or redirecting light. One or more of facets 902A-D may be manufactured in a different and / or non-uniform manner from one another. For example, one or more of facets 902A-D may have a facet angle uniformity difference. The facet angle is sometimes also referred to as the tilt angle of the facet. Therefore, the facet angle is the angle between the normal direction of the facet (e.g., the direction perpendicular to the facet surface) and the rotation axis of the steering device (e.g., axis 904). Ideally, different facets should have the same desired facet angle (e.g., all angles are 45 degrees or all angles are 90 degrees). However, due to large-scale production processes, the facet angle may vary from facet to facet. Therefore, there may be a facet angle uniformity difference (e.g., the facet angle of facet 902A may be 89.5 degrees, the facet angle of facet 902B may be 90 degrees, the facet angle of facet 902C may be 91 degrees, etc.).

[0115] In some embodiments, one or more of facets 902A-D can have facet size uniformity differences. For example, ideally, all facets of the steering device should have the same dimensions (width, length, facet internal angle, etc.). However, due to manufacturing errors and inaccuracies, multiple facets may have different dimensions (e.g., facet 902A has a longer width and / or length than facet 902D, there is an internal angle difference between facets 902B and 902C, etc.).

[0116] In some embodiments, one or more of facets 902A-D may have a surface shape error difference. Surface shape error (sometimes also referred to as surface error or surface shape error) is the deviation of an optical surface from its ideal or perfect shape. For reflective surfaces (e.g., the surface of a facet of a steering device), surface shape errors include one or more of cone errors, radius of curvature errors, area errors, curling, asymmetric local surface deviations, etc. They are typically caused by manufacturing or processing tolerances. Different facets 902A-D may therefore have different surface forms. For example, facet 902B may have a slightly larger curvature (e.g., more convex or more concave) than facet 902C. Therefore, the direction of light reflected or redirected by facet 902B may be slightly different from the direction of facet 902C.

[0117] In some embodiments, one or more of the facets 902A-D can have a difference in surface roughness. For example, one facet 902A can have a rougher surface than another facet 902D. The roughness or irregularity of the surface of the facets can cause changes in the amount of light reflected and / or the direction of reflection. More polished surfaces can generally reflect light better and more accurately according to the desired reflection direction; while rough surfaces can scatter light into different undesirable directions. It should be understood that differences in facet angle uniformity, differences in facet size uniformity, differences in surface shape errors, and differences in surface roughness are examples of many non-uniformities that can occur between facets of a multi-faceted light redirecting device. The light redirecting device can also have other differences between these facets.

[0118] In some embodiments, errors associated with the assembly of the multi-faceted light steering device to the LiDAR scanning system may also occur. For example, as described above, when the multi-faceted light steering device is assembled to the motor used to rotate the device, one or more assembly errors may occur, resulting in differences between devices. One type of assembly error is the relative position error between the motor rotor axis and the stator axis. The motor used to rotate the multi-faceted light steering device includes a rotor and a stator. The multi-faceted light steering device is mounted to the rotor and rotates with the rotor. The stator of the motor does not rotate or move in other ways. The stator of the motor typically includes an electrical winding for generating a magnetic field to rotate the rotor on which the magnet is mounted. Ideally, the stator and the rotor should be concentric. Therefore, the central axis of the stator and the rotational axis of the rotor should be coaxially aligned (e.g., completely overlapped). If there is an assembly error and the stator and rotor are not assembled to have the desired concentricity, the rotational movement of the light steering device may deviate from the center. In turn, this may cause the scan lines of the LiDAR scanning pattern to deviate from their desired shape and / or position.

[0119] As another example of assembly error, relative position error may occur when assembling a motor for rotating the light steering device to a motor mounting fixture. As a result, the motor may not be positioned in the desired position in the motor mounting fixture. This type of position error may cause the light steering device to shift its relative position relative to the galvanometer mirror due to the mounting error of the motor. For example, due to motor mounting error, the light steering device (e.g., Fig. 7A The relative angular relationship, height relationship, horizontal relationship, vertical relationship, relative orientation, etc. between the device 702 in FIG. 1 and the galvanometer mirror (e.g., mirror 706) may deviate from their respective expected relationships. This type of deviation from the expected configuration of the relative relationship may also cause the scan lines of the LIDAR scan pattern to deviate from their expected shape and / or position.

[0120] As another example of assembly error, when assembling a light redirecting device and a galvanometer mirror, relative position errors may occur even if both the light redirecting device and the galvanometer mirror themselves are assembled correctly. Fig. 7A The relative angular relationship, height relationship, horizontal relationship, vertical relationship, relative orientation, etc. between the device 702 in FIG. 1 and the galvanometer mirror (e.g., mirror 706) may deviate from their respective expected relationships. This type of deviation from the expected configuration of the relative relationship may also cause the scan lines of the LIDAR scan pattern to deviate from their expected shape and / or position.

[0121] As another example of assembly error, a multi-faceted light redirecting device may have an angular error between its actual rotation axis and its intended rotation axis. For example, due to assembly error, the actual rotation axis and the intended rotation axis may not be coaxially aligned (e.g., completely overlap). They may be displaced relative to each other. They may also form an angle relative to each other. Therefore, during operation, the facets of the light redirecting device reflect light in a direction that deviates from the intended direction. Therefore, this type of deviation of the actual rotation axis from its intended rotation axis may also cause the scan lines of the LIDAR scan pattern to deviate from their desired shape and / or position.

[0122] In mass production, the assembly of one light redirecting device may produce different results than another light redirecting device. For example, during operation, the movement of a first light redirecting device in an undesirable radial direction may be greater than the movement of a second light redirecting device due to variations in the assembly of the devices. Movement in a radial direction, as opposed to rotation about the axis of the light redirecting device, may cause reflected light to deviate from its intended path. Therefore, movement in a radial direction of a light redirecting device may also cause the scan lines of a LiDAR scan pattern to deviate from their intended shape and / or position.

[0123] The above-mentioned manufacturing-induced non-uniformity differences between facets and assembly errors may cause the scan lines to be deformed or deviate from their intended shapes in the LiDAR scan pattern. Fig. 9An example LiDAR scan pattern 910 with an enlarged view of a portion of the pattern is shown in accordance with some embodiments. As described above, the scan lines of the LiDAR scan pattern may be deformed or deviate from their ideal or intended shape (e.g., a straight line shape) due to non-uniform differences in the facets of the light redirecting device and / or due to errors or inaccuracies in the assembly process of the light redirecting device. Enlarged view 920 shows, for example, that a portion 922 of the scan line has a non-linear or discontinuous scan line shape, where there are multiple segments with different slopes in the scan line. In other words, due to manufacturing-induced facet non-uniform differences and assembly errors, the scan line may deviate from a straight line at one or more portions of the scan line. Thus, depending on the specific non-uniform differences and assembly errors, the scan line may have a non-linear shape, a piecewise linear shape, segments with different slopes, discontinuities, or any other shape. Such deviations from a straight line shape may result in inaccuracies or errors in subsequent processing (e.g., when processing a point cloud generated by the scan line), and in turn, may result in perception difficulties.

[0124] The present disclosure provides methods and systems for compensating for facet non-uniformity differences and assembly errors caused by the manufacture of a multi-faceted light steering device by intelligently configuring the movement curve of the galvanometer mirror. As described above, because the multi-faceted steering device and the galvanometer mirror move simultaneously (e.g., the steering device rotates while the galvanometer mirror oscillates), the scan lines in the LiDAR scan pattern typically have a non-zero slope. In other words, the left and right ends of the scan line may not be the same in the vertical dimension. The slope of the scan line reflects the oscillation of the galvanometer mirror to scan light in the vertical dimension while the light steering device rotates to produce a scan line from left to right. Therefore, because the scan line has a non-zero slope, the movement curve of the galvanometer mirror can be configured to reduce or eliminate the deviation of the scan line from its ideal straight line shape caused by facet non-uniformity differences and assembly errors caused by the manufacture of the multi-faceted light steering device. For example, if the facet non-uniformity differences cause the slope of a particular portion of the scan line to be greater than it should be (and therefore causes that particular portion to deviate from being continuously straight from the previous portion), the movement profile of the galvanometer mirror can be configured to cause the galvanometer mirror to oscillate more slowly or more quickly, or even temporarily stop. Thus, controlling the galvanometer mirror movement in this manner can modify the vertical component of the scan line at a particular portion to compensate for undesirable slope deviations caused by facet non-uniformity differences and assembly errors of the multi-faceted light redirecting device.

[0125] Fig.101000 is a block diagram illustrating the use of a galvanometer mirror movement curve to modify the mirror position based on light steering device calibration data. In one embodiment, after the multi-faceted light steering device is manufactured (e.g., using a molding-based method) and / or assembled with a motor, the device is calibrated to determine uniformity differences between facets and to determine assembly-introduced errors or inaccuracies. As described above, such uniformity differences include facet angle uniformity differences, facet size uniformity differences, surface shape error differences, surface roughness differences, etc. Assembly errors can include, for example, relative position errors between the motor rotor axis and the stator axis; relative position errors between the motor and the motor mounting fixture; relative position errors of the multi-faceted light steering device relative to the galvanometer mirror; and angular errors between the actual rotational axis of the multi-faceted light steering device and the expected rotational axis of the multi-faceted light steering device.

[0126] The multi-faceted light redirecting device can be calibrated to quantify facet uniformity differences and / or assembly errors. For example, these uniformity differences and / or assembly errors can be measured using appropriate equipment (e.g., instruments for measuring surface roughness, surface curvature, dimensions, angles, etc. of optical components). The results of the measurements are stored. In some embodiments, the measurements are compared with expected values ​​or configurations of various aspects of the light redirecting device. The results of the comparison (e.g., differences) are stored for generating calibration data for movement curves.

[0127] refer to Fig.10 , the galvanometer movement curve 1002 is provided to the processor 1006. The movement curve 1002 includes an angular position-time relationship associated with the movement of the galvanometer mirror between a first angular position and a second angular position. The first and second angular positions are relative to an axis about which the galvanometer mirror rotates or oscillates (e.g., Fig. 7A The movement curve 1002 may be, for example, Figure 8 810 is shown. In one embodiment, movement curve 1002 represents the position-time relationship of a galvanometer mirror that is configured to oscillate between two angular positions without modification to account for differences in facet non-uniformity and assembly errors of the light redirecting device. As an example, the first angular position and the second angular position can be approximately -20 degrees and approximately +20 degrees, respectively. Thus, the galvanometer mirror oscillates to cover a range of approximately 40 degrees in one dimension (e.g., the vertical dimension) of the FOV.

[0128] like Fig.10As shown, the light steering device calibration data 1004 is also provided to the processor 1006. Data 1004 includes measurement and / or comparison data of a multi-faceted light steering device such as a polygon. As described above, the calibration data includes measurements of uniformity differences and assembly errors of a specific light steering device. The calibration data is usually different for light steering devices. In this case, the calibration data 1004 is associated with the light steering device 1042. The processor 1006 can be implemented by hardware and / or software. It can be a discrete component, a part of the galvanometer controller 1014, a part of another processor 1010, and / or a part of any other component in the LiDAR system. It can also be a processor placed outside the LiDAR system (for example, in a cloud computing environment). The processor 1006 receives the movement curve 1002 of the galvanometer mirror 1022 and the calibration data 1004 of the light steering device 1042 (for example, a multi-faceted mirror) used with the galvanometer mirror 1022. It performs one or more signal processing operations based on the movement curve 1002 and the calibration data 1004. Based on the calibration data 1004 and the movement curve 1002, the processor 1006 generates a galvanometer movement curve 1007. In one embodiment, the movement curve 1007 includes a different position-time relationship than the position-time relationship of the movement curve 1002 because it takes into account the calibration data 1004. In one embodiment, the movement curve 1007 is in the form of a galvanometer mirror position command.

[0129] One or more signal processing operations may be used to generate the movement curve 1007. Some of these signal processing operations include data sampling, filtering, analog-to-digital conversion, superposition, data compensation, position control, data transformation, digital-to-analog conversion, subtraction, addition, multiplication, division, and / or any other desired operation. The movement curve 1007 represents a movement curve modified from the original movement curve 1002 using the calibration data 1004. For example, based on the calibration data 1004, one or more specific angular positions associated with one or more specific time positions in the movement curve 1002 may be modified (increased or decreased) to compensate for facet angle uniformity differences between different facets. The original movement curve 1002 may be similarly modified to account for assembly errors. Thus, uniformity differences caused by manufacturing and / or assembly errors may be compensated at least to some extent so that the galvanometer mirror 1022 and the light steering device 1042 are controlled to generate an improved or desired scan line (e.g., a straight scan line).

[0130] Fig.11 An example movement curve 1007 is shown that may be used to modify the position of the galvanometer mirror. Fig.11 Also shown is an enlarged view of a portion of a movement curve 1007 according to some embodiments. Movement curve 1007 is obtained by considering Fig.101042. The light redirecting device 1042 may be, for example, the device 702, 752, 762, and 902. Figure 8 As described, the moving curve 810 is an example curve of a galvanometer mirror without considering the facet non-uniformity differences and assembly errors. Compared with the moving curve 810, the moving curve 1007 also has a generally sawtooth shape. Unlike the curve 810, the trajectory 1115 of the moving curve 1007 no longer has a straight line shape. On the contrary, as shown in the enlarged view 1120 of a portion 1117 of the trajectory 1115, the trajectory may have a non-linear shape, a piecewise linear shape, a discontinuous or any other desired shape. The shape, slope, length, etc. of the segments in the trajectory 1115 are configured so that the galvanometer controlled by using the trajectory 1115 can compensate for the facet non-uniformity differences and assembly errors associated with the light redirection device 1042 at least to some extent. As described above, this is performed by configuring the trajectory 1115 to increase or decrease the angular position of the galvanometer mirror at one or more specific time positions. In some embodiments, if the light redirecting device has five facets, each of the five segments of portion 1117 can correspond to a corresponding facet for compensating for differences in non-uniformity and / or assembly errors of the corresponding facets. Thus, for a five-faceted light redirecting device, portion 1117 has five corresponding segments, each having a corresponding slope (or some possibly having the same slope). Portion 1117 can repeat. Thus, trajectory 1115 can have a periodic repeating pattern including many such repeating portions. For example, by controlling the oscillation of the galvanometer mirror according to trajectory 1115, the scan line of a LiDAR system can be improved to have a desired or expected shape (e.g., a straight line).

[0131] It should be understood that in some cases, it may not be necessary to modify the movement curve 1002 because the calibration data 1004 does not indicate any uniformity differences or assembly errors, or because the differences and / or errors are below some acceptable thresholds. In this case, the movement curve 1007 can be the same as the movement curve 1002.

[0132] Return to reference Fig.10, in some embodiments, the movement curve 1007 is provided to the processor 1010. The processor 1010 can be implemented by hardware and / or software. It can be a discrete component, part of the galvanometer controller 1014, part of another processor 1006, and / or part of any other component in the LiDAR system. It can also be a processor placed outside the LiDAR system (for example, in a cloud computing environment). The processor 1010 is configured to receive the movement curve 1007, the galvanometer position feedback data 1012, and the light steering device position feedback data 1032. Based on the received data, the processor 1010 can generate one or more adjustment signals 1011. As described above, the movement curve 1007 represents an adjusted movement curve modified from the original movement curve 1002 using the calibration data 1004. Therefore, the movement curve 1007 can be used to control the movement of the galvanometer mirror 1022 to compensate for the facet non-uniformity differences and assembly errors associated with the light steering device 1042.

[0133] During operation, the angular position of the galvanometer mirror 1022 may not always be precisely controlled (e.g., according to the movement curve 1007 of the mirror 1022), and may have positional inaccuracies over time. Positional inaccuracies may arise due to many factors, such as controller inaccuracies, assembly-induced inaccuracies, inaccuracies caused by optical component manufacturing tolerances, vibrations, shocks, temperature changes, and / or other environmental changes. For example, during operation of a motor vehicle equipped with a LiDAR system, the LiDAR system and its components often experience vibrations or shocks. Vibrations and shocks may affect the positional accuracy of one or more optical components in the LiDAR system, including the mirror 1022. Therefore, in some embodiments, there will be a difference between the expected angular position included in the movement curve 1007 and the actual angular position of the galvanometer mirror 1022. In order to reduce or eliminate such differences, the position feedback data 1012 may be considered when generating control signals using the movement curve 1007.

[0134] In addition, Fig.10 In the embodiments of , positional inaccuracies of the light redirecting device 1042 may also be taken into account, at least to some extent, by providing position feedback data 1032 associated with the light redirecting device 1042 to the processor 1010. As described above, since the light redirecting device 1042 and the mirror 1022 move simultaneously, differences in facet non-uniformity and assembly errors of the light redirecting device 1042 may be compensated, at least to some extent, by intelligently configuring the movement curve of the galvanometer mirror. Similarly, positional inaccuracies of the light redirecting device 1042 may also be at least partially corrected by further modifying (e.g., tuning) the movement curve 1007. As an example, return to reference Figure 8, each scan line 822 has a non-zero slope. Therefore, each data point of the scan line 822 has a corresponding angular position of the light steering device 1042 (e.g., for scanning the horizontal dimension) and a corresponding angular position of the galvanometer mirror 1022 (e.g., for scanning the vertical dimension) at any particular time. If at a certain time, the angular position of the light steering device 1042 is slightly greater than its expected position, but the angular position of the mirror 1022 is unchanged, the resulting portion of the scan line will have a changing slope (e.g., a smaller slope). This will cause the scan line to deviate from the desired straight line shape. By using the real-time position data of the light steering device 1042, the angular position of the mirror 1022 can be modified in real time to at least partially compensate for the positional inaccuracy of the light steering device 1042.

[0135] In some embodiments, the light steering device 1042 rotates at a very high speed, for example, several thousand revolutions per minute (e.g., 9000rpm). Therefore, due to the high moment of inertia or momentum of the light steering device 1042, it may be difficult or impractical to directly control the light steering device to correct its speed inaccuracy in a short time. The galvanometer mirror 1022 usually oscillates at a slower speed and is therefore easier to control. Therefore, by modifying the movement curve 1007 to control the galvanometer mirror 1022, the angular position inaccuracy of the light steering device 1042 can be at least partially compensated. In some embodiments, the light steering device position feedback data 1032 is provided to the light steering device motor controller 1034 for more directly correcting the speed inaccuracy of the light steering device 1042. For example, based on the light steering device position feedback data 1032, if it is determined that the speed of the light steering device 1042 is constantly or frequently greater than or less than the expected speed, the light steering device motor controller 1034 can be configured to reduce or increase the rotation speed based on the feedback data 1032. It should be appreciated that the light steering device position feedback data 1032 may be provided directly (by directly controlling the light steering device) and / or indirectly (by controlling the galvanometer mirror 1022) in any desired manner.

[0136] In some embodiments, the multi-faceted light steering device 1042 and the galvanometer mirror 1022 each have a rotary position encoder and / or a Hall effect sensor or any other desired position encoder. The position encoder provides corresponding position feedback data 1012 and 1032 to the processor 1010. Using the movement curve 1007, the position feedback data 1012 associated with the galvanometer mirror 1022, and the position feedback data 1032 associated with the multi-faceted light steering device 1042, the processor 1010 generates an adjustment signal 1011 by performing one or more of the signal processing operations. Some of these signal processing operations include data sampling, filtering, analog-to-digital conversion, superposition, data compensation, position control, data transformation, digital-to-analog conversion, subtraction, addition, multiplication, division and / or any other desired operation. The adjustment signal 1011 represents a modified movement curve based on the position feedback data 1012 and / or 1032. For example, based on the position feedback data 1012 and / or 1032, a specific angular position associated with a specific time in the movement curve 1007 can be modified (increased or decreased) to compensate for the angular position inaccuracy of the galvanometer mirror 1022 and / or the angular position inaccuracy of the light steering device 1042 during operation. Thus, the oscillation trajectory of the galvanometer mirror 1022 can be controlled and adjusted in real time during operation. It should be understood that in some cases, the movement curve 1007 may not need to be modified because the position feedback data 1012 and / or 1032 do not indicate any inaccuracy or the inaccuracy is below a threshold. If there is no inaccuracy or there is an inaccuracy below a threshold, the movement curve 1007 can be used alone to generate the adjustment signal 1011.

[0137] Still refer to Fig.10 , one or more regulated signals 1011 are provided to a galvanometer controller 1014. Using the regulated signals 1011, the controller 1014 generates control signals 1017 for controlling a galvanometer driver 1018. In some embodiments, the control signals 1017 are pulse width modulated (PWM) signals (e.g., 3.3V signals with milliamp current levels). These pulse width modulated signals are provided to the galvanometer driver 1018, which can generate a stronger signal 1019 to drive the galvanometer motor 1020. In one embodiment, the galvanometer driver 1018 includes an amplifier for amplifying the input PWM control signal 1017 to generate a 12V PWM signal 1019 with an ampere level current. This high power signal 1019 is then used to drive the galvanometer motor 1020 to oscillate the galvanometer mirror 1022. In some embodiments, two or more of the galvanometer controller 1014, the galvanometer mirror 1022, the motor 1020, the position encoder (not shown), and the motor driver 1018 are included in the galvanometer mirror assembly.

[0138] As described above, the angular position of the light steering device 1042 may also have positional inaccuracies, and therefore the position feedback data 1032 may be provided to one or both of the processor 1010 and the light steering device controller 1034 for at least partially compensating for the positional inaccuracies of the light steering device 1042. The light steering device 1042 may also obtain its angular position by using a position encoder. The position encoder may be a rotary position encoder and / or a Hall effect sensor, or any other desired position encoder. The position encoder provides position feedback data 1032 to the processor 1010 and / or the light steering device controller 1034. As described above, using the light steering device position feedback data 1032 together with the movement curve 1007 and the galvanometer position feedback data 1012, the processor 1010 generates an adjustment signal 1011.

[0139] In one embodiment, using the position feedback data 1032, the light steering device controller 1034 generates one or more control signals 1037 for directly controlling the light steering device 1042 to compensate for its position inaccuracy. For example, based on the position feedback data 1032, the specific angular position of the light steering device 1042 at a specific time can be modified (increased or reduced) to compensate for the angular position error of the device 1042 during operation. Thus, one or more aspects (e.g., speed) of the rotational movement of the light steering device 1042 can be controlled and adjusted in real time during operation. It should be understood that in some cases, the rotational movement of the light steering device 1042 may not need to be adjusted because the position feedback data 1032 indicates that there is no inaccuracy (or the inaccuracy or error is below a threshold). If there is no inaccuracy or there is an inaccuracy below a threshold, the light steering device controller 1034 may not generate any signal for adjusting the movement of the light steering device 1042.

[0140] Still reference Fig.10 In some embodiments, the control signal 1037 is a pulse width modulated (PWM) signal (e.g., a 3.3V signal with a milliampere current level). These pulse width modulated signals are provided to the light steering device driver 1038, which can generate a stronger signal 1039 to drive the light steering device motor 1040. In one embodiment, the light steering device driver 1038 includes an amplifier for amplifying the input PWM control signal 1037 to generate a 12V PWM signal 1039 with an ampere level current. The signal 1039 has high power and is then used to drive the motor 1040 to rotate the light steering device 1042. In some embodiments, two or more of the light steering device controller 1034, the light steering device driver 1038, the light steering device motor 1040, the position encoder (not shown), and the light steering device 1042 (e.g., a polygonal mirror) are included in the multi-faceted light steering device assembly.

[0141] refer to Fig.10 In some embodiments, the processor 1006, the processor 1010, the galvanometer controller 1014, and the light steering device controller 1034 may be included in the controller device 1030. The controller device 1030 may have one or more processors, a memory, and processor executable instructions stored in the memory. These instructions may be executed by one or more processors (e.g., 1006 and 1010) to perform one or more steps of the method described in the present disclosure. In addition, the controller device 1030 may also include a storage element (e.g., a memory, a hard disk, a flash memory, etc.) for storing the light steering device calibration data 1004. The controller device 1030 may be implemented using, for example, a microcontroller, one or more processors, a field programmable gate array (FPGA), a digital signal processor (DSP), and / or any other computing device that can implement the methods and processes described herein.

[0142] By using the position feedback data of the light steering device 1042 and / or the galvanometer mirror 1022, in addition to compensating for the facet non-uniformity differences and assembly errors of the light steering device 1042, the movement curve 1007 of the galvanometer mirror 1022 can be further modified to compensate for the real-time position inaccuracies of one or both of the light steering device 1042 and the mirror 1022. In some embodiments, the position feedback data of the light steering device 1042 is also directly used to control the device 1042 to compensate for its position error. Therefore, the deviation of the scan line shape from its desired shape (e.g., a straight line shape) can be reduced or eliminated. An improved LiDAR scan pattern can be obtained for subsequent processing (e.g., perception). Therefore, the overall performance of the LiDAR system is also enhanced. It should be understood that Fig.10 The blocks in are for illustration purposes, and they may be separated, combined, added, removed, or changed in any desired manner. For example, processors 1006 and 1010 may be combined with each other or with a controller (e.g., controller 1014). The controller and driver may also be combined in one component or device.

[0143] Fig.12 An example LiDAR scan pattern 1210 and a magnified view of a portion 1220 of the pattern 1210 are shown in accordance with some embodiments. The LiDAR scan pattern 1210 is obtained using the systems and processes described in the present disclosure to compensate for differences in one or more facet non-uniformities of the light steering device, assembly errors, and real-time positional inaccuracies of the light steering device and / or the galvanometer mirror. Fig.12 As shown, the LiDAR scan pattern is compensated so that the scan line is improved to have a desired or expected shape (e.g., a straight line). This scan pattern improves the efficiency of subsequent processing, such as perception using point cloud data representing the scan pattern.

[0144] Fig.13 13 is a flow chart showing a method 1300 for controlling a LiDAR scanning system. In some embodiments, the method 1300 starts at steps 1302 and 1304. Step 1302 receives a first movement curve of a galvanometer mirror of the LiDAR scanning system. For example, the first movement curve is movement curve 810, which does not take into account any facet non-uniformity differences and assembly errors of the multi-faceted light redirecting device. Step 1304 receives calibration data of the multi-faceted light redirecting device of the LiDAR scanning system. Using the first movement curve and the calibration data, step 1306 generates a second movement curve of the galvanometer mirror. The second movement curve is a modified curve that can be used to control the galvanometer mirror to compensate for the facet non-uniformity differences and assembly errors of the multi-faceted light redirecting device.

[0145] In some embodiments, step 1312 receives position feedback data associated with the galvanometer mirror. Step 1314 receives position feedback data associated with the multi-faceted light steering device. Using the second movement curve, the received position feedback data of the mirror, and the received position feedback data of the light steering device, step 1316 generates one or more adjustment signals. Based on the adjustment signals, step 1308 generates one or more control signals for controlling a driver of the galvanometer mirror. In some embodiments, using the position feedback data associated with the multi-faceted light steering device, step 1318 generates one or more control signals for controlling a driver of the light steering device.

[0146] In some embodiments, method 1300 may further include step 1310 of sensing the angular position of the multi-faceted light steering device and step 1320 of sensing the angular position of the galvanometer mirror. The sensed position feedback data is provided to steps 1312 and 1314, respectively. Thus, the process may be repeated such that the second movement profile (or the previously modified profile) is modified to generate an adjustment signal for performing real-time control of the light steering device and / or the galvanometer mirror. Thus, positional inaccuracies of the light steering device and / or the galvanometer mirror may be corrected or compensated in real time.

[0147] The foregoing description should be understood to be illustrative and exemplary in every respect, rather than restrictive, and the scope of the invention disclosed herein is not determined by the description, but by the claims interpreted according to the full scope allowed by the patent law. It should be understood that the embodiments shown and described herein are only used to illustrate the principles of the present invention, and those skilled in the art can implement various modifications without departing from the scope and spirit of the present invention. Those skilled in the art can implement various other feature combinations without departing from the scope and spirit of the present invention.

Claims

1. A light detection and ranging LiDAR scanning system, comprising: a multi-faceted light redirecting device configured to rotate for scanning of a light beam in one dimension of a field of view; a galvanometer mirror configured to oscillate for scanning of the light beam in another dimension of the field of view different from the one dimension; as well as A controller device comprising one or more processors, a memory, and processor-executable instructions stored in the memory, the processor-executable instructions comprising instructions for: receiving a first movement profile of the galvanometer mirror of the LiDAR scanning system; Receiving calibration data of the multi-faceted light redirection device of the LiDAR scanning system; generating a second movement curve for the galvanometer mirror based on the calibration data and the first movement curve; as well as One or more control signals are provided based on the second movement profile to adjust movement of the galvanometer mirror.

2. The system according to claim 1, wherein: A first movement profile of the galvanometer mirror includes a position-time relationship associated with movement of the galvanometer mirror between a first angular position and a second angular position.

3. The system according to claim 2, wherein: The difference between the first angular position and the second angular position is approximately 40 degrees, the first angular position and the second angular position being relative to an axis about which the galvanometer mirror oscillates.

4. The system according to claim 1, wherein: The calibration data of the multi-faceted light redirecting device includes data associated with calibration of a difference between at least two facets of the multi-faceted light redirecting device, the difference being caused by a manufacturing process.

5. The system according to claim 4, wherein: The difference between at least two facets of the multi-faceted light redirecting device comprises at least one of the following: Differences in facet angle uniformity; Differences in facet size uniformity; Surface form error differences; and Difference in surface roughness.

6. The system according to claim 1, wherein: The calibration data of the multi-faceted light redirecting device includes assembly errors associated with an assembly process of the multi-faceted light redirecting device to the LiDAR scanning system.

7. The system of claim 6, further comprising a motor for rotating the multi-faceted light redirecting device, wherein: The assembly error includes at least one of the following: The relative position error between the motor rotor axis and the motor stator axis; Relative position error between the motor and the motor mounting fixture; a relative position error of the multi-faceted light redirecting device with respect to the galvanometer mirror; and An error between an actual axis of rotation of the multi-faceted light redirecting device and an intended axis of rotation of the multi-faceted light redirecting device.

8. The system according to claim 1, wherein: Generating a second movement curve of the galvanometer mirror based on the calibration data and the first movement curve comprises: Perform one or more signal processing operations using the calibration data and the first movement profile, the one or more signal processing operations comprising at least one of the following: Data sampling; Filtering; Analog-to-digital conversion; Data calibration and compensation; superposition; Data transformation; and Digital to analog conversion.

9. The system according to claim 1, wherein: The second movement profile comprises a position-time relationship that is different from the position-time relationship of the first movement profile.

10. The system according to claim 1, wherein: The multi-faceted light redirecting device comprises a multi-faceted mirror.

11. The system according to claim 1, wherein: Providing the one or more control signals to adjust the movement of the galvanometer mirror based on the second movement profile includes: receiving position feedback data associated with the galvanometer mirror; receiving position feedback data associated with the multi-faceted light redirecting device; and One or more adjustment signals are generated based on the second movement profile, position feedback data associated with the galvanometer mirror, and position feedback data associated with the multi-faceted light redirecting device.

12. The system according to claim 11, wherein: Generating the one or more adjustment signals comprises: Perform one or more signal processing operations using the second movement profile, position feedback data associated with the galvanometer mirror, and position feedback data associated with the multi-faceted light steering device, the signal processing operations comprising at least one of: Data sampling; Filtering; Analog-to-digital conversion; Data calibration and compensation; superposition; Data transformation; and Digital to analog conversion.

13. The system of claim 11, further comprising generating one or more control signals for controlling a driver of the galvanometer mirror based on the one or more adjustment signals.

14. The system according to claim 13, wherein: One or more control signals for controlling the driver of the galvanometer mirror may include a pulse width modulated signal.

15. The system of claim 1, further comprising a galvanometer mirror assembly, the galvanometer mirror assembly comprising: First motor; said galvanometer mirror; as well as A first motor driver is configured to oscillate the galvanometer mirror according to the one or more control signals.

16. The system of claim 15, wherein: The galvanometer mirror assembly also includes a mirror position encoder configured to provide position feedback data associated with the galvanometer mirror.

17. The system of claim 1, wherein: The processor executable instructions also include instructions for: receiving position feedback data associated with the multi-faceted light redirecting device; and Based on position feedback data associated with the multi-faceted light redirecting device, one or more second control signals are provided to adjust movement of the multi-faceted light redirecting device.

18. The system of claim 17, further comprising a multi-faceted light redirecting device assembly, the multi-faceted light redirecting device assembly comprising: Second motor; The multi-faceted light redirecting device comprises a multi-faceted mirror having a plurality of reflective surfaces; as well as A second motor driver is configured to rotate the multi-faceted light redirecting device according to the one or more second control signals.

19. A method for controlling a light detection and ranging (LiDAR) scanning system, the method being executed by one or more processors and a memory, the method comprising: receiving a first movement curve of a galvanometer mirror of the LiDAR scanning system; Receiving calibration data of a multi-faceted light steering device of the LiDAR scanning system; generating a second movement curve for the galvanometer mirror based on the calibration data and the first movement curve; as well as providing one or more control signals to adjust movement of the galvanometer mirror based on the second movement profile, Wherein, the multi-faceted light redirecting device is configured to rotate for scanning the light beam in one dimension of the field of view, and the galvanometer mirror is configured to oscillate for scanning the light beam in another dimension of the field of view different from the one dimension.

20. The method according to claim 19, wherein: A first movement profile of the galvanometer mirror includes a position-time relationship associated with movement of the galvanometer mirror between a first angular position and a second angular position.

21. The method according to claim 19, wherein: The calibration data of the multi-faceted light redirecting device includes data associated with calibration of a difference between at least two facets of the multi-faceted light redirecting device, the difference being caused by a manufacturing process.

22. The method according to claim 19, wherein: The calibration data of the multi-faceted light redirecting device includes assembly errors associated with an assembly process of the multi-faceted light redirecting device to the LiDAR scanning system.

23. The method according to claim 19, wherein: Generating a second movement curve of the galvanometer mirror based on the calibration data and the first movement curve comprises: Perform one or more signal processing operations using the calibration data and the first movement profile, the one or more signal processing operations comprising at least one of the following: Data sampling; Filtering; Analog-to-digital conversion; Data calibration and compensation; superposition; Data transformation; and Digital to analog conversion.

24. The method according to claim 19, wherein: The second movement profile comprises a position-time relationship that is different from the position-time relationship of the first movement profile.

25. The method of claim 19, wherein: Providing the one or more control signals to adjust the movement of the galvanometer mirror based on the second movement profile includes: receiving position feedback data associated with the galvanometer mirror; receiving position feedback data associated with the multi-faceted light redirecting device; and One or more adjustment signals are generated based on the second movement profile, position feedback data associated with the galvanometer mirror, and position feedback data associated with the multi-faceted light redirecting device.

26. The method according to claim 25, wherein: Generating the one or more adjustment signals comprises performing one or more signal processing operations using the second movement profile, position feedback data associated with the galvanometer mirror, and position feedback data associated with the multi-faceted light steering device; the method further comprising One or more control signals for controlling a driver of the galvanometer mirror are generated based on the one or more adjustment signals.

27. The method of claim 19, further comprising: receiving position feedback data associated with the multi-faceted light redirecting device; as well as Based on position feedback data associated with the multi-faceted light redirecting device, one or more second control signals are provided to adjust movement of the multi-faceted light redirecting device.

28. A non-transitory computer-readable medium storing processor-executable instructions, comprising instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform the following processing of a light detection and ranging (LiDAR) scanning system, wherein the LiDAR scanning system comprises a multi-faceted light steering device configured to rotate for scanning of a light beam in one dimension of a field of view; and a galvanometer mirror configured to oscillate for scanning of a light beam in another dimension of the field of view different from the one dimension, receiving a first movement curve of a galvanometer mirror of the LiDAR scanning system; Receiving calibration data of a multi-faceted light steering device of the LiDAR scanning system; generating a second movement curve for the galvanometer mirror based on the calibration data and the first movement curve; as well as One or more control signals are provided based on the second movement profile to adjust movement of the galvanometer mirror.

29. A vehicle comprising a light detection and ranging (LiDAR) scanning system, the LiDAR scanning system comprising: a multi-faceted light redirecting device configured to rotate for scanning of a light beam in one dimension of a field of view; a galvanometer mirror configured to oscillate for scanning of the light beam in another dimension of the field of view different from the one dimension; as well as A controller device comprising one or more processors, a memory, and processor-executable instructions stored in the memory, the processor-executable instructions comprising instructions for: receiving a first movement profile of the galvanometer mirror of the LiDAR scanning system; Receiving calibration data of the multi-faceted light redirection device of the LiDAR scanning system; generating a second movement curve for the galvanometer mirror based on the calibration data and the first movement curve; as well as One or more control signals are provided based on the second movement profile to adjust movement of the galvanometer mirror.

Images