Method, vehicle, and computer-readable medium for LiDAR scan smoothing

Abstract

The present invention relates to methods, vehicles, and computer-readable media for LiDAR scan smoothing. The following technology is also described, which is used to identify a first LiDAR data point and a plurality of LiDAR data points in a vicinity of the first LiDAR data point in a light detection and ranging scan line, i.e., a LiDAR scan line. The technology may also include identifying a coefficient of the first LiDAR data point based on a comparison of the first LiDAR data point with at least one LiDAR data point in a plurality of LiDAR echo points, wherein the coefficient is related to image smoothness. The technology may also include: identifying whether to include the first LiDAR data point in an updated LiDAR scan line based on a comparison of the coefficient with a threshold; and identifying a location of an autonomous vehicle based on the updated LiDAR scan line.

Description

Technical Field

[0001] This specification relates to Light Detection and Ranging (LiDAR) scan smoothing. Background Art

[0002] Typically, a vehicle, such as an autonomous vehicle, can use a positioning process to identify the location of the vehicle geographically. Specifically, the vehicle can acquire a depth image by, for example, performing a LiDAR scan. The LiDAR scan can then be compared to data associated with a known geographic location to identify the location of the vehicle. Summary of the invention

[0003] According to one aspect of the invention, one or more non-transitory computer-readable media include instructions that, when executed by one or more processors of a vehicle, cause the vehicle to: identify a first LiDAR data point in a light detection and ranging scan line, or LiDAR scan line; identify a plurality of LiDAR data points in the LiDAR scan line within a vicinity of the first LiDAR data point; identify a coefficient of the first LiDAR data point based on a comparison of the first LiDAR data point with at least one of the plurality of LiDAR data points, wherein the coefficient is related to image smoothness; identify whether to include the first LiDAR data point in an updated LiDAR scan line based on a comparison of the coefficient with a threshold; and identify a location of the vehicle based on the updated LiDAR scan line.

[0004] According to yet another aspect of the invention, a method includes: identifying, by at least one processor, a first LiDAR data point in a light detection and ranging scan line (LiDAR scan line), a second LiDAR data point in the LiDAR scan line that is adjacent to the first LiDAR data point, and a third LiDAR data point in the LiDAR scan line that is adjacent to the first LiDAR data point; identifying, by the at least one processor, a coefficient of the first LiDAR data point based on a comparison of the first LiDAR data point with the second LiDAR data point and the third LiDAR data point, wherein the coefficient is related to image smoothness; identifying, by the at least one processor, whether to include the first LiDAR data point in an updated LiDAR scan line based on a comparison of the coefficient with a threshold; and identifying a location of a vehicle based on the updated LiDAR scan line.

[0005] According to another aspect of the present invention, a vehicle includes: a light detection and ranging system (LiDAR system) for generating one or more LiDAR scan lines including multiple LiDAR data points; and at least one processor coupled to the LiDAR system, wherein the at least one processor is used to: identify a first LiDAR data point among the multiple LiDAR data points, a second LiDAR data point in the scan line that is adjacent to the first LiDAR data point, and a third LiDAR data point in the scan line that is adjacent to the first LiDAR data point; identify a coefficient of the first LiDAR data point based on a comparison of the first LiDAR data point with the second LiDAR data point and the third LiDAR data point, wherein the coefficient is related to image smoothness; and identify whether to include the first LiDAR data point in an updated LiDAR scan line based on a comparison of the coefficient with a threshold. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An example of an autonomous vehicle with autonomous capabilities is shown.

[0007] Figure 2 A computer system is shown.

[0008] Figure 3 An example architecture of an autonomous vehicle is shown.

[0009] Figure 4 Shows examples of inputs and outputs that a perception module can use.

[0010] Figure 5 An example of a LiDAR system is shown.

[0011] Figure 6 The LiDAR system is shown in operation.

[0012] Figure 7 The operation of the LiDAR system is shown in more detail.

[0013] Figure 8 Depicted is an example LiDAR scan with multiple scan lines in accordance with an embodiment.

[0014] Fig. 9 Depicting a graphical example of an algorithm for calculating a smoothness coefficient associated with data points of a LiDAR scan, in accordance with an embodiment.

[0015] Fig.10 Depicting an alternative graphical example of an algorithm for calculating a smoothness coefficient associated with data points of a LiDAR scan, in accordance with an embodiment.

[0016] Fig.11Depicted is an example technique for updating a LiDAR scan line in accordance with an embodiment.

[0017] Fig.12 An alternative example technique for updating LiDAR scan lines is depicted in accordance with an embodiment. DETAILED DESCRIPTION

[0018] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be apparent that the present invention can be implemented without these specific details. In other examples, well-known configurations and devices are shown in block diagram form to avoid unnecessarily obscuring the present disclosure.

[0019] In the accompanying drawings, for ease of description, the specific arrangement or order of schematic elements (such as those representing devices, modules, instruction blocks, and data elements) is shown. However, it should be understood by those skilled in the art that the specific order or arrangement of the schematic elements in the accompanying drawings is not intended to mean that a specific processing order or sequence, or separation of processing processes is required. In addition, the inclusion of schematic elements in the accompanying drawings is not intended to mean that such elements are required in all embodiments, nor is it intended to mean that the features represented by such elements cannot be included in some embodiments or cannot be combined with other elements in some embodiments.

[0020] In addition, in the accompanying drawings, connecting elements, such as solid or dotted lines or arrows, are used to illustrate the connection, relationship or association between two or more other schematic elements, and there is no such connecting element and is not intended to mean that there can be no connection, relationship or association. In other words, the connection, relationship or association between some elements are not shown in the accompanying drawings, so as not to make the present disclosure vague. In addition, for the convenience of illustration, a single connecting element is used to represent multiple connections, relationships or associations between elements. For example, if the connecting element represents the communication of signal, data or instruction, it will be understood by those skilled in the art that this element represents one or more signal paths (for example, bus) that may be needed to affect the communication.

[0021] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments described. However, it will be apparent to one of ordinary skill in the art that the various embodiments described may be implemented without these specific details. In other cases, well-known methods, procedures, components, circuits, and networks are not described in detail in order not to unnecessarily obscure aspects of the embodiments.

[0022] Several features described below can each be used independently of one another or in combination with other features. However, any individual feature may not solve any of the problems discussed above, or may only solve one of the problems discussed above. Some of the problems discussed above may not be fully solved by any one of the features described herein. Although a title is provided, information related to a specific title but not found in the section with that title may also be found elsewhere in this specification. This article describes an embodiment according to the following summary:

[0023] 1. General Overview

[0024] 2. System Overview

[0025] 3. Autonomous Vehicle Architecture

[0026] 4. Autonomous Vehicle Input

[0027] 5. Scan line overview

[0028] 6. Smoothness coefficient example

[0029] 7. Use of smoothness coefficient

[0030] General Overview

[0031] To smooth the LiDAR scan, a data point (also referred to as a "return" or "point") in the LiDAR scan line is identified, and a smoothness coefficient is calculated for the data point based on neighboring data points in the scan line. Based on the smoothness coefficient, the data point is either discarded from the LiDAR scan line (e.g., if the smoothness coefficient is equal to or exceeds a threshold), or the data point is retained in the LiDAR scan line (e.g., if the smoothness coefficient is equal to or below a threshold).

[0032] Embodiments herein provide a number of advantages, particularly when implemented as part of a positioning process performed by a vehicle. Specifically, in some embodiments, determining the presence of a relatively high smoothness coefficient associated with a data point can indicate the presence of transient elements in the environment, such as a person, bicycle, leaves, etc., associated with the data point. These transient elements may change frequently (e.g., a person may walk away, a bicycle may ride away, leaves may change, etc.), and therefore, it may be difficult for a vehicle to accurately perform a positioning process based on the presence of these elements. By detecting and removing data points associated with these elements, less transient structures such as buildings, walls, etc. can be more accurately identified. The use of these less transient structures can also improve the accuracy and repeatability of the positioning process of the vehicle, thereby improving the overall efficiency of the vehicle. Additionally or alternatively, comparison between at least one LiDAR data point included in a vehicle-obtained LiDAR scan and at least one LiDAR data point included in an earlier generated LiDAR scan associated with a particular area can be reduced by distrusting (e.g., removing) at least one LiDAR data point included in the vehicle-obtained LiDAR scan that is associated with (e.g., corresponds to) a transient element.

[0033] System Overview

[0034] Figure 1 An example of an autonomous vehicle 100 having autonomous capabilities is shown.

[0035] As used herein, the term "autonomous capability" refers to a function, feature, or facility that enables a vehicle to operate partially or fully without real-time human intervention, including but not limited to fully autonomous vehicles, highly autonomous vehicles, and conditionally autonomous vehicles.

[0036] As used herein, an autonomous vehicle (AV) is a vehicle with autonomous capabilities.

[0037] As used herein, "vehicle" includes a mode of transport of goods or people. For example, a car, bus, train, airplane, drone, truck, boat, ship, submersible, spacecraft, etc. An unmanned car is an example of a vehicle.

[0038] As used herein, a "track" refers to a path or route that navigates an AV from a first spatiotemporal location to a second spatiotemporal location. In an embodiment, the first spatiotemporal location is referred to as an initial location or starting location, and the second spatiotemporal location is referred to as a destination, final location, target, target location, or target location. In some examples, a track consists of one or more road segments (e.g., several sections of a road), and each road segment consists of one or more blocks (e.g., a portion of a lane or intersection). In an embodiment, a spatiotemporal location corresponds to a real-world location. For example, a spatiotemporal location is a pickup or drop-off location to get people or cargo on or off the vehicle.

[0039] As used herein, "sensor(s)" includes one or more hardware components for detecting information about the environment surrounding the sensor. Some hardware components may include sensing components (e.g., image sensors, biometric sensors), transmission and / or reception components (e.g., laser or radio frequency wave transmitters and receivers), electronic components (such as analog-to-digital converters), data storage devices (such as RAM and / or non-volatile memory), software or firmware components and data processing components (such as application specific integrated circuits), microprocessors and / or microcontrollers.

[0040] As used herein, a “scene description” is a data structure (e.g., a list) or data stream that includes one or more categorized or labeled objects detected by one or more sensors on an AV vehicle, or provided by a source external to the AV.

[0041] As used herein, a "road" is a physical area that can be traversed by a vehicle and may correspond to a named thoroughfare (e.g., a city street, an interstate highway, etc.) or may correspond to an unnamed thoroughfare (e.g., a driveway within a house or office building, a section of a parking lot, a section of a vacant parking lot, a dirt road in a rural area, etc.). Because some vehicles (e.g., four-wheel drive pickup trucks, off-road vehicles (SUVs), etc.) are able to traverse a variety of physical areas that are not particularly suitable for vehicle travel, a "road" may be any physical area that is not formally defined as a thoroughfare by a municipality or other governmental or administrative agency.

[0042] As used herein, a "lane" is a portion of a road that can be traversed by a vehicle. Lanes are sometimes identified based on lane markings. For example, a lane may correspond to most or all of the space between lane markings, or to only some of the space between lane markings (e.g., less than 50%). For example, a road with lane markings that are far apart may accommodate two or more vehicles between the markings, so that one vehicle can pass another vehicle without crossing the lane markings, and thus may be interpreted as having a lane that is narrower than the space between the lane markings, or having two lanes between the lanes. Lanes may also be interpreted in the absence of lane markings. For example, lanes may be defined based on physical features of the environment, such as rocks and trees along a pathway in a rural area, or natural obstacles to be avoided, such as in an undeveloped area. Lanes may also be interpreted independently of lane markings or physical features. For example, a lane may be interpreted based on an arbitrary path without obstacles in an area that otherwise lacks features that would be interpreted as lane boundaries. In an example scenario, an AV may interpret a lane through an obstacle-free portion of a field or open space. In another example scenario, an AV may interpret a lane through a wide (e.g., wide enough for two or more lanes) road that does not have lane markings. In this scenario, the AV may communicate information about the lane to other AVs so that the other AVs may use the same lane information to coordinate path planning between AVs.

[0043] The term “Over-the-Air (OTA) Client” includes any AV, or any electronic device (e.g., computer, controller, IoT device, electronic control unit (ECU)) embedded in, coupled to, or communicating with an AV.

[0044] The term "Over-the-Air (OTA) Update" means any update, change, deletion or addition to software, firmware, data or configuration settings, or any combination thereof, delivered to an OTA client using proprietary and / or standardized wireless communication technologies including, but not limited to: cellular mobile communications (e.g., 2G, 3G, 4G, 5G), radio wireless area networks (e.g., WiFi) and / or satellite Internet.

[0045] The term “edge node” refers to one or more edge devices coupled to a network that provide a portal for communicating with AVs and can communicate with other edge nodes and cloud-based computing platforms to schedule and deliver OTA updates to OTA clients.

[0046] The term "edge device" refers to a device that implements an edge node and provides a physical wireless access point (AP) to the core network of an enterprise or service provider (such as VERIZON, AT&T). Examples of edge devices include, but are not limited to: computers, controllers, transmitters, routers, routing switches, integrated access devices (IADs), multiplexers, metropolitan area networks (MANs), and wide area networks (WAN) access devices.

[0047] “One or more” includes a function performed by one element, a function performed by multiple elements, such as in a distributed manner, several functions performed by one element, several functions performed by several elements, or any combination of the above.

[0048] It will also be understood that although in some cases, the terms "first," "second," etc. are used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the various described embodiments, a first contact may be referred to as a second contact, and similarly, a second contact may be referred to as a first contact. Both the first contact and the second contact are contacts, but they are not the same contact.

[0049] The terms used in the specification of the various embodiments described herein are only used for the purpose of describing specific embodiments and are not intended to be limiting. As used in the specification of the various embodiments described and the appended claims, the singular forms "a", "an" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that "and / or" as used herein refers to and includes any and all possible combinations of one or more related list items. It will also be understood that when the terms "include", "comprises", "have" and / or "have" are used in this specification, the stated features, integers, steps, operations, elements and / or components are specifically stated, but the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof are not excluded.

[0050] As used herein, the term "if" is alternatively understood to mean "when" or "at the time" or "in response to determining that" or "in response to detecting," depending on the context. Similarly, the phrases "if it has been determined" or "if [the stated condition or event] has been detected" are alternatively understood to mean "upon determination" or "in response to determining that" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context.

[0051] As used herein, an AV system refers to an AV and an array of hardware, software, stored data, and real-time generated data that supports AV operation. In an embodiment, the AV system is incorporated into the AV. In an embodiment, the AV system is distributed across several locations. For example, some software of the AV system is implemented on a cloud computing environment.

[0052] In general, this document describes technologies applicable to any vehicle having one or more autonomous capabilities, including fully autonomous vehicles, highly autonomous vehicles, and conditionally autonomous vehicles, such as so-called Level 5, Level 4, and Level 3 vehicles, respectively (see SAE International Standard J3016: Classification and Definitions of Terms Relating to Automated Driving Systems for On-Road Motor Vehicles, which is incorporated by reference in its entirety into this document for more details on vehicle autonomy levels). The technologies described in this document are also applicable to partially autonomous vehicles and driver-assisted vehicles, such as so-called Level 2 and Level 1 vehicles (see SAE International Standard J3016: Classification and Definitions of Terms Relating to Automated Driving Systems for On-Road Motor Vehicles). In embodiments, one or more Level 1, Level 2, Level 3, Level 4, and Level 5 vehicle systems may automatically perform certain vehicle operations (e.g., steering, braking, and using maps) under certain operating conditions based on processing of sensor inputs. The technologies described in this document may benefit any level of vehicle ranging from fully autonomous vehicles to human-operated vehicles.

[0053] Autonomous vehicles have advantages over vehicles that require human drivers. One advantage is safety. For example, in 2016, the United States experienced 6 million automobile accidents, 2.4 million injuries, 40,000 deaths, and 13 million vehicle crashes, with an estimated social cost of more than $910 billion. From 1965 to 2015, the number of U.S. traffic fatalities has decreased from about 6 to about 1 per 100 million miles traveled, in part due to additional safety measures deployed in vehicles. For example, it is believed that an extra half-second of warning related to an impending collision mitigates 60% of front-to-rear collisions. However, passive safety features (e.g., seat belts, airbags) may have reached their limits in improving this number. Therefore, active safety measures such as automatic control of vehicles are a possible next step to improve these statistics. Because human drivers are believed to be the cause of severe pre-crash events in 95% of crashes, automated driving systems have the potential to achieve better safety outcomes by, for example: reliably identifying and avoiding emergency situations better than humans; making better decisions than humans, obeying traffic laws better than humans, and predicting future events better than humans; and reliably controlling a vehicle better than humans.

[0054] refer to Figure 1 , the AV system 120 causes the vehicle 100 to operate along a trajectory 198, through an environment 190 to a destination 199 (sometimes referred to as a final location), while avoiding objects (e.g., natural obstacles 191, vehicles 193, pedestrians 192, cyclists, and other obstacles) and complying with road rules (e.g., operating rules or driving preferences).

[0055] In an embodiment, the AV system 120 includes a device 101 equipped to receive and act upon operating commands from a computer processor 146. We use the term "operating command" to refer to an executable instruction (or set of instructions) that causes a vehicle to perform an action (e.g., a driving maneuver). Operating commands may include, but are not limited to, instructions for the vehicle to begin moving forward, stop moving forward, begin moving backward, stop moving backward, accelerate, decelerate, perform a left turn, and perform a right turn. In an embodiment, the computer processor 146 and the computer processor 146 are connected to the computer processor 146 and the computer processor 146. Figure 2 The processor 204 is similarly described. Examples of devices 101 include steering controls 102, brakes 103, gears, an accelerator pedal or other acceleration control mechanism, windshield wipers, side door locks, window controls, and turn indicators.

[0056] In an embodiment, the AV system 120 includes sensors 121 for measuring or inferring attributes of the state or condition of the vehicle 100, such as the position, linear and angular velocity and acceleration of the AV, and heading (e.g., the direction of the front end of the vehicle 100). Examples of sensors 121 are GPS, an inertial measurement unit (IMU) that measures both linear acceleration and angular rate of the vehicle, wheel rate sensors for measuring or estimating wheel slip, wheel brake pressure or brake torque sensors, engine torque or wheel torque sensors, and steering angle and angular rate sensors.

[0057] In an embodiment, the sensors 121 also include sensors for sensing or measuring properties of the environment of the AV, such as monocular or stereo cameras 122 in the visible, infrared, or thermal (or both) spectrum, LiDAR 123, RADAR, ultrasonic sensors, time-of-flight (ToF) depth sensors, velocity sensors, temperature sensors, humidity sensors, and precipitation sensors.

[0058] In an embodiment, the AV system 120 includes a data storage unit 142 and a memory 144 for storing machine instructions associated with a computer processor 146 or data collected by the sensor 121. In an embodiment, the data storage unit 142 is associated with the following Figure 2190 . In an embodiment, the memory 144 is similar to the main memory 206 described below. In an embodiment, the data storage unit 142 and the memory 144 store historical, real-time, and / or predictive information about the environment 190. In an embodiment, the stored information includes maps, driving performance, traffic congestion updates, or weather conditions. In an embodiment, data related to the environment 190 is transmitted from the remote database 134 to the vehicle 100 via a communication channel.

[0059] In an embodiment, the AV system 120 includes a communication device 140 for transmitting measured or inferred properties of the state and condition of other vehicles (such as position, linear and angular velocity, linear and angular acceleration, and linear and angular heading) to the vehicle 100. These devices include vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication devices and devices for wireless communication through point-to-point or ad hoc networks or both. In an embodiment, the communication device 140 communicates across the electromagnetic spectrum (including radio and optical communications) or other media (e.g., air and acoustic media). The combination of vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) communications (and in some embodiments, one or more other types of communications) is sometimes referred to as vehicle-to-everything (V2X) communications. V2X communications typically comply with one or more communication standards for communication with and between autonomous vehicles.

[0060] In an embodiment, the communication device 140 includes a communication interface. For example, a wired, wireless, WiMAX, Wi-Fi, Bluetooth, satellite, cellular, optical, near field, infrared, or radio interface. The communication interface transmits data from the remote database 134 to the AV system 120. In an embodiment, the remote database 134 is embedded in a cloud computing environment. The communication device 140 transmits data collected from the sensor 121 or other data related to the operation of the vehicle 100 to the remote database 134. In an embodiment, the communication device 140 transmits information related to teleoperation to the vehicle 100. In some embodiments, the vehicle 100 communicates with other remote (e.g., "cloud") servers 136.

[0061] In an embodiment, the remote database 134 also stores and transmits digital data (e.g., data such as roads and street locations). Such data is stored in a memory 144 on the vehicle 100 or transmitted from the remote database 134 to the vehicle 100 via a communication channel.

[0062] In an embodiment, the remote database 134 stores and transmits historical information (e.g., speed and acceleration profiles) related to driving attributes of the vehicle that has previously traveled along the trajectory 198 at similar times of day. In one implementation, such data may be stored in the memory 144 on the vehicle 100 or transmitted from the remote database 134 to the vehicle 100 via a communication channel.

[0063] A computer processor 146 located onboard the vehicle 100 algorithmically generates control actions based on both real-time sensor data and a priori information, allowing the AV system 120 to perform its autonomous driving capabilities.

[0064] In an embodiment, the AV system 120 includes a computer peripheral device 132 coupled to the computer processor 146 for providing information and reminders to a user of the vehicle 100 (e.g., an occupant or a remote user) and receiving input from the user. In an embodiment, the peripheral device 132 is similar to the one described below with reference to Figure 2 Display 212, input device 214 and cursor control 216 are discussed. The coupling may be wireless or wired. Any two or more of the interface devices may be integrated into a single device.

[0065] In an embodiment, the AV system 120 receives and enforces the privacy level of the occupant, for example, specified by the occupant or stored in a profile associated with the occupant. The privacy level of the occupant determines how to permit the use of specific information associated with the occupant (e.g., occupant comfort data, biometric data, etc.) stored in the occupant profile and / or stored on the cloud server 136 and associated with the occupant profile. In an embodiment, the privacy level specifies specific information associated with the occupant that is deleted once the ride is completed. In an embodiment, the privacy level specifies specific information associated with the occupant and identifies one or more entities authorized to access the information. Examples of designated entities authorized to access information may include other AVs, third-party AV systems, or any entity that can potentially access the information.

[0066] The privacy level of an occupant may be specified at one or more levels of granularity. In an embodiment, the privacy level identifies specific information to be stored or shared. In an embodiment, the privacy level applies to all information associated with the occupant, such that the occupant may specify that her personal information is not to be stored or shared. The designation of entities permitted to access specific information may also be specified at various levels of granularity. The various sets of entities permitted to access specific information may include, for example, other AVs, cloud servers 136, specific third-party AV systems, etc.

[0067] In an embodiment, the AV system 120 or the cloud server 136 determines whether the AV 100 or another entity may access certain information associated with the occupant. For example, a third-party AV system attempting to access occupant input related to a particular spatiotemporal location must obtain authorization, such as from the AV system 120 or the cloud server 136, to access information associated with the occupant. For example, the AV system 120 uses the occupant's specified privacy level to determine whether the occupant input related to the spatiotemporal location can be presented to a third-party AV system, the AV 100, or another AV. This enables the occupant's privacy level to specify which other entities are allowed to receive data related to the occupant's actions or other data associated with the occupant.

[0068] Figure 2 Computer system 200 is illustrated. In implementation, computer system 200 is a special purpose computing device. The special purpose computing device is hardwired to perform these techniques, or includes a digital electronic device such as one or more application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs) that are permanently programmed to perform the above-mentioned techniques, or may include one or more general purpose hardware processors that are programmed to perform these techniques according to program instructions in firmware, memory, other memory, or a combination. Such a special purpose computing device can also combine customized hardwired logic, ASICs or FPGAs with customized programming to complete these techniques. In various embodiments, the special purpose computing device is a desktop computer system, a portable computer system, a handheld device, a network device, or any other device that includes hardwired and / or program logic to implement these techniques.

[0069] In an embodiment, the computer system 200 includes a bus 202 or other communication mechanism for communicating information, and a processor 204 coupled to the bus 202 to process information. The processor 204 is, for example, a general-purpose microprocessor. The computer system 200 also includes a main memory 206, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 202 to store information and instructions, which are executed by the processor 204. In one implementation, the main memory 206 is used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 204. When these instructions are stored in a non-transitory storage medium accessible to the processor 204, the computer system 200 becomes a special-purpose machine that is customized to perform the operations specified in the instructions.

[0070] In an embodiment, computer system 200 also includes a read only memory (ROM) 208 or other static storage device coupled to bus 202 for storing static information and instructions for processor 204. A storage device 210, such as a magnetic disk, optical disk, solid state drive, or three-dimensional cross point memory, is provided and coupled to bus 202 to store information and instructions.

[0071] In an embodiment, the computer system 200 is coupled to a display 212 such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, a light emitting diode (LED) display, or an organic light emitting diode (OLED) display for displaying information to a computer user via a bus 202. An input device 214 including alphanumeric and other keys is coupled to the bus 202 for communicating information and command selections to the processor 204. Another type of user input device is a cursor controller 216, such as a mouse, a trackball, a touch display, or cursor direction keys, for communicating direction information and command selections to the processor 204 and for controlling movement of a cursor on the display 212. Such input devices typically have two degrees of freedom in two axes, a first axis (e.g., an x-axis) and a second axis (e.g., a y-axis) that allow the device to specify a position on a plane.

[0072] According to one embodiment, the techniques herein are performed by computer system 200 in response to processor 204 executing one or more sequences of one or more instructions contained in main memory 206. These instructions are read into main memory 206 from another storage medium, such as storage device 210. Execution of the sequences of instructions contained in main memory 206 causes processor 204 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry is used in place of or in combination with software instructions.

[0073] As used herein, the term "storage medium" refers to any non-temporary medium that stores data and / or instructions that cause a machine to operate in a particular manner. Such storage media include non-volatile media and / or volatile media. Non-volatile media include, for example, optical disks, magnetic disks, solid-state drives, or three-dimensional cross-point memories such as storage device 210. Volatile media include dynamic memories such as main memory 206. Common forms of storage media include, for example, floppy disks, floppy disks, hard disks, solid-state drives, magnetic tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with a hole pattern, RAM, PROM and EPROM, FLASH-EPROM, NV-RAM, or any other memory chip or storage box.

[0074] Storage media are distinct from transmission media, but can be used in conjunction with transmission media. Transmission media participate in the transmission of information between storage media. For example, transmission media include coaxial cables, copper wires, and optical fibers, including wires that provide bus 202. Transmission media can also take the form of sound waves or light waves, such as those generated during radio wave and infrared data communications.

[0075] In an embodiment, various forms of media are involved in carrying one or more sequences of one or more instructions to the processor 204 for execution. For example, the instructions are initially executed on a disk or solid-state drive of a remote computer. The remote computer loads the instructions into its dynamic memory and sends the instructions over a telephone line using a modem. The local modem of the computer system 200 receives the data on the telephone line and uses an infrared transmitter to convert the data to an infrared signal. An infrared detector receives the data carried in the infrared signal, and appropriate circuitry places the data on the bus 202. The bus 202 carries the data to the main memory 206, from which the processor 204 retrieves and executes the instructions. The instructions received by the main memory 206 may optionally be stored on the storage device 210 before or after execution by the processor 204.

[0076] Computer system 200 also includes a communication interface 218 coupled to bus 202. Communication interface 218 provides a two-way data communication coupled to a network link 220 connected to a local network 222. For example, communication interface 218 is an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem for providing a data communication connection with a corresponding type of telephone line. As another example, communication interface 218 is a local area network (LAN) card for providing a data communication connection with a compatible LAN. In some implementations, a wireless link is also implemented. In any such implementation, communication interface 218 sends and receives an electrical, electromagnetic or optical signal carrying a digital data stream representing various types of information.

[0077] The network link 220 typically provides data communication to other data devices through one or more networks. For example, the network link 220 provides a connection to a host computer 224 or to a cloud data center or device operated by an Internet Service Provider (ISP) 226 through a local network 222. The ISP 226, in turn, provides data communication services through a worldwide packet data communication network now commonly referred to as the "Internet" 228. Both the local network 222 and the Internet 228 use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link 220 and through the communication interface 218 are example forms of transmission media, where these signals carry digital data to and from the computer system 200. In an embodiment, the network 220 comprises a cloud or a portion of a cloud.

[0078] Computer system 200 sends messages and receives data, including program code, through network(s), network link 220, and communication interface 218. In an embodiment, computer system 200 receives code for processing. The received code is executed by processor 204 as it is received, and / or stored in storage device 210, or other non-volatile storage for later execution.

[0079] Autonomous Vehicle Architecture

[0080] Figure 3 An example of a method for autonomous vehicles (e.g., Figure 1 100). The architecture 300 includes a perception module 302 (sometimes referred to as a perception circuit), a planning module 304 (sometimes referred to as a planning circuit), a control module 306 (sometimes referred to as a control circuit), a positioning module 308 (sometimes referred to as a positioning circuit), and a database module 310 (sometimes referred to as a database circuit). Each module plays a role in the operation of the vehicle 100. Collectively, the modules 302, 304, 306, 308, and 310 may be Figure 1 10 is a portion of the AV system 120 shown. In some embodiments, any of modules 302, 304, 306, 308, and 310 is a combination of computer software (e.g., executable code stored on a computer-readable medium) and computer hardware (e.g., one or more microprocessors, microcontrollers, application specific integrated circuits [ASICs], hardware memory devices, other types of integrated circuits, other types of computer hardware, or any or all combinations of these). Each of modules 302, 304, 306, 308, and 310 is sometimes referred to as a processing circuit (e.g., computer hardware, computer software, or a combination of both). A combination of any or all of modules 302, 304, 306, 308, and 310 is also an example of a processing circuit.

[0081] In use, planning module 304 receives data representing destination 312 and determines data representing trajectory 314 (sometimes referred to as a route) that vehicle 100 may travel in order to reach (e.g., arrive at) destination 312. In order for planning module 304 to determine data representing trajectory 314, planning module 304 receives data from perception module 302, positioning module 308, and database module 310.

[0082] The perception module 302 uses, for example, Figure 1 One or more sensors 121 are shown to identify nearby physical objects. Objects are classified (e.g., grouped into types such as pedestrians, bicycles, cars, traffic signs, etc.), and a scene description including the classified objects 316 is provided to the planning module 304.

[0083] The planning module 304 also receives data representing the AV position 318 from the positioning module 308. The positioning module 308 determines the AV position by using data from the sensor 121 and data from the database module 310 (e.g., geographic data) to calculate the position. For example, the positioning module 308 uses data from a GNSS (Global Navigation Satellite System) sensor and geographic data to calculate the longitude and latitude of the AV. In an embodiment, the data used by the positioning module 308 includes a high-precision map with lane geometry attributes, a map describing the road network connection attributes, a map describing the physical attributes of the lane (such as traffic speed, traffic volume, the number of vehicles and bicycle lanes, lane width, lane traffic direction, or lane marking type and location, or a combination thereof), and a map describing the spatial location of road features (such as intersections, traffic signs, or various types of other driving signals, etc.). In an embodiment, the high-precision map is constructed by adding data to the low-precision map via automatic or manual annotations.

[0084] The control module 306 receives data representing the trajectory 314 and data representing the AV position 318, and operates the control functions 320a-320c of the AV (e.g., steering, throttle, brakes, and ignition) in a manner that will cause the vehicle 100 to travel the trajectory 314 to reach the destination 312. For example, if the trajectory 314 includes a left turn, the control module 306 will operate the control functions 320a-320c in the following manner: the steering angle of the steering function will cause the vehicle 100 to turn left, and the throttle and brakes will cause the vehicle 100 to pause and wait for a passing pedestrian or vehicle before making the turn.

[0085] AV Input

[0086] Figure 4 The perception module 302 ( Figure 3 ) used by the inputs 402a-402d (e.g., Figure 1 1) and outputs 404a-404d (e.g., sensor data). One input 402a is a light detection and ranging (LiDAR) system (e.g., Figure 1 123). LiDAR is a technology that uses light (e.g., a beam of light such as infrared light) to obtain data about physical objects in its line of sight. The LiDAR system produces LiDAR data as output 404a. For example, LiDAR data is a collection of 3D or 2D points (also called a point cloud) used to construct a representation of the environment 190.

[0087] Another input 402b is a RADAR (radar) system. RADAR is a technology that uses radio waves to obtain data related to nearby physical objects. RADAR can obtain data related to objects that are not within the line of sight of the LiDAR system. The RADAR system produces RADAR data as output 404b. For example, RADAR data is one or more radio frequency electromagnetic signals used to construct a representation of the environment 190.

[0088] Another input 402c is a camera system. The camera system uses one or more cameras (e.g., a digital camera using a light sensor such as a charge coupled device [CCD]) to obtain information about nearby physical objects. The camera system generates camera data as output 404c. The camera data is typically in the form of image data (e.g., data in an image data format such as RAW, JPEG, PNG, etc.). In some examples, the camera system has multiple independent cameras, such as for the purpose of stereoscopic imaging (stereoscopic vision), which enables the camera system to perceive depth. Although the objects perceived by the camera system are described here as "nearby", this is relative to the AV. In some embodiments, the camera system is configured to "see" distant objects (e.g., as far as 1 km or more in front of the AV). Therefore, in some embodiments, the camera system has features such as sensors and lenses that are optimized for perceiving distant objects.

[0089] Another input 402d is a traffic light detection (TLD) system. The TLD system uses one or more cameras to obtain information about traffic lights, street signs, and other physical objects that provide visual navigation information. The TLD system generates TLD data as an output 404d. TLD data often takes the form of image data (e.g., data in image data formats such as RAW, JPEG, PNG, etc.). The difference between a TLD system and a system including a camera is that the TLD system uses a camera with a wide field of view (e.g., using a wide-angle lens or a fisheye lens) to obtain information about as many physical objects that provide visual navigation information as possible, so that the vehicle 100 can access all relevant navigation information provided by these objects. For example, the viewing angle of the TLD system is about 120 degrees or greater.

[0090] In some embodiments, sensor fusion techniques are used to combine the outputs 404a-404d. Thus, the individual outputs 404a-404d are provided to other systems of the vehicle 100 (e.g., to Figure 3The combined output may be provided to other systems in the form of a single combined output or multiple combined outputs of the same type (e.g., using the same combining technique or combining the same outputs, or both) or a single combined output or multiple combined outputs of different types (e.g., using different individual combining techniques or combining different individual outputs, or both). In some embodiments, an early fusion technique is used. An early fusion technique is characterized in that the outputs are combined before one or more data processing steps are applied to the combined output. In some embodiments, a late fusion technique is used. A late fusion technique is characterized in that the outputs are combined after one or more data processing steps are applied to the individual outputs.

[0091] Figure 5 An example of a LiDAR system 502 is shown (eg, Figure 4 4a). The LiDAR system 502 emits light 504a-504c from a light emitter 506 (e.g., a laser transmitter). The light emitted by the LiDAR system is typically not in the visible spectrum; for example, infrared light is often used. Some of the emitted light 504b encounters a physical object 508 (e.g., a vehicle) and reflects back to the LiDAR system 502. (The light emitted from the LiDAR system typically does not penetrate a physical object, such as a physical object that is solid in form.) The LiDAR system 502 also has one or more light detectors 510 for detecting the reflected light. In an embodiment, one or more data processing systems associated with the LiDAR system generate an image 512 representing a field of view 514 of the LiDAR system. The image 512 includes information representing the boundaries 516 of the physical object 508. As such, the image 512 is used to determine the boundaries 516 of one or more physical objects in the vicinity of the AV.

[0092] Figure 6 604. The LiDAR system 502 is shown in operation. In the scenario shown in the figure, the vehicle 100 receives both the camera system output 404c in the form of an image 602 and the LiDAR system output 404a in the form of LiDAR data points 604. In use, the data processing system of the vehicle 100 compares the image 602 with the data points 604. In particular, the physical objects 606 identified in the image 602 are also identified in the data points 604. In this way, the vehicle 100 perceives the boundaries of the physical objects based on the contours and density of the data points 604.

[0093] Figure 7 5 shows additional details of the operation of the LiDAR system 502. As described above, the vehicle 100 detects the boundaries of the physical object based on the characteristics of the data points detected by the LiDAR system 502. Figure 7As shown, a flat object such as the ground 702 will reflect light 704a-704d emitted from the LiDAR system 502 in a consistent manner. In other words, because the LiDAR system 502 emits light using consistent intervals, the ground 702 will reflect light back to the LiDAR system 502 at the same consistent intervals. As the vehicle 100 travels over the ground 702, the LiDAR system 502 will continue to detect light reflected by the next valid ground point 706 if nothing is blocking the road. However, if an object 708 is blocking the road, the light 704e-704f emitted by the LiDAR system 502 will be reflected from points 710a-710b in a manner that is inconsistent with the expected consistent manner. Based on this information, the vehicle 100 can determine that the object 708 is present.

[0094] Scan Line Overview

[0095] As mentioned above, for example, Figure 6 and 7 , a vehicle such as an autonomous vehicle uses LiDAR to collect data related to the vehicle's surroundings. In an embodiment, the LiDAR scan includes a plurality of scan lines (e.g., 20 scan lines, 40 scan lines, etc.). Each line is associated with a LiDAR system (e.g., Figure 4 Corresponding to a planar representation of the environment in which the LiDAR system 402a) is located.

[0096] Figure 8 An example LiDAR scan 800 is depicted having a plurality of scan lines 805a, 805b, 805c, 805d (collectively, scan lines 805) in accordance with various embodiments. As previously discussed, while only four scan lines 805 are depicted in the LiDAR scan 800, in other embodiments, the LiDAR scan 800 may include more or fewer scan lines. For example, in one embodiment, the LiDAR scan 800 may include 20 to 40 scan lines, while in other embodiments, the LiDAR scan 800 includes more or fewer scan lines, depending on factors such as the processing power of the LiDAR system, the rate at which scans are being performed, the acceptable quality of the scans, or some other factor. Additionally, while the scan lines 805 are generally depicted as being horizontal, in other embodiments, the scan lines 805 are rotated so that they are vertical or at some angle between horizontal and vertical. Additionally, the scan lines 805 are depicted as being separated by a certain vertical distance (relative to the vertical). Figure 8 ), however, in real-world implementations, the distances between scan lines 805 may be greater or less than the distances depicted. Figure 8 It should be construed as a high-level example for the purpose of illustrating and discussing the concepts herein and not be considered a limiting example of a real-world implementation.

[0097] Each of scan lines 805 includes a plurality of LiDAR data points 810, which are similar to data points 604, for example. Figure 8 In the depicted embodiment, each scan line 805 is shown as including sixteen LiDAR data points 810. However, it will be understood that other embodiments will include more or fewer LiDAR data points than depicted.

[0098] Each of the LiDAR data points 810 is associated with data provided to the LiDAR system. Specifically, each of the LiDAR data points 810 provides TOF data to the LiDAR system. The TOF data relates to the length between the time a light signal leaves the LiDAR system's transmitter and the time a reflected light signal is received by the LiDAR system's receiver. By using this TOF data for each of the LiDAR data points 810 for the scan line 805 of the LiDAR scan 800, the LiDAR system is able to construct or facilitate construction of a three-dimensional (3D) image of objects surrounding the LiDAR system (e.g., as described above with respect to Figure 4 and 5 as described).

[0099] Smoothness factor example

[0100] As previously described, the results of a LiDAR scan (e.g., LiDAR scan 800) are used by the autonomous vehicle for a positioning process. That is, in an embodiment, the results of the LiDAR scan 800 are compared with pre-identified information of a geographic location. Based on this comparison, the autonomous vehicle is able to identify the location where the autonomous vehicle is currently located.

[0101] However, in some cases, the localization process is complicated by inconsistent or transient objects or structures. For example, the shape of some vegetation, such as bushes or trees, changes with seasons, pruning, etc. If other transient objects, such as people, bicycles, etc., move or change shape, then such objects will also produce inconsistent results within the LiDAR scan, making the scan of a location at one time different from the scan of the same location at a different time.

[0102] In contrast, more consistent or permanent (e.g., non-transient) structures will provide more consistent results during the positioning process. Such objects include man-made structures such as buildings, roadblocks, walls, etc. Because these objects are man-made, they often have relatively flat or smooth outlines compared to the outlines of objects such as trees (where leaves or branches may generate significant changes within a LiDAR scan). Therefore, in order to increase the consistency of the results of the LiDAR scan, embodiments of the present invention remove data points from the LiDAR scan that are identified as belonging to transient structures such as vegetation, the presence of a person or bicycle, etc.

[0103] One way to remove a data point from a LiDAR scan is to analyze the data point in conjunction with other data points around the data point. Specifically, in an embodiment, the data point is analyzed and a smoothness coefficient c is calculated. The smoothness coefficient c is then compared to a threshold. If the smoothness coefficient c is greater than (or optionally greater than or equal to) the threshold, the LiDAR system (or a processor coupled thereto) identifies that the data point belongs to an area with a relatively high degree of variability, and therefore identifies that the data point is associated with a transient object. In some embodiments, the LiDAR system may then determine that the data point should be removed from the scan line. If the smoothness coefficient c is less than (or optionally less than or equal to) the threshold, the LiDAR system (or a processor coupled thereto) identifies that the data point belongs to an area with a relatively low degree of variability, and therefore should be retained in the scan line (or included in the output "smoothed" scan line).

[0104] Equation 1 presents the point Example equation for calculating the smoothness coefficient c.

[0105]

[0106] Specifically, X L refers to a data point in a LiDAR scan such as LiDAR scan 800. k is an identifier for a given scan line. In Equation 1, the data point in question is replaced by is compared to the number of data points S. i and j can refer to specific locations within the scan line. Specifically, The data associated with the data point in question is compared with the data associated with each other data point in S. Based on this comparison, a smoothness coefficient c is calculated. Specifically, the data point in question is calculated or identified. and the vectors between each other point in S. The vectors are then summed to produce the smoothness coefficient c.

[0107] It will be noted that in one embodiment, various vectors between the data point in question and various other data points are calculated to generate the coefficients c. However, in other embodiments, one or more vectors may have been pre-calculated and stored so that the vectors can be identified rather than recalculated. For example, in one embodiment, based on a data point in S and the data point currently in question The vector between has already calculated the coefficient c for that one data point. Thus, recalculation of the vector may not be necessary, and instead the vector may be identified as previously calculated.

[0108] Fig. 9Depicting a graphical example of an algorithm for calculating a smoothness coefficient associated with data points of a LiDAR scan, in accordance with an embodiment. Fig.10 Depicting an alternative graphical example of an algorithm for calculating a smoothness coefficient associated with data points of a LiDAR scan, in accordance with an embodiment.

[0109] Specifically, in Fig. 9 and 10 In , an X-axis X and a Y-axis Y are depicted. In general, it can be assumed that the LiDAR system is located at the intersection of the X-axis and the Y-axis. Fig. 9 and 10 Multiple LiDAR data points are depicted in Figure 8 The LiDAR data point 810 is similar.

[0110] Specifically, Fig. 9 LiDAR data points 905a, 905b, 905c, 905d, and 905e (collectively referred to as LiDAR data points 905) are depicted. Fig.10 Depicted are LiDAR data points 1005a, 1005b, 1005c, 1005d, and 1005e (collectively referred to as LiDAR data points 1005). Fig. 9 and 10 In the depiction of FIG. 1 , LiDAR data points 905 and 1005 are based on TOF data associated with each data point, where Fig. 9 and 10 Represents a top-down view of TOF data relative to where the LiDAR system might be.

[0111] Go to Fig. 9 , LiDAR data point 905c is the data point under current analysis and is consistent with Eq. In this example, S is equal to 5.

[0112] According to Equation 1, vectors 910a, 910b, 910d, and 910e (collectively, vector 910) are calculated between data point 905c and each of the other data points 905a, 905b, 905d, and 905e. Vectors 910 are then summed. Fig. 9 910a and 910e may cancel each other. Similarly, vectors 910b and 910d may cancel each other. Thus, coefficient c is approximately 0, which indicates that data point 905c is "smooth" and should therefore be retained in the scan line or included in an updated scan line. It will be noted that the indication that data point 905c is "smooth" is consistent with the indication that data point 905c is "smooth". Fig. 9 The visual depiction of the group of data points 905 in FIG.

[0113] In contrast, Fig.10 A group S of LiDAR data points 1005 is depicted, where LiDAR data point 1005c is considered to have a higher coefficient c than the coefficient c of data point 905c. Fig. 9 , at data point 1005c and Fig.10 A plurality of vectors 1010a, 1010b, 1010d, and 1010e (collectively referred to as vectors 1010) are calculated between the other data points of . Then, as discussed above with respect to Equation 1, vectors 1010 are summed to calculate coefficients c.

[0114] The horizontal component (e.g., the component along the X-axis) of each vector 1010 is the same as described above with respect to Fig. 9 The vectors 1010a and 1010e are shown in Figure 10. The vectors 1010b and 1010d are shown in Figure 10. The vectors 1010a and 1010e are shown in Figure 10. The vectors 1010b and 1010d are shown in Figure 10.

[0115] However, each vector 1010 also has a vertical component (eg, a component along the Y axis) that is not cancelled. Thus, the sum of vectors 1010 is greater than zero, resulting in a non-zero c.

[0116] The description of how to calculate the smoothness coefficient c can be considered a simplified example of one embodiment, and other embodiments may vary. For example, the specific equation 1 is intended as an example, and other variations can be based on different equations for calculating the smoothness coefficient (such as c). As an example variation, the set of data points S can include data points from multiple scan lines, rather than about equation 1 or Fig. 9 or 10 to describe a single scan line.

[0117] Use of smoothness factor

[0118] Fig.11 An example technique for updating LiDAR scan lines according to various embodiments is depicted. In general, the technique involves the use of a smoothness coefficient c as described above. The technique may be performed by, for example, a LiDAR system 402a ( Figure 4 ) or the like, such as a LiDAR system, such as a processor 204 ( Figure 2 ) or the like, such as a control module 306 ( Figure 3 ) and other control modules, such as positioning module 308 ( Figure 3 ) or the like, a partial combination thereof, or at least one additional element of an autonomous vehicle.

[0119] Initially, a scan line in a LiDAR scan is identified at 1105. The scan line is, for example, similar to one of LiDAR scan lines 805. The technique also includes identifying a LiDAR data point based on the scan line at 1110. The LiDAR data point is, for example, similar to LiDAR data point 905 or 1005 and is based on LiDAR data point 810 in one or more scan lines 805.

[0120] The technique also includes identifying a target LiDAR data point and neighboring LiDAR data points at 1115. The target LiDAR data point is, for example, The corresponding LiDAR data point is 905c or 1005c. The adjacent LiDAR data point is, for example, LiDAR data point 905a / 905b / 905d / 905e / 1005a / 1005b / 1005d / 1005e corresponding to other data points in S.

[0121] The technique also includes calculating a smoothness coefficient for the target LiDAR data point at 1120. The smoothness coefficient is, for example, the coefficient c calculated according to Equation 1 as described above. In other embodiments, additionally or alternatively, the smoothness coefficient is a different coefficient calculated according to a different equation or according to at least one other variable.

[0122] The technique also includes comparing the smoothness coefficient (e.g., c) to a threshold at 1125. In one embodiment, the comparison is performed based on an identification of whether the smoothness coefficient is greater than (or greater than or equal to) the threshold. In other embodiments, the comparison is based on an identification of whether the smoothness coefficient is less than (or less than or equal to) the threshold. In other embodiments, the comparison is a different type of comparison.

[0123] In one embodiment, the threshold is a pre-identified threshold. That is, the threshold is pre-identified based on previous testing or some other factor. In other embodiments, the threshold is at least partially dynamic. For example, the threshold is based on an analysis of other data points in a given LiDAR scan, other data points in a scan line, previously calculated coefficients, etc.

[0124] If the coefficient is identified at 1125 as being less than (or in an embodiment less than or equal to) the threshold, the particular LiDAR data point (e.g., ). As used herein, discarding a data point refers to removing a particular LiDAR data point from a scanline or LiDAR scan used for positioning purposes. Conversely, if a coefficient is identified as being greater than (or greater than or equal to) a threshold value, the LiDAR data point is included in the scanline processed by a positioning module (e.g., positioning module 408). Such a scanline is referred to as an "updated" scanline. In embodiments, an updated scanline is an existing scanline with the discarded data point removed. In other embodiments, the updated scanline is a new scanline being generated based on the included data points.

[0125] More specifically, in embodiments, if the coefficient is less than (or less than or equal to) a threshold, the LiDAR data point may be discarded from the scanline, while in other embodiments, the LiDAR data point may be marked for removal from the scanline, for example, in a batch operation. In other embodiments, if the coefficient is greater than (or greater than or equal to) the threshold, the LiDAR data point may not be removed from the scanline, and instead may be retained in the scanline. In other embodiments, the LiDAR data point may be included in a new iteration of the scanline based on the LiDAR data points marked as having an acceptable smoothness coefficient (e.g., a coefficient greater than or greater than or equal to the threshold).

[0126] Fig.12 Describes an alternative example technique for updating LiDAR scan lines according to an embodiment. In general, Fig.12 Considered to be Fig.11 are complementary and include similar elements. Fig.11 , which can be accomplished by, for example, a LiDAR system 402a ( Figure 4 ) or the like, such as a LiDAR system, such as a processor 204 ( Figure 2 ) or the like, such as a control module 306 ( Figure 3 ) and other control modules, such as positioning module 308 ( Figure 3 ) or the like, a partial combination thereof, or at least one additional element of an autonomous vehicle.

[0127] The technique includes identifying a first LiDAR data point in a LiDAR scan line at 1205. The LiDAR scan line is, for example, similar to one of LiDAR scan lines 805. The LiDAR data point is, for example, similar to LiDAR data points 810, 905, or 1005. More specifically, the first LiDAR data point is similar to one of LiDAR data points 905c or 1005c.

[0128] The technique also includes identifying, at 1210, a plurality of LiDAR data points in the LiDAR scan line within a vicinity of the first LiDAR data point. The plurality of LiDAR data points are, for example, other data points in S and include data points 905a / 905b / 905d / 905e / 1005a / 1005b / 1005d / 1005e. As previously described, in embodiments, all of the plurality of LiDAR data points are in the same scan line as the first LiDAR data point, while in other embodiments, at least one of the plurality of LiDAR data points is in a different scan line than the first LiDAR data point.

[0129] The technique also includes identifying a coefficient for the first LiDAR data point based on a comparison of the first LiDAR data point to the plurality of LiDAR data points at 1215, wherein the coefficient is related to image smoothness. The comparison is, for example, the comparison described above with respect to equation 1, and the coefficient is c. However, as previously described, in other embodiments, the comparison is based on a different equation or some other type of comparison of at least one additional or alternative factor.

[0130] The technique also includes: identifying whether to include a LiDAR echo point in the updated LiDAR scan line based on a comparison of the coefficient with a threshold at 1220. The threshold is, for example, the threshold described above with respect to element 1125. Specifically, the threshold is a predetermined threshold or a dynamic threshold as described above. In addition, as described above, the comparison of the threshold is to identify whether the coefficient is greater than (or greater than or equal to) the threshold or less than (or less than or equal to) the threshold.

[0131] If the coefficient is identified as being greater than (or greater than or equal to) the threshold, the first data point is discarded from the updated scan line. Conversely, if the coefficient is identified as being less than (or less than or equal to) the threshold, the first data point is included in the updated scan line. As previously described, in one embodiment, including in the updated scan line includes not removing the data point from the existing scan line. In other embodiments, including in the updated scan line is based on including the data point in a new scan line being created.

[0132] The technique then includes identifying the location of the AV based on the updated LiDAR scan line at 1225. Specifically, as described above with respect to the localization module 308, the localization module 308 determines the AV location by using data from the sensor 121 (e.g., updated scan lines) and data from the database module 310 (e.g., geographic data) to calculate the location. As described above, by removing data points associated with transient elements such as trees, people, etc., the localization module 308 will then identify the location of the AV based on non-transient elements such as buildings, roadways, etc. As a result, the consistency of the localization module 308 will be increased, thereby increasing the overall efficiency of the navigation of the AV.

[0133] It will be understood that Fig.11 and 12 The described techniques are intended to be high-level examples of techniques, and other embodiments will include variations of these examples. For example, various embodiments have more or fewer elements than those depicted in the example techniques, or have elements in a different arrangement or order than depicted. Other variations will exist in other embodiments.

[0134] In the previous description, embodiments of the present invention have been described with reference to many specific details, which may vary depending on implementation. Therefore, the specification and the accompanying drawings should be considered illustrative, rather than restrictive. The only and exclusive indication of the scope of the present invention, and the applicant's expectation that the content of the scope of the present invention is the literal and equivalent scope of the claims announced according to the present application in the specific form of the authorization announcement claims, including any subsequent amendments. Any definition of the terms clearly set forth herein for being included in such claims should be based on the meaning of such terms as used in the claims. In addition, when the term "also includes" is used in the previous specification or the attached claims, the following of the phrase can be an additional step or entity, or a sub-step / sub-entity of the previously described step or entity.

Claims

1. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors of a vehicle, cause the vehicle to: Identifying a first LiDAR data point in a light detection and ranging scan line, i.e., a LiDAR scan line; identifying, in the LiDAR scan line, a plurality of LiDAR data points within a region proximate to the first LiDAR data point; identifying a coefficient of the first LiDAR data point based on a comparison of the first LiDAR data point to at least one LiDAR data point of the plurality of LiDAR data points, wherein the coefficient is related to image smoothness; comparing the coefficient to a threshold value; If the coefficient is equal to or greater than the threshold, the instructions cause the vehicle to remove the first LiDAR data point from the LiDAR scan line to obtain an updated LiDAR scan line; as well as The location of the vehicle is identified based on the updated LiDAR scan lines.

2. One or more non-transitory computer-readable media according to claim 1, in, If the coefficient is below the threshold, the instructions cause the vehicle to include the first LiDAR data point into the updated LiDAR scan line.

3. One or more non-transitory computer-readable media according to claim 1, in, The threshold is a predetermined threshold associated with the presence of a transient element.

4. One or more non-transitory computer-readable media according to claim 1, in, The identification of the coefficients is based on a comparison of vectors between the first LiDAR data point and respective ones of the plurality of LiDAR data points.

5. One or more non-transitory computer readable media according to claim 4, in, The instructions further cause the vehicle to: generating a vector between the first LiDAR data point and a second LiDAR data point in the plurality of LiDAR data points; as well as A previously determined vector between the first LiDAR data point and a third LiDAR data point in the plurality of LiDAR data points is identified.

6. One or more non-transitory computer readable media according to claim 5, in, At least one of the second LiDAR data point and the third LiDAR data point is adjacent to the first LiDAR data point in the LiDAR scan line.

7. A method for a vehicle, include: identifying, by at least one processor, a first LiDAR data point in a light detection and ranging scan line (LiDAR scan line), a second LiDAR data point in the LiDAR scan line adjacent to the first LiDAR data point, and a third LiDAR data point in the LiDAR scan line adjacent to the first LiDAR data point; identifying, by the at least one processor, a coefficient for the first LiDAR data point based on a comparison of the first LiDAR data point with the second LiDAR data point and the third LiDAR data point, wherein the coefficient is related to image smoothness; comparing, by the at least one processor, the coefficient to a threshold; When the coefficient is equal to or greater than the threshold, removing, by the at least one processor, the first LiDAR data point from the LiDAR scan line to obtain an updated LiDAR scan line; as well as The location of the vehicle is identified based on the updated LiDAR scan lines.

8. The method according to claim 7, further comprising: include: In a case where the coefficient is below the threshold, the first LiDAR data point is included, by the at least one processor, into the updated LiDAR scanline.

9. The method according to claim 7, in, The threshold is a predetermined threshold associated with the presence of a transient element.

10. The method according to claim 7, in, Identifying the coefficients includes: comparing, by the at least one processor, a vector between the first LiDAR data point and the second LiDAR data point to identify a first comparison value; comparing, by the at least one processor, a vector between the first LiDAR data point and the third LiDAR data point to identify a second comparison value; and The coefficient is calculated, by the at least one processor, based on the first comparison value and the second comparison value.

11. The method according to claim 10, in, Comparing the vector between the first LiDAR data point and the second LiDAR data point includes calculating, by the at least one processor, the vector between the first LiDAR data point and the second LiDAR data point.

12. The method according to claim 10, in, Comparing a vector between the first LiDAR data point and the second LiDAR data point includes identifying, by the at least one processor, a pre-identified vector between the first LiDAR data point and the second LiDAR data point.

13. A vehicle, include: A light detection and ranging system, i.e., a LiDAR system, for generating one or more LiDAR scan lines including a plurality of LiDAR data points; as well as At least one processor coupled to the LiDAR system, the at least one processor configured to: identifying, among the plurality of LiDAR data points, a first LiDAR data point, a second LiDAR data point in the LiDAR scan line adjacent to the first LiDAR data point, and a third LiDAR data point in the LiDAR scan line adjacent to the first LiDAR data point; identifying a coefficient of the first LiDAR data point based on a comparison of the first LiDAR data point with the second LiDAR data point and the third LiDAR data point, wherein the coefficient is related to image smoothness; comparing the coefficient to a threshold value; as well as When the coefficient is equal to or higher than the threshold, the first LiDAR data point is removed from the LiDAR scan line to obtain an updated LiDAR scan line.

14. The vehicle according to claim 13, in, In a case where the coefficient is below the threshold, the at least one processor adds the first LiDAR data point to the updated LiDAR scanline.

15. The vehicle according to claim 13, in, Identifying the coefficients includes: identifying a first comparison value based on a comparison of a vector between the first LiDAR data point and the second LiDAR data point; identifying a second comparison value based on a comparison of a vector between the first LiDAR data point and the third LiDAR data point; and The coefficient is calculated based on the first comparison value and the second comparison value.

16. The vehicle according to claim 15, in, The at least one processor calculates a vector between the first LiDAR data point and the second LiDAR data point.

17. The vehicle according to claim 15, in, The at least one processor identifies a pre-computed vector between the first LiDAR data point and the second LiDAR data point.

Images