Filtering return points in a point cloud based on radial velocity measurements

Abstract

Aspects and implementations of the present disclosure relate to filtering return points from a point cloud based on radial velocity measurements. An example method includes receiving, by a sensing system of an autonomous vehicle (AV), data representing a point cloud comprising a plurality of return points, each return point comprising a radial velocity value and a position coordinate representing a location of a reflection region reflecting a transmitted signal transmitted by the sensing system; applying at least one threshold condition related to the radial velocity value of a given return point to each of the plurality of return points to identify a subset of return points within the plurality of return points; removing the subset of return points from the point cloud to generate a filtered point cloud; and identifying an object represented by remaining return points in the filtered point cloud.

Description

Technical Field

[0001] This disclosure generally relates to autonomous vehicles. More specifically, this disclosure relates to improving object classification and tracking in an autonomous driving environment. Background Technology

[0002] Autonomous (fully and partially automated) vehicles (VAs) operate by sensing their external environment using various electromagnetic (e.g., radar and optical) and non-electromagnetic (e.g., audio and humidity) sensors. Some VAs map driving paths through their environment based on the sensed data. These driving paths can be determined using Global Positioning System (GPS) data and road map data. GPS and road map data provide information about static aspects of the environment (buildings, street layouts, road closures, etc.), while dynamic information (such as information about other vehicles, pedestrians, streetlights, etc.) is obtained from the sensed data collected in real-time. The accuracy and safety of the driving path and speed state selected by an VA largely depend on the timely and accurate identification of various objects present in the driving environment, and the ability of driving algorithms to process environmental information quickly and efficiently and provide correct instructions to the vehicle controller and drivetrain. Attached Figure Description

[0003] This disclosure is illustrated by way of example rather than limitation, and can be more fully understood by referring to the following detailed description when considered in conjunction with the accompanying drawings.

[0004] Figure 1 This is a diagram illustrating components of an example autonomous vehicle using Doppler-assisted object identification and tracking according to some embodiments of this disclosure.

[0005] Figure 2 This is an illustration of a speed-reconstructed Doppler-assisted object recognition and tracking setup as part of the perception system of an autonomous vehicle, according to some embodiments of this disclosure.

[0006] Figure 3 This is an illustration of a speed reconstruction setup as part of the perception system of an autonomous vehicle, according to some embodiments of this disclosure.

[0007] Figure 4 This is a schematic diagram of a possible geometric representation that can be used by the perception system of an autonomous vehicle according to some embodiments of this disclosure.

[0008] Figure 5 This is an illustration of classifying return points into static and dynamic types according to some embodiments of this disclosure.

[0009] Figure 6 This is an illustration of ground point classification according to some embodiments of this disclosure.

[0010] Figure 7 This is a diagram illustrating the classification of self-return points according to some embodiments of this disclosure.

[0011] Figure 8 A flowchart depicts an example method for classifying return points according to some embodiments of this disclosure.

[0012] Figure 9 A flowchart is depicted illustrating an example method for filtering return points in a point cloud based on radial velocity measurements according to some embodiments of this disclosure.

[0013] Figure 10 A block diagram of an example computer device is depicted, which enables Doppler-assisted object recognition, tracking, and prediction for applications in autonomous vehicles. Summary of the Invention

[0014] The following is a simplified overview of various aspects of this disclosure in order to provide a basic understanding of these aspects. This summary is not a broad overview of the disclosure. It is not intended to identify key or essential elements of the disclosure, nor to depict any scope of a particular implementation of the disclosure or any scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that follows.

[0015] In one aspect of this disclosure, a method for filtering return points in a point cloud based on radial velocity measurements includes: receiving data representing a point cloud comprising a plurality of return points by a sensing system of an autonomous vehicle, each return point including a radial velocity value and position coordinates representing a reflection area reflecting a transmitted signal emitted by the sensing system. At least one threshold condition is applied to each of the plurality of return points, the threshold condition being correlated with the radial velocity value of a given return point, to identify a subset of return points within the plurality of return points. The subset of return points is removed from the point cloud to generate a filtered point cloud. Objects represented by the remaining return points in the filtered point cloud are identified. In some embodiments, a driving path for the autonomous vehicle is determined based on the identified objects.

[0016] In some embodiments, the AV's sensing system includes coherent optical detection and ranging devices, and the signals emitted by the sensing system include frequency-modulated continuous wave (FMCW) signals. In some embodiments, the radial velocity value of each of the plurality of return points is derived from the reflected FMCW signal.

[0017] In some implementations, applying at least one threshold condition includes classifying return points determined to have radial velocities within a threshold radial velocity when offset by the AV as static points. A subset of return points corresponds to the classified static points. In some implementations, the threshold radial velocity ranges from 0.01 m / s to 0.2 m / s.

[0018] In some implementations, the return points in the return point subset correspond only to points with location coordinates determined based on road map data to be located outside the roads traversed by the AV.

[0019] In some implementations, applying at least one threshold condition includes, for each return point, classifying return points identified as self-returning points as being contained within a low-polygon mesh representing the AV and having a radial velocity substantially zero relative to the AV. The low-polygon mesh is defined as the physical geometry containing the AV, and a subset of return points corresponds to the classified self-returning points. In some implementations, the low-polygon mesh is defined by 4 to 1000 vertices.

[0020] In some implementations, applying at least one threshold condition includes classifying return points determined to be less than or equal to a height threshold relative to the ground and having a radial velocity that is substantially zero when offset by the velocity of the AV as ground points. A subset of return points corresponds to the classified ground points. In some implementations, the height threshold is 5 cm to 30 cm.

[0021] In another aspect of this disclosure, there is a system including a sensing system of an autonomous vehicle to perform any implementation of the above-described methods.

[0022] In another aspect of this disclosure, a non-transitory computer-readable medium has instructions stored thereon that, when executed by a computing device, cause the computing device to perform any implementation of the methods described above. Detailed Implementation

[0023] Autonomous vehicles can use light detection and ranging (LiDAR) technology to detect the distance to various objects in the environment, and sometimes even the velocity of these objects. LiDAR emits one or more laser signals (pulses) that arrive at the object, and then detects the arrival signal reflected from the object. By determining the time delay between the signal emission and the arrival of the reflected wave, time-of-flight (ToF) LiDAR can determine the distance to the object.

[0024] ToF LiDAR can determine the velocity (speed and direction of motion) of a returning point by rapidly transmitting two or more signals (as part of different sensing frames) and detecting the position of a reflective surface as the surface moves between each additional frame. The interval between consecutive signals can be short enough that the object does not significantly change its position relative to other objects in the environment between consecutive signals (frames), but long enough to allow LiDAR to accurately detect changes in the object's position. However, ToF LiDAR devices typically cannot determine the velocity of an object based on a single sensing frame.

[0025] Coherent LiDAR utilizes phase information encoded into the transmitted signal and carried by the electromagnetic waves emitted to the target and returning, as well as providing additional functionality not available with standard ToF LiDAR technology. Coherent LiDAR detects changes in the frequency (and accompanying phase) of the reflected wave caused by the motion of the reflecting surface, a phenomenon known as the Doppler effect. The frequency / phase of the reflected wave is related to the component V of the reflecting surface velocity parallel to the wave propagation direction. r (This is referred to in this paper as "radial" or "longitudinal" velocity) sensitivity. In addition to obtaining range information, coherent LiDAR also allows the correlation of radial velocity with the return point of the point cloud (radial velocimetry). This additional information enables velocity reconstruction and tracking of various objects by determining the characteristics of the (translational and rotational) motions performed by the object. In particular, the motion of a rigid body (such as the body of a car, truck, airplane, etc.) is often constrained by conditions, i.e., the distance between two arbitrarily chosen points of the body remains fixed (constant) during its motion. Therefore, the motion of a rigid body can be described as a combination of translational and rotational motions, with translational motion expressed as a translational vector velocity. Rotational motion can be described using angular velocity (hereinafter referred to as "velocity"). (Hereinafter referred to as "angular velocity") is used to describe it.

[0026] Frequency-modulated continuous wave (FMCW) LiDAR can provide range information and radial velocity using the frequency of a linearly "chirping" continuous light signal. The chirp is typically allowed to occur long enough for the signal to reach the object from which it is reflected. The reflected signal is mixed with a local oscillator, and the detector measures the frequency difference. The frequency difference processed from the different portions of the chirped signal can be used to calculate the Doppler shift, which is proportional to the radial velocity of the object relative to the sensor.

[0027] A typical LiDAR emits signals from multiple directions to obtain a wide field of view of the external environment. For example, the transmitter of a LiDAR device can cover a full 360-degree view across a series of consecutive frames identified by timestamps.

[0028] The term "frame" or "sensing frame" as used herein may refer to the entire 360-degree view of the acquired environment, or alternatively, any smaller sector, such as 1 degree, 5 degrees, 10 degrees, or any other acquired angle. Each frame may include a number of return points (or simply "points") corresponding to reflections from various objects in the environment. Each point may be associated with a distance to the corresponding object, or more specifically, with a distance to an element of the reflective surface (reflective area) of the object responsible for the corresponding return point. The set of points within a given frame may be referred to as a "point cloud." A point cloud may include returns from multiple objects. Typically, it is unknown how many objects are priori within a given frame. A single object, such as other vehicles, road signs, pedestrians, etc., as well as the autonomous vehicle itself, may generate multiple return points. For example, a 10-degree frame may include returns from one or more road signs, multiple vehicles located at different distances from the LiDAR device (mounted on the autonomous vehicle) and moving in different directions at different speeds, pedestrians crossing roads, walking or standing on sidewalks, and many other objects. Segmenting a given point cloud (performed by the perception system of an autonomous vehicle) into clusters corresponding to different objects is a fundamental task in autonomous driving technology. Points that are close together (e.g., separated by a small angular distance and corresponding to reflections at similar distances) can still belong to different objects. For example, a traffic sign and a pedestrian standing near the sign can generate a close return point. Similarly, a car driving along a bicycle in an adjacent lane can generate a close return point.

[0029] Point clouds contain numerous points, a large number of which are returned by the ground, buildings, and other non-moving objects that do not need to be identified outside the driving path. Therefore, identifying and classifying objects based on such points requires considerable computational power. However, some detected non-moving points still need processing, for example, if they correspond to objects obstructing the driving path, road signs that need to be read, etc.

[0030] Non-moving points can be classified as "ground points" and "static points." Ground points correspond to points reflected from roads or sidewalks. Moving points can also be classified as ground points if they are detected within a threshold vertical distance from the ground. The location of the ground can be derived from an elevation map and calculated based on the known vertical displacement of the LiDAR device relative to the ground. "Static points" correspond to the return points of non-moving objects that have a low or essentially zero velocity relative to the ground (or, alternatively, an equal and opposite velocity relative to the known velocity of the autonomous vehicle).

[0031] "Self-return points" are a third type of point, corresponding to reflections of the autonomous vehicle itself. While these points have velocity relative to the ground, their velocity relative to the autonomous vehicle is zero. The standard method for classifying points as self-return points involves computing a large 3D triangular mesh representation of the autonomous vehicle's outer surface and attempting to identify return points that are close to or contained within the mesh. However, this method is computationally expensive and often inaccurate, leading to the filtering of points from dynamic objects very close to the autonomous vehicle (e.g., rocks, other vehicles very close by, etc.).

[0032] Any point not classified as a static point, ground point, or self-returning point can simply be classified as a "relevant point," indicating its relevance to the classification of objects that may affect the AV's driving path. While removing static, ground, and self-returning points during object classification can improve downstream computational efficiency, it is important to avoid false positives when static points are misclassified as moving objects. This is especially true in the case of self-returning points, where unfiltered self-returning points may be misclassified as moving objects, causing the autonomous vehicle to make potentially dangerous sudden stops or maneuvers to avoid perceived potential collisions. Furthermore, it is important to avoid false negatives when moving points are misclassified as static points, which can occur when moving points are classified as ground points due to being within a relatively large threshold vertical distance relative to the ground (e.g., small animals below the threshold height).

[0033] The aspects and embodiments of this disclosure overcome these and other drawbacks of the prior art by utilizing radial velocity signals as the basis for filtering static points, ground points, and self-returning points. For example, FMCW LiDAR can detect points corresponding to objects with radial velocities that are the same as those along the same direction as AV (this can be independently known or determined from a large number of points in the cloud) and determine that such points correspond to non-moving objects. While algorithms exist to “filter” point clouds to eliminate non-moving points, these typically require complex algorithms, such as those utilizing machine learning models. By utilizing radial velocity (easily detected with FMCW LiDAR) and using zero or non-zero radial velocity as the filtering criterion, computationally intensive filtering algorithms can be eliminated.

[0034] Various aspects of this disclosure further facilitate the identification of ground points. Current methods utilize a height threshold above the ground surface and treat any point below the threshold as a ground point. This approach indiscriminately removes both dynamic and static points that may still be relevant to the driving path, such as small dynamic objects (e.g., animals). Some embodiments described herein also utilize radial velocity to identify non-moving points below this threshold, thereby eliminating static points close to the ground (e.g., within 30 cm) while retaining dynamic points.

[0035] Various aspects of this disclosure further facilitate the identification of self-return points. Current methods for identifying self-return points utilize complex 3D meshes to accurately simulate the external contours of the AV (e.g., using small and large triangular faces), with vehicles exhibiting larger curves requiring even more complex meshes. This approach requires a trade-off between accuracy and computational complexity; more complex meshes yield higher accuracy but require significant processing power, while less complex meshes require less processing power but sacrifice accuracy. By utilizing radial velocity as a criterion for classifying self-return points, a less complex mesh can be utilized by identifying points within the mesh (which may be relatively close to or slightly away from the actual AV geometry) and then classifying points with zero radial velocity as self-returning. Points near the AV with a radial velocity of zero relative to the sensor are likely to correspond to self-returning points. This allows for a relaxation of the requirement for complex meshes, thereby maintaining highly accurate classification while preserving efficiency.

[0036] Figure 1 This is a diagram illustrating components of an example autonomous vehicle (AV) 100 using Doppler-assisted object recognition and tracking according to some embodiments of the present disclosure. Figure 1 The operation of an example autonomous vehicle is illustrated. Autonomous vehicles can include motorized vehicles (cars, trucks, buses, motorcycles, all-terrain vehicles, recreational vehicles, any specialized agricultural or construction vehicles, etc.), aircraft (airplanes, helicopters, drones, etc.), naval vehicles (ships, boats, yachts, submarines, etc.), or any other self-propelled vehicle capable of operating in an autonomous mode (without or with reduced human input) (e.g., a sidewalk transport robot vehicle).

[0037] The driving environment 110 may include any object (active or inactive) located outside the AV, such as roads, buildings, trees, bushes, sidewalks, bridges, mountains, other vehicles, pedestrians, etc. The driving environment 110 may be urban, suburban, rural, etc. In some embodiments, the driving environment 110 may be an off-road environment (e.g., farmland or agricultural land). In some embodiments, the driving environment may be an indoor environment, such as an industrial plant, transport warehouse, hazardous area of ​​a building, etc. In some embodiments, the driving environment 110 may be substantially flat, with various objects moving parallel to the surface (e.g., parallel to the Earth's surface). In other embodiments, the driving environment may be three-dimensional and may include objects capable of moving in all three directions (e.g., balloons, leaves, etc.). Hereinafter, the term "driving environment" should be understood to include all environments in which an autonomously propelled vehicle can perform autonomous movement. For example, "driving environment" may include any possible flight environment for an aircraft or the marine environment for a naval vessel. Objects in the driving environment 110 may be located at any distance from the AV, from a few feet (or less) to several miles (or more).

[0038] Example AV 100 may include sensing system 120. Sensing system 120 may include various electromagnetic (e.g., optical) and non-electromagnetic (e.g., acoustic) sensing subsystems and / or devices. Throughout this disclosure, the terms “optical” and “light” should be understood to include any electromagnetic radiation (waves) that can be used for object sensing, such as distance sensing, velocity sensing, acceleration sensing, rotational motion sensing, etc., to facilitate autonomous driving. For example, “optical” sensing may utilize the visible light range (e.g., the 380 to 700 nm wavelength range), the UV range (below 380 nm), the infrared range (above 700 nm), the radio frequency range (above 1 meter), etc. In embodiments, “optical” and “light” may include any other suitable range of the electromagnetic spectrum.

[0039] Sensing system 120 may include radar unit 126, which may be any system that uses radio frequency or microwave frequency signals to sense objects within the driving environment 110 of AV 100. The radar unit may be configured to sense both the spatial position of objects (including their spatial dimensions) and their velocity (e.g., using Doppler shift technology). Hereinafter, “velocity” refers to both the speed at which an object moves (the velocity of the object) and the direction of the object’s motion. The term “angular velocity” refers to the speed at which an object rotates about an axis and the direction of that axis of rotation. For example, the axis of rotation of a car turning left (right) points upward (downward), and the value of the angular velocity is equal to the rate of change of the angle of rotation (e.g., measured in radians per second).

[0040] Sensing system 120 may include one or more LiDAR sensors 122 (e.g., LiDAR rangefinders), which may be laser-based units capable of determining the distance to objects in driving environment 110 (e.g., using Time-of-Flight technology). LiDAR sensors can utilize electromagnetic wavelengths shorter than radio wave wavelengths, thus providing higher spatial resolution and sensitivity compared to radar units. The (multiple) LiDAR sensors may include coherent LiDAR sensors, such as FMCW LiDAR sensors. The (multiple) LiDAR sensors may use optical heterodyne detection to determine velocity. In some embodiments, the functionality of the Time-of-Flight and (multiple) coherent LiDAR sensors is combined into a single (e.g., hybrid) unit capable of determining both distance and radial velocity to a reflecting object. Such a hybrid unit may be configured to operate in an incoherent sensing mode (Time-of-Flight mode) and / or a coherent sensing mode (e.g., a mode using heterodyne detection) or simultaneously in both modes. In some implementations, multiple LiDAR sensor units 122 may be mounted on the AV, for example, at different locations spatially separated, to provide additional information about the lateral component of the velocity of the reflecting object, as described in more detail below.

[0041] The multiple LiDAR sensors 122 may include one or more laser sources that generate and emit signals, and one or more detectors that receive signals reflected from an object. The multiple LiDAR sensors 122 may include spectral filters to filter out parasitic electromagnetic waves having wavelengths (frequency) different from the emitted signal. In some embodiments, the multiple LiDAR sensors 122 may include directional filters (e.g., apertures, diffraction gratings, etc.) to filter out electromagnetic waves that may reach the detector in a direction different from the retroreflection direction of the emitted signal. The multiple LiDAR sensors 122 may use a variety of other optical components (lenses, mirrors, gratings, optical thin films, interferometers, spectrometers, local oscillators, etc.) to enhance the sensor's sensing capabilities.

[0042] In some embodiments, the (multiple) LiDAR sensors 122 may be 360-degree units in the horizontal direction. In some embodiments, the (multiple) LiDAR sensors 122 are capable of spatial scanning along both the horizontal and vertical directions. In some embodiments, the field of view can be up to 90 degrees in the vertical direction (such that at least a portion of the upper hemisphere is covered by the LiDAR signal). In some embodiments, the field of view may be a complete sphere (composed of two hemispheres). For the sake of brevity and conciseness, when referring to "LiDAR technology," "LiDAR sensing," "LiDAR data," and "LiDAR" in this disclosure, such references should be understood, where applicable, to also include other electromagnetic sensing technologies, such as radar technology.

[0043] The sensing system 120 may also include one or more cameras 129 to capture images of the driving environment 110. The images may be two-dimensional projections of the driving environment 110 (or a portion thereof) onto the projection plane (planar or non-planar, such as a fisheye) of the camera. Some of the cameras 129 of the sensing system 120 may be cameras configured to capture a continuous (or quasi-continuous) stream of images of the driving environment 110. In some embodiments, the sensing system 120 may also include one or more sonars 128, which may be ultrasonic sonars.

[0044] The sensing data acquired by sensing system 120 can be processed by data processing system 130 of AV 100. For example, data processing system 130 may include perception system 132. Perception system 132 can be configured to detect and track objects in driving environment 110 and identify detected objects. For example, perception system 132 can analyze images captured by camera 129 and is able to detect traffic light signals, road signs, road layouts (e.g., boundaries of traffic lanes, topology of intersections, designation of parking spaces, etc.), the presence of obstacles, etc. Perception system 132 can also receive LiDAR sensing data (coherent Doppler data and incoherent ToF data) to determine the distance to various objects in environment 110 and the velocity of these objects (radial, and in some embodiments, lateral). In some embodiments, perception system 132 may combine data captured by camera(s) 129 to use LiDAR data. In one example, camera(s) 129 may detect images of rocks partially obstructing traffic lanes. Using data from cameras(multiple)129, the perception system 132 can determine the angular size of the rock, but not its linear size. Using LiDAR data, the perception system 132 can determine the distance from the rock to the AV; therefore, by combining the distance information with the angular size of the rock, the perception system 132 can also determine the linear size of the rock.

[0045] In another embodiment, using LiDAR data, the sensing system 132 can determine how far a detected object is from the AV, and can further determine the velocity component of the object along the direction of motion of the AV. Furthermore, using a series of rapid images acquired by the camera, the sensing system 132 can also determine the lateral velocity of the detected object in a direction perpendicular to the direction of motion of the AV. In some embodiments, the lateral velocity can be determined solely from LiDAR data, for example, by identifying the edges of the object (using horizontal scanning) and further determining the speed at which the edges of the object move laterally. The sensing system 132 may have a point cloud filtering module (PFM) 133. The PFM 133 can receive one or more sensor data frames from the sensing system 120. Each frame in the sensor frame may include multiple points. Each point may correspond to a reflective surface from which a signal emitted by the sensing system 120 (e.g., an FMCW sensor of (multiple) LiDAR sensors 122) is reflected. The type and / or properties of the reflective surface may be unknown. Each point may be associated with various data, such as the timestamp of the frame, the coordinates of the reflective surface, the radial velocity of the reflective surface, the intensity of the reflected signal, etc. In one implementation, the coordinates can be spherical (or cylindrical) coordinates. For example, coordinates may include radial distance, polar angle (the angle formed by the direction to the corresponding reflecting surface and the vertical or horizontal plane), and azimuth angle (an angle indicating the direction within the horizontal plane). The radial distance can be determined from LiDAR data, while the angle can be independently determined from synchronizer data or clock data, for example, based on the transmitter's known rotation frequency within the horizontal plane. In one implementation, velocity data can be provided to the PFM 133 by (multiple) LiDAR sensors 122 using Doppler-assisted sensing technology. The PFM 133 can use one or more algorithms to process, classify, and filter various point sets, which the sensing system 132 can use to effectively and reliably detect and track objects.

[0046] The sensing system 132 can also receive information from a GPS transceiver (not shown) configured to obtain information about the AV's position relative to the Earth. The GPS data processing module 134 can combine the sensing data with GPS data to help accurately determine the position of the AV relative to fixed objects in the driving environment 110 (such as roads, lane boundaries, intersections, sidewalks, crosswalks, road signs, surrounding buildings, etc.), the position of which can be provided by map information 135. In some embodiments, the data processing system 130 can receive non-electromagnetic data, such as sonar data (e.g., ultrasonic sensor data), temperature sensor data, pressure sensor data, meteorological data (e.g., wind speed and direction, precipitation data), etc.

[0047] The data processing system 130 may also include an environmental monitoring and prediction component 136, which can monitor how the driving environment 110 evolves over time, for example, by tracking the position and speed of moving objects (relative to the Earth). In some embodiments, the environmental monitoring and prediction component 136 can track changes in the appearance of the environment due to the motion of the AV relative to the environment. In some embodiments, the environmental monitoring and prediction component 136 can predict where various moving objects in the driving environment 110 will be located within a predicted time frame. The prediction can be based on the current position and speed of the moving objects and the tracking dynamics of the moving objects during a certain (e.g., predetermined) time period. For example, based on stored data of object 1 indicating that object 1 accelerated during a previous 3-second time period, the environmental monitoring and prediction component 136 can determine that object 1 is resuming its motion from a stop sign or red traffic light signal. Therefore, given the road layout and the presence of other vehicles, the environmental monitoring and prediction component 136 can predict the position of object 1 that it may be in motion over the next 3 or 5 seconds. As another example, based on stored data of object 2 indicating its deceleration during a previous 2-second time period, environmental monitoring and prediction component 136 can determine that object 2 has stopped at a stop sign or red traffic light signal. Therefore, environmental monitoring and prediction component 136 can predict the position of object 2 in the next 1 or 3 seconds. Environmental monitoring and prediction component 136 can periodically check the accuracy of its predictions and modify them based on new data obtained from sensing system 120.

[0048] Data generated by the perception system 132, the GPS data processing module 134, and the environmental monitoring and prediction component 136 can be used by an autonomous driving system, such as an AV control system (AVCS) 140. AVCS 140 may include one or more algorithms to control how the AV behaves in various driving situations and environments. For example, AVCS 140 may include a navigation system for determining a global driving route to a destination. AVCS 140 may also include a driving path selection system for selecting a specific path through intermediate driving environments, which may include selecting traffic lanes, negotiating traffic congestion, selecting U-turn locations, selecting parking strategies, etc. AVCS 140 may also include an obstacle avoidance system for safely avoiding various obstacles in the AV driving environment (rocks, stalled vehicles, jaywalking pedestrians, etc.). The obstacle avoidance system can be configured to assess the size and trajectory of obstacles (if the obstacle is moving) and select the optimal driving strategy (e.g., braking, steering, acceleration, etc.) to avoid the obstacle.

[0049] The algorithms and modules of AVCS 140 can be used for various systems and components of a vehicle (such as powertrain and steering 150, vehicle electronics 160, signaling 170, and more). Figure 1 (Other systems and components not explicitly shown) generate commands. Powertrain and Steering 150 may include an engine (internal combustion engine, electric motor, etc.), transmission, differential, axles, wheels, steering mechanism, and other systems. Vehicle Electronics 160 may include an onboard computer, engine management, ignition, communication systems, telematics, in-vehicle entertainment systems, and other systems and components. Signals 170 may include high and low beam headlights, parking lights, turn signals and reversing lights, horns and alarms, interior lighting systems, instrument panel notification systems, passenger notification systems, radio and wireless network transmission systems, etc. Some commands output by AVCS 140 can be directly transmitted to Powertrain and Steering 150 (or Signals 170), while other commands output by AVCS 140 are first transmitted to Vehicle Electronics 160, which generates commands for Powertrain and Steering 150 and / or Signals 170.

[0050] In one example, AVCS 140 can determine that an obstacle identified by data processing system 130 will be avoided by slowing the vehicle down until a safe speed is reached, and then maneuvering the vehicle around the obstacle. AVCS 140 can (directly or via vehicle electronics 160) output commands to the powertrain and steering 150 to 1) reduce fuel flow to the engine by modifying throttle settings to lower engine speed, 2) downshift the powertrain to a lower gear via the automatic transmission, 3) engage the braking unit to reduce (in coordination with the engine and transmission) vehicle speed until a safe speed is reached, and 4) perform steering maneuvers using the power steering mechanism until the obstacle is safely bypassed. Subsequently, AVCS 140 can output commands to the powertrain and steering 150 to restore the vehicle's previous speed settings.

[0051] Figure 2 The illustration 200 shows a Doppler-assisted object recognition and tracking setup utilizing speed reconstruction as part of the perception system of an autonomous vehicle, according to some embodiments of this disclosure. Figure 2The image depicts an AV 202 (which could be AV 100 or any other AV) approaching an intersection. AV 202 has a sensor 206, which can be a LiDAR, such as a coherent LiDAR, an FMCW LiDAR, a hybrid coherent / ToF LiDAR, a combination of coherent and incoherent LiDAR, or any other device that allows sensing radial velocity information in addition to range (distance) information. Sensor 206 performs a scan of the driving environment of AV 202. Specifically, sensor 206 can sense multiple return points for each sensing frame. Sensing frames can be separated by a time increment Δτ. The time increment refers to the time difference between signals transmitted in the same direction (or returning from the same direction), as different directions can be detected by signals at slightly different times. More specifically, Δτ can be the duration of a sensor (e.g., a LiDAR transmitter) cycle (e.g., the period of rotation of the sensor transmitter); with N points located around the entire 360-degree horizontal view, any two adjacent sensing directions can be detected with a lead / lag time of Δτ / N.

[0052] like Figure 2 As shown, object 210 (e.g., a car, truck, bus, motorcycle, or any other object) may approach an intersection and turn left. Two consecutive positions of the AV corresponding to two consecutive LiDAR frames taken at times τ and τ+Δτ are shown, for example, 204(1) and 204(2). Similarly, the positions of object 210 at the two frames τ and τ+Δτ are shown as 212(1) and 212(2), respectively.

[0053] It should be understood that, for the purpose of explanation, Figure 2 The displacement of AV 202 and object 210 shown between two consecutive frames is exaggerated, and in reality, various objects can change their positions over a time increment Δτ far less than depicted. For example, when object 210 completes... Figure 2 When turning left as shown, sensor 206 can acquire many frames.

[0054] Object 210 performs a combination of translational and rotational motion. For example, a reference point of object 210 is translated by a vector... Furthermore, object 210 rotates around this reference point by an angle Δφ. In a flat driving environment, describing rotational motion via a single component value Δφ is sufficient; however, in a non-flat 3D environment (especially in flight or navigation), the rotation angle can be a vector. Its three components describe the pitch angle, yaw angle, and roll angle, respectively. The angular velocity of object 210 characterizes the rate of rotation (turning) of object 210. The linear velocity of the reference point similarly determines the rate of translation of object 210. Assuming object 210 is rigid, let's consider a reference point O (with coordinates). angular velocity and linear velocity It can determine the velocity at other points.

[0055] like Figure 2 As shown, at position 212(1), object 210 can reflect multiple signals output by sensor 206 (indicated by solid lines) and generate multiple return points for the first frame (shown by black circles). Return points should be understood as data entries generated by sensing system 132 based on measurements performed by sensor 206 as part of sensing system 120 (e.g., indexed by the angular direction of the output signal, or in any other way). Each return point may include: (1) a distance r to the actual physical reflection area, and (2) a velocity equal to the velocity associated with point j. radial velocity V of the component r (j). The radial velocity is along the direction toward (or away from) sensor 206 (by unit vector). The components of the vector velocity (description): In some implementations, only some of the return points may include radial velocity values. For example, while a ToF distance measurement can be performed at every return point, only some of the points (e.g., every fifth, every tenth, etc.) can be detected by coherent LiDAR and include velocity data. Radial velocity V r (j) is the velocity measured in the reference frame of AV 202. Therefore, since AV 202 is also moving under normal circumstances, the measured velocity V... r (j) may be different from the velocity of the corresponding physical reflection point relative to the ground, and the velocity can then be determined by adding the velocity of the object 210 measured in the AV 202 frame to the velocity of the AV 202 relative to the ground (which can be known independently, for example, from speedometer / odometer data, map / GPS data, etc.) (in vector form).

[0056] At position 212(2), object 220 can similarly reflect the set of new signals output by sensor 206 (indicated by the dashed line) and generate multiple return points for the second frame. One or more mapping algorithms implemented by sensing system 132 can determine the geometric transformation that maps the point cloud of the first frame to the point cloud of the second frame. Such a mapping can be achieved using the ICP algorithm, which iteratively corrects the transformation and minimizes an error metric (e.g., mean square error or some other predetermined metric) based on a comparison of the transformed first and second point clouds (and vice versa). In some implementations, other mapping algorithms, such as the Kabsch algorithm, Procrustes overlay, etc., can be used. Although only two sensing frames (with their respective point clouds) are depicted for simplicity, similar mappings can be generated between various consecutive sensing frames (e.g., between the second and third frames, between the third and fourth frames, etc.) for both object recognition and tracking.

[0057] When object 210 moves from position 212(1) to position 212(2), the return point in the second frame corresponds to a reflective surface of object 210, which may be different from the surface that caused the signal reflection in the first frame. For example, additional return points can be detected when a previously occluded portion of object 210 is rotated into the field of view of sensor 206. Conversely, some previously exposed return points may not exist (because the corresponding physical reflective surface disappears from the view), etc. To address such dynamic aspects of point clouds, an algorithm performed by perception system 132 can determine the bounding boxes of various identified objects, which may be three-dimensional (3D) bounding boxes.

[0058] Figure 3 This is a diagram 300 illustrating a velocity reconstruction setup as part of a sensing system for an autonomous vehicle, according to some embodiments of this disclosure. An object 310 (e.g., a car, truck, etc.) is depicted moving at a certain translational speed. Moving (e.g., moving forward) while maintaining an angular velocity Rotation (e.g., left turn). A portion of AV 302 with sensor 306 mounted thereon is also shown. Sensor 306 may include a rotating transmitter / receiver 308 capable of sending and receiving sensing signals (e.g., laser, radar, etc.) of the external (relative to AV 302) environment. A sensing frame corresponding to a single cycle of transmitter / receiver 308 can generate multiple return points from various reflective areas of object 310 (delineated by black circles).

[0059] The sensing signal 320 can be emitted by the transmitting circuit of the transmitter / receiver 308, reflected from the reflective area 322 of the object 310, return along the same path, and be received by the receiving circuit of the transmitter / receiver 308. The sensing system 120 can associate the return point 330 with the sensing signal 320 and / or the reflective area 322. The return point 330 can include various data that can be extracted from the sensing signal 320, such as coordinates (which can be any suitable coordinate system, such as Cartesian coordinates). spherical coordinates Cylindrical coordinates (or any other coordinate system); as shown, the origin of the coordinate system can be associated with the transmitter / receiver 308. The return point can also include, for example, the radial velocity V. r Data such as the timestamp τ associated with the sensing signal 320 (e.g., the time of signal transmission or return), the strength of the returned signal, and other information such as the polarization of the transmitted and / or received signals. Although in Figure 3 Only the data associated with return point 330 is depicted, but other return points associated with the reflective areas depicted by black circles may include the same or similar types of information. Because object 310 can rotate, the velocity of each in the reflective areas... They can be different from each other. Accordingly, the detected radial velocity V associated with each return point... r (1),V r (2),V r (3),V r (4)... can also be different from each other. PFM 133 can use this difference in detected radial velocity to classify return points and filter the point cloud to remove static points, ground points and self-return points.

[0060] Specifically, if object 310 is a rigid body, then it has a radius vector. The velocities of the elements of an object can be found from the rigid body equations:

[0061]

[0062] in It is the radius vector of a certain reference point. The reference point can be any element of the object, such as an element associated with the return point (3) or any other return point. The choice of reference point O can be arbitrary, because the same rigid body equations exist for any other reference point O', such as

[0063]

[0064] in It is the linear velocity of other reference points O'. Although the linear velocity changes when the reference point changes, the angular velocity is independent of the choice of reference point. This independence provides additional flexibility by allowing the selection of reference points based on convenience (e.g., near the center of a cluster of points detected by sensing system 120).

[0065] Figure 4 This is a schematic depiction of a possible geometric representation 400 for velocity reconstruction of an object's motion by a sensing system for an autonomous vehicle according to some embodiments of this disclosure. The schematic depiction shows a moving and rotating object 410 (e.g., a car, airplane, motorcycle body, etc.) with a translational velocity. and angular velocity A fixed coordinate system x, y, z is shown, which can be associated with AV (not shown), for example, with any axis having a fixed orientation relative to AV. A reference point can be selected from the cluster of return points. (In some arbitrary way). A reference point can be independently selected for any cluster of return points. For example, a reference point (e.g., a point in the middle of a cluster) can be used to analyze the first cluster of points corresponding to a truck directly in front of the AV, while a completely different reference point can be used. Let's analyze the second point cluster corresponding to the car located to the left of AV.

[0066] The second coordinate system can be associated with a reference point; for example, the first axis r is parallel to the radius vector. The direction of the axis is such that axis t is perpendicular to axis r and parallel to the horizontal plane xy, and axis s is perpendicular to the coordinate system r,s,t of the first and second axes. Regarding the radius vector... With respect to the polar angle θ and azimuth angle φ, the directions (unit vectors) of the axes in the second coordinate system are:

[0067]

[0068]

[0069]

[0070] The radial component of velocity (“radial velocity”) detected by, for example, a LiDAR sensor is velocity. and the radius vector along the return point The dot product of unit vectors in the direction: Where R is the magnitude of the radius vector (e.g., the distance from the sensor to the return point). According to the rigid body equations, the radial velocity is...

[0071]

[0072] In some implementations, the return point detected by the FMCW sensor may include information from which radial velocity can be directly derived. The transmitted signal (or FMCW signal) of the FMCW sensor is a frequency-modulated signal from which both range and radial velocity can be extracted. Different types of frequency-modulated signals can be used to modulate the transmitted signal, including but not limited to sawtooth modulation, triangular modulation, square wave modulation, step modulation, and sinusoidal modulation. Triangular modulation can be particularly useful because it readily calculates the radial velocity by comparing the “beat frequencies” between the transmitted and reflected signals, which are functions of time, representing the rising and falling portions of a triangular frequency waveform. The beat frequency is the difference between the actual frequency of the transmitted signal and the frequency of the delayed reflected signal. The Doppler frequency shift f is then calculated based on the beat frequency f1 measured for the rising half of the triangular signal and the beat frequency f2 measured for the falling half of the triangular signal. D In other words,

[0073]

[0074] Then, the radial velocity at the given return point is calculated as follows:

[0075]

[0076] Where λ is the wavelength of the light at the operating frequency of the transmitted signal.

[0077] Figure 5 This is an illustration 500 that classifies return points as static and dynamic according to some embodiments of this disclosure. AV 504 (which may be the same as or similar to AV 202) is depicted as navigating through road 502, which may be a geometrically defined area derived from road map data (e.g., stored in map information 135). AV 202 may be determined to be within the road based on, for example, its GPS coordinates (e.g., determined by GPS data processing module 134). By determining the position of a static or dynamic object relative to AV 504, a return point reflected from the static or dynamic object can be determined to be within the road.

[0078] Various objects are depicted, including moving objects 510 (e.g., pedestrians on a sidewalk), moving objects 512 (e.g., vehicles), and static objects 514 (e.g., fire hydrants). Other objects, not shown, may also be present, such as static objects within the road (e.g., parked cars, pedestrians standing in the road, road signs above the road, etc.) and dynamic objects not within the road (e.g., moving bicycles, etc.). In some embodiments, a road may be defined as including points and objects just outside the usable driving path of AV 504 (e.g., 3-6 feet from the edge of road 502) to include, for example, road signs and pedestrians or vehicles about to enter the driving path as part of road 502.

[0079] Moving objects 510 and 512 are depicted as each having a translational velocity. Each return point (depicted as a black circle) has a corresponding radial velocity V relative to sensor 206. r This radial velocity is related to the translational velocity of the object corresponding to the return point. For example, the return point V r (1) Corresponding to the moved object 510, return point V r (2) and V r (3) Corresponding to the moved object 512, and returning to point V r (4) Corresponding to static object 514. Similarly, AV 504 can have its own translation speed.

[0080] In some implementations, when sensor 206 is an FMCW sensor (e.g., an FMCW of (a plurality of) LiDAR sensors 122), radial velocity can be directly detected using sensor 206 (which can be static or dynamic) as a reference frame. To illustrate the movement of AV 504, each radial velocity is offset by the velocity of AV 504 calculated along a vector from sensor 206 to a given return point. In the case of moving objects 510 and 512, the radial velocity V is offset by the velocity of AV 504. r (1), V r (2), and V r (3) It can correspond to non-zero values ​​(i.e., V) r ′(1), V r ′(2), and V r ′ (Each of the values ​​in (3) can be non-zero). When the offset is applied, the static object 514 can have zero (or a very small) radial velocity. This is because the velocity of AV 504 along the vector from sensor 206 to static object 514 can be equal to and opposite to the “original” (i.e., directly measured) radial velocity V of static object 514. r (4). Therefore, although the measured V r (4) The value relative to sensor 206 can be non-zero (i.e., it appears close to sensor 206), but the offset radial velocity V r ′(4) can be zero.

[0081] For simplicity, a point cloud containing only four points corresponds to the initial or unfiltered point cloud. Static points can be filtered based on their offset radial velocity values. For example, in some implementations, return points can be classified as static or non-static (dynamic) depending on whether the radial velocity of the return point (when offset by the velocity of AV 504) is zero (or within a threshold range near zero) or non-zero (or outside a threshold range near zero). In Figure 500, for these return points, the offset radial velocity V...r ′(1), V r ′(2), and V r (3) Each can be non-zero because their respective objects are not stationary. Offset radial velocity V r ′(4) can be zero (or close to zero and within the threshold range) because it corresponds to static object 514. Therefore, in some implementations, the initial point cloud can be filtered to remove return points corresponding to static object 514 before any downstream object classification, leaving only return points corresponding to dynamic objects. In some implementations, static return points detected within the road may not be filtered from the point cloud. For example, if static object 514 is detected within road 502, it will not be filtered and will be used for downstream object classification to ensure that the driving path of AV 504 is adjusted to avoid collisions.

[0082] Figure 6 Figure 600 illustrates ground point classification according to some embodiments of this disclosure. Figure 600 shows a side view of a road (e.g., road 502) and illustrates a sensor 606 (which may be the same as or similar to sensor 204) located above the ground surface 602. For simplicity, the ground surface 602 is depicted as a flat surface, although convex and concave surfaces may also exist. A threshold 610 corresponds to a vertical height (e.g., 5 to 30 cm) above the ground surface 602, below which return points of static objects can be classified as ground points. Return points 620A and 620B correspond to return points that have been detected as being within the road (as described above regarding...). Figure 5 (as discussed), and respectively with a measured radial velocity V r (A) and V r (B) corresponds to the static object. If sensor 606 is stationary relative to ground surface 602, then the offset radial velocity (as mentioned above regarding...) Figure 5 The one under discussion can be the same as the measured radial velocity (e.g., V). r ′(A)=V r (A) and V r ′(B)=V r (B)).

[0083] In some implementations, a return point can be classified as a ground point if its offset radial velocity is zero (or within a threshold range near zero) and within threshold 610. Because return points 620A and 620B are within the road, they cannot be classified as static points for filtering purposes. Conversely, return point 620A can be classified as a ground point because it is determined to be static based on its offset velocity and because it is below threshold 610. Return point 620B may not be classified as a ground point.

[0084] Figure 7 This is a diagram 700 illustrating the classification of self-return points according to some embodiments of this disclosure. Diagram 700 depicts AV 702 (which may be the same as or similar to AV 202) and sensor 706 (which may be the same as or similar to sensor 206, such as an FMCW sensor). A three-dimensional mesh 710 is spatially defined relative to sensor 706 as containing at least a portion of AV 702 (e.g., only the portion directly line-of-sight with sensor 706) and is defined by a plurality of rectangular or triangular faces connected to each other at vertices 712. For simplicity, mesh 710 is depicted as rectangular; however, it should be understood that mesh 710 corresponds to a three-dimensional rectangular prism containing at least a portion of AV 702 and including at least eight vertices. In some embodiments, meshes of other geometries may be used, which may include at least four, five, six, seven, or eight vertices. In some embodiments, the mesh is an open mesh. In some implementations, the number of vertices can be increased to obtain a mesh geometry that more closely tracks the geometry of the outer surface of the AV 702. The mesh 710 can be further defined as leaving a small buffer (e.g., a 1-6 inch buffer) between the outer surface of the AV 702 and the boundary of the mesh 710.

[0085] Return point 720A represents a self-returning point, which is a reflection from the surface of AV 702 itself. Since return point 720A is static relative to sensor 706, its measured radial velocity (without any offset) can be zero or substantially zero. In some embodiments, return point 720A can be classified as a self-returning point because it is detected within grid 710. Conversely, return point 720B may not be classified as a self-returning point because it is outside grid 710, regardless of its measured radial velocity. In such embodiments, using a direct measurement of the radial velocity (e.g., by an FMCW sensor) as the criterion for classifying a return point as a self-returning point eliminates the need for a highly complex grid that closely tracks the outer surface of AV 702. This represents an improvement in accuracy compared to current methods that use the grid as the sole criterion for identifying self-returning points (which can lead to incorrect classification of self-returning points). Furthermore, the grid does not need to be complex, and significantly fewer vertices can be used (e.g., less than or equal to 1000, less than or equal to 500, or less than or equal to 100 vertices). Low-poly meshes can be used, and self-returning points can still be accurately classified because points with radial velocities within a short distance of the outermost surface of the AV 702 are likely to correspond to self-returning points. As used herein, a “low-poly mesh” refers to a mesh defined by no more than 1000 vertices.

[0086] Figure 8A flowchart depicts an example method 800 for classifying return points according to some embodiments of this disclosure. Method 800, as well as method 900 described below, and / or each of their respective functions, routines, subroutines, or operations, can be executed by a processing device having one or more processing units (CPUs) and a storage device communicatively coupled to the CPUs. The processing device executing methods 800 and 900 can execute instructions from various components (e.g., PFM 133) of the sensing system 132. In some embodiments, methods 800 and 900 can be executed by a single processing thread. Alternatively, methods 800 and 900 can be executed by two or more processing threads, each thread executing one or more individual functions, routines, subroutines, or operations of the method. In the illustrative example, the processing threads implementing methods 800 and 900 can be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, the processing threads implementing methods 800 and 900 can execute asynchronously with each other. Figure 8 and Figure 9 Compared to the order shown, the various operations of methods 800 and 900 can be executed in a different order. Some operations of these methods can be executed concurrently with other operations. Some operations are optional.

[0087] Method 800 may relate to autonomous vehicles (such as...) Figure 1 The system and components of the autonomous vehicle 100. Method 800 can be used to improve the performance of data processing system 130 and / or AVCS 140, and can be applied sequentially or simultaneously to each detected return point in the point cloud. In block 810, method 800 determines whether a given return point is within a three-dimensional grid representing AV (e.g., grid 710). If the point is determined to fall within the grid, method 800 proceeds to block 860, in which method 800 then determines whether the return point has a zero radial velocity V. r (For example, directly detected by the FMCW signal and calculated based on the FMCW signal). If the radial velocity is zero, the point is classified as a self-returning point in box 870. Otherwise, the point is classified as a relevant point in box 890 and can be used downstream in the object recognition and classification process.

[0088] If, within box 810, method 800 determines that the point is not within the grid, then method 800 proceeds to box 820, where method 800 determines whether the point has a zero (or substantially zero) offset radial velocity V′. r If the offset radial velocity is non-zero, then in box 890, the point is classified as a relevant point. Otherwise, method 800 proceeds to box 830, where method 800 determines whether the point is within the road (e.g., based on road map data and the point's physical location relative to the AV and its GPS coordinates).

[0089] If in box 830, method 800 determines that the point is within the road, and method 800 proceeds to box 840. In box 840, method 800 determines whether the point is within the ground threshold. If it is determined that the point is not within the road, then in box 880 the point is classified as a static point.

[0090] If within box 840, method 800 determines that the point is at a ground threshold (e.g., regarding...). Figure 6 If the threshold (610) is within the range described, then in box 850, the point is classified as a ground point. Otherwise, in box 890, the point is classified as a relevant point.

[0091] Figure 9 A flowchart is depicted of an example method 900 for filtering return points in a point cloud based on radial velocity measurements according to some embodiments of the present disclosure. Method 900 can be combined with method 800 to perform classification of return points.

[0092] In box 910, the AV's sensing system (e.g., sensing system 120) receives data representing a point cloud including multiple return points, each return point having a corresponding radial velocity and corresponding position coordinates representing a reflective area reflecting the emitted signal by the sensing system. Acquiring return points may include some or all of the following: One or more sensors of the AV's sensing system (e.g., LiDAR, FMCW, hybrid ToF / coherent LiDAR, etc.) can emit a series of signals (e.g., optical signals). The signals can be emitted in different directions in a periodic (periodic) pattern, for example, forming a 360-degree panoramic pattern. These signals can strike various objects (moving or stationary) and be reflected back to the sensor from various reflective areas of the objects. The number of reflective areas can vary depending on the nature and size of the object, and the degree to which the object is occluded by other objects. The intensity of the reflected signal depends on the physical properties of the reflective area (i.e., material, surface type, etc.), the wavelength of the signal, polarization, etc. Based on the characteristics and timing of the reflected signals, the AV's sensing system can generate multiple return points. A return point is a data entry associated with the reflection of one of the emitted signals from an environmental object. Return points can be generated in real time. Each return point can include various data entries, such as a timestamp of the sensor cycle, the strength of the returned signal, or the polarization of the returned signal. Each return point can include two or more coordinates of its respective reflective region. For example, the coordinates can be the distance to the reflective region (e.g., determined based on the time of flight of the returned signal), and one or more directional angles, such as an azimuth angle specifying a direction in the horizontal plane and a polar angle specifying an altitude above (or below) the horizontal plane. Each return point can also include a velocity value; the velocity value can represent the radial component of the velocity of the reflective region relative to the radial direction from the transmitter of the sensing system toward the reflective region.

[0093] The sensor can be a coherent optical detection and ranging device (LiDAR) capable of detecting radial velocity using, for example, Doppler-assisted sensing. In some embodiments, the coherent LiDAR can be a frequency-modulated continuous wave (FM-CW) LiDAR, and the signal emitted by the sensor can include phase-modulated or frequency-modulated electromagnetic waves. The sensor can also simultaneously emit various other signals, such as pulse signals, which can be used for Time-of-Flight (ToF) distance measurements. In some embodiments, the sensor can include separate ToF LiDARs and coherent LiDARs, each emitting separate signals, which can be synchronized, mixed, and transmitted along the same optical path. In some embodiments, the sensor is an FMCW LiDAR configured to transmit and receive FMCW signals. In some embodiments, radial velocity values ​​can be derived from each of a plurality of return points based on reflected FMCW signals.

[0094] The return point may belong to (e.g., be associated with) various sensing frames corresponding to different periods (e.g., revolutions) of the sensing system's (multiple) transmitters. For example, a first sensing frame may correspond to a first period, and a second sensing frame may correspond to a different (e.g., earlier or later) period. The terms "first" and "second" should be understood only as identifiers and should not presuppose a particular order. In particular, there may be any number of intermediate frames between the first and second frames.

[0095] In block 920, the AV's sensing system (e.g., one or more processing devices implementing PFM 133) applies at least one threshold condition associated with the radial velocity value of a given return point to each of a plurality of return points. Points satisfying the threshold condition can be identified as a subset within the plurality of return points sharing a common classification. In some embodiments, applying the threshold condition includes classifying return points determined to have radial velocities within a threshold radial velocity when offset by the AV as static points. The threshold radial velocity can range from 0.01 m / s (i.e., determining whether the magnitude of the offset radial velocity is less than 0.01 m / s) to 0.2 m / s (i.e., determining whether the magnitude of the offset radial velocity is less than 0.2 m / s).

[0096] In some implementations, the return points in the subset correspond only to points with location coordinates determined from road map data to be located outside the road traversed by the AV. For example, a return point may be classified as a static point if it has a radial offset velocity of zero or substantially zero (i.e., non-moving relative to the ground surface or static reference frame) and is outside the designated road.

[0097] In some implementations, applying threshold conditions includes classifying return points that are determined to be less than or equal to a height threshold relative to the ground and have a radial velocity that is substantially zero when offset by the velocity of the AV as ground points. For example, the height threshold (e.g., threshold 610) can range from 5 cm to 30 cm.

[0098] In some implementations, applying threshold conditions includes classifying return points determined to be contained within a low-polygon mesh representing the AV and having a radial velocity substantially zero relative to the AV as self-returning points. In some implementations, the sensing system computes a mesh (e.g., mesh 710) that at least partially contains the physical geometry of the AV and compares the position coordinates of the return points to determine whether they are contained within the mesh. In some implementations, the low-polygon mesh is defined by 4 to 100 vertices.

[0099] In block 930, the perception system removes any identified subset of return points from the point cloud (e.g., return points classified as static points, ground points, or return points from their origin) to generate a filtered point cloud containing the relevant points. In block 940, the perception system identifies objects represented by the remaining return points in the filtered point cloud. In some embodiments, method 900 may continue, wherein the AV control system (e.g., AVCS 14) causes the AV's driving path to be determined based on the identified objects. For example, the perception system may identify an object moving at 20 mph when making a left turn with a radius of 15 meters and transmit this information to the AV control system. The AV control system can then determine that the AV is about to enter the same intersection before the object can compete for the turn. The AV control system can determine a new path for the AV, including braking, lane changing, stopping, reversing, etc. The control system can then output commands to the powertrain and steering 150, vehicle electronics 160, signaling 170, etc., to ensure that the AV follows the determined driving path.

[0100] Figure 10 A block diagram depicts an example computer device 1000 capable of Doppler-assisted object recognition, tracking, and prediction for autonomous vehicle applications. The example computer device 1000 can be connected to other computer devices in a local area network, intranet, extranet, and / or the Internet. The computer device 1000 can operate as a server in a client-server network environment. The computer device 1000 can be a personal computer (PC), set-top box (STB), server, network router, switch, or bridge, or any device capable of executing a set of instructions (sequential or otherwise) specifying the actions to be taken by the device. Furthermore, although only a single example computer device is shown, the term "computer" should also be understood to include any collection of computers that individually or collectively execute a set of instructions to perform any one or more methods discussed herein.

[0101] Example computer device 1000 may include processing device 1002 (also referred to as processor or CPU), main memory 1004 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous dynamic random access memory (SDRMB), etc.), static memory 1006 (e.g., flash memory, static random access memory, etc. (SRAM)), and auxiliary memory (e.g., data storage device 1018) that can communicate with each other via bus 1030.

[0102] Processing device 1002 represents one or more general-purpose processing devices, such as microprocessors, central processing units, etc. More specifically, processing device 1002 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or a processor implementing combinations of instruction sets. Processing device 1002 may also be one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, etc. According to one or more aspects of this disclosure, processing device 1002 may be configured to execute instructions (perform methods 800 and 900 as described above).

[0103] Example computer device 1000 may also include a network interface device 1008 that can be communicatively coupled to network 1020. Example computer device 1000 may also include a video display 1010 (e.g., a liquid crystal display, touch screen, or cathode ray tube, CRT), an alphanumeric input device 1012 (e.g., a keyboard), a cursor control device 1014 (e.g., a mouse), and a sound signal generation device 1016 (e.g., a speaker).

[0104] Data storage device 1018 may include a computer-readable storage medium (or more specifically, a non-transitory computer-readable storage medium) 1028 thereon storing a set of executable instructions 1022. According to one or more aspects of this disclosure, the executable instructions 1022 may include executable instructions for performing the methods 800 and 900 as described above.

[0105] During the execution of executable instructions 1022 by example computer device 1000, executable instructions 1022 may also reside wholly or at least partially in main memory 1004 and / or processing device 1002, which also constitute computer-readable storage media. Executable instructions 1022 may also be transmitted or received over a network via network interface device 1008.

[0106] Although the computer-readable storage medium 1028 is Figure 10 While referred to as a single medium, the term "computer-readable storage medium" should be understood to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store a set of operational instructions(s). The term "computer-readable storage medium" should also be understood to include any medium capable of storing or encoding a set of instructions executable by a machine, which cause the machine to perform any one or more methods described herein. Therefore, the term "computer-readable storage medium" should be understood to include, but is not limited to, solid-state memories as well as optical and magnetic media.

[0107] Some of the detailed descriptions above are presented based on algorithms and symbolic representations of operations on data bits within computer memory. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively communicate the essence of their work to others skilled in the art. Algorithms here are generally considered as a self-consistent series of steps leading to a desired result. These steps require physical operations on physical quantities. Typically, though not always necessary, these quantities take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated. It has proven convenient to sometimes refer to these signals as bits, values, elements, symbols, characters, terms, numbers, etc., primarily for reasons of generality.

[0108] However, it should be remembered that all these and similar terms are associated with appropriate physical quantities and are merely convenient labels applied to those quantities. Unless otherwise stated, it will be apparent from the discussion below that throughout the description, the use of terms such as “apply,” “remove,” “identify,” “determine,” “store,” “adjust,” “cause,” “return,” “compare,” “create,” “stop,” “load,” “copy,” “replace,” “execute,” etc., refers to the actions and processes of a computer system or similar electronic computing device that manipulate and convert data represented as physical (electronic) quantities in the registers and memory of the computer system into other data represented as physical quantities in the memory or registers or other such information storage, transmission, or display devices of the computer system.

[0109] Examples of this disclosure also relate to apparatus for performing the methods described herein. This apparatus may be specifically constructed for a desired purpose, or it may be a general-purpose computer system selectively programmed by a computer program stored in a computer system. Such a computer program may be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including optical discs, CD-ROMs (Read-Only Optical Disc Drives), and magneto-optical discs, read-only memory (ROM), random access memory (RAM), EPROMs (Electrically Programmable Read-Only Memory), EEPROMs (Electrically Erasable Read-Only Memory), magnetic disk storage media, optical storage media, flash memory devices, other types of machine-accessible storage media, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0110] The methods and demonstrations presented herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used with programs based on the teachings herein, or it may be convenient to construct more specialized devices to perform the required method steps. The necessary structures for various such systems will appear in the description below. Furthermore, the scope of this disclosure is not limited to any particular programming language. It should be understood that the teachings of this disclosure can be implemented using various programming languages.

[0111] It should be understood that the above description is intended to be illustrative and not limiting. Many other embodiments will become apparent to those skilled in the art upon reading and understanding the above description. Although specific examples have been described in this disclosure, it will be appreciated that the systems and methods of this disclosure are not limited to the examples described herein but can be practiced with modifications within the scope of the appended claims. Therefore, the specification and drawings are to be considered illustrative and not restrictive. Accordingly, the scope of this disclosure should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. A method for an autonomous vehicle (AV), comprising: The AV's sensing system receives data representing a point cloud including multiple return points, each return point including a radial velocity value and position coordinates representing a reflection area that reflects the transmitted signal emitted by the sensing system; At least one threshold condition associated with the radial velocity value of a given return point is applied to each of the plurality of return points to identify a subset of return points within the plurality of return points; Remove a subset of returned points from the point cloud to generate a filtered point cloud; Identify the objects represented by the remaining return points in the filtered point cloud; as well as This enables the AV to determine its driving path based on the identified objects. The application of the at least one threshold condition includes: Return points identified as being contained by a low-polygon mesh representing AV and having a radial velocity of zero relative to AV will be classified as self-return points, wherein the low-polygon mesh is defined as being at least a portion of the physical geometry containing AV, and wherein a subset of return points includes the classified self-return points.

2. The method according to claim 1, wherein, The AV's sensing system includes a coherent optical detection and ranging device, LiDAR, and the signals emitted by the sensing system include frequency modulated continuous wave (FMCW) signals.

3. The method according to claim 2, wherein, The radial velocity value of each of the plurality of return points is derived from the reflected FMCW signal.

4. The method according to claim 1, wherein, Applying the at least one threshold condition further includes: Return points that are determined to have a radial velocity within a threshold radial velocity when canceled by the velocity of AV are classified as static points, wherein a subset of return points also includes classified static points.

5. The method according to claim 4, wherein, The static points in the subset of return points correspond only to points with location coordinates that are determined to be located outside the roads traversed by the AV based on road map data.

6. The method according to claim 1, wherein, Applying the at least one threshold condition further includes: Return points that are determined to be less than or equal to a height threshold relative to the ground and have a radial velocity that is zero when canceled out by the velocity of the AV are classified as ground points, wherein a subset of return points also includes classified ground points.

7. A system for autonomous vehicles (AVs), comprising: The AV sensing system, wherein the sensing system is used for: Receive data representing a point cloud including multiple return points, each return point including a radial velocity value and position coordinates representing a reflection area reflecting the transmitted signal emitted by the sensing system; At least one threshold condition associated with the radial velocity value of a given return point is applied to each of the plurality of return points to identify a subset of return points within the plurality of return points; Remove a subset of returned points from the point cloud to generate a filtered point cloud; Identify the objects represented by the remaining return points in the filtered point cloud; and This enables the AV to determine its driving path based on the identified objects. In order to apply the at least one threshold condition, the sensing system is further configured to: Return points identified as being contained by a low-polygon mesh representing AV and having a radial velocity of zero relative to AV will be classified as self-return points, wherein the low-polygon mesh is defined as being at least a portion of the physical geometry containing AV, and wherein a subset of return points includes the classified self-return points.

8. The system according to claim 7, wherein, The AV's sensing system includes a coherent optical detection and ranging device, LiDAR, and the signals emitted by the sensing system include frequency modulated continuous wave (FMCW) signals.

9. The system according to claim 8, wherein, The radial velocity value of each of the plurality of return points is derived from the reflected FMCW signal.

10. The system according to claim 7, wherein, To apply the at least one threshold condition, the sensing system is further configured to: Return points that are determined to have a radial velocity within a threshold radial velocity when canceled by the velocity of AV are classified as static points, wherein a subset of return points also includes classified static points.

11. The system according to claim 10, wherein, The static points in the subset of return points correspond only to points with location coordinates determined based on road map data to be outside the roads traversed by the AV.

12. The system according to claim 7, wherein, To apply the at least one threshold condition, the sensing system is further configured to: Return points that are determined to be less than or equal to a height threshold relative to the ground and have a radial velocity that is zero when canceled out by the velocity of the AV are classified as ground points, wherein a subset of return points also includes classified ground points.

13. A non-transitory computer-readable medium having instructions stored thereon, which, when executed by a computing device, cause the computing device to: Receive data representing a point cloud including multiple return points, each return point including a radial velocity value and position coordinates of a reflection area representing a transmission signal emitted by the sensing system of the autonomous vehicle (AV); At least one threshold condition associated with the radial velocity value of a given return point is applied to each of the plurality of return points to identify a subset of return points within the plurality of return points; Remove a subset of returned points from the point cloud to generate a filtered point cloud; Identify the objects represented by the remaining return points in the filtered point cloud; as well as This enables the AV to determine its driving path based on the identified objects. In order to apply the at least one threshold condition, the computing device further: Return points identified as being contained by a low-polygon mesh representing AV and having a radial velocity of zero relative to AV will be classified as self-return points, wherein the low-polygon mesh is defined as being at least a portion of the physical geometry containing AV, and wherein a subset of return points includes the classified self-return points.

14. The non-transitory computer-readable medium according to claim 13, wherein, The signals emitted by the sensing system include frequency modulated continuous wave (FMCW) signals.

15. The non-transitory computer-readable medium according to claim 14, wherein, The radial velocity value of each of the plurality of return points is derived from the reflected FMCW signal.

16. The non-transitory computer-readable medium according to claim 13, wherein, To apply the at least one threshold condition, the computing device further: Return points that are determined to have a radial velocity within a threshold radial velocity when canceled by the velocity of AV are classified as static points, wherein a subset of return points also includes classified static points.

17. The non-transitory computer-readable medium according to claim 16, wherein, The static points in the subset of return points correspond only to points with location coordinates determined based on road map data to be outside the roads traversed by the AV.

18. The non-transitory computer-readable medium according to claim 13, wherein, To apply the at least one threshold condition, the computing device further: Return points that are determined to be less than or equal to a height threshold relative to the ground and have a radial velocity that is zero when canceled out by the velocity of the AV are classified as ground points, wherein a subset of return points also includes classified ground points.

Images