Radar object classification based on radar cross section data

Abstract

The present disclosure describes techniques for classifying objects detected by an autonomous vehicle in a driving environment using radar cross section (RCS) data. In some examples, the variance of the RCS data associated with an object may be evaluated to determine signal interference caused by multipath fading. The variance of the RCS data may be used to classify the object and determine whether an autonomous vehicle can safely drive over the object. For example, objects such as manhole covers, storm drains, and expansion joints may provide significant radar signals but low RCS variance indicating that a vehicle can drive over them. Based on the classification of the object, the autonomous vehicle may determine a trajectory to avoid the object or directly over the object.

Description

[Technical field]

[0001] This disclosure relates to radar object classification based on radar cross section data. [Background technology]

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This PCT international application claims the benefit of priority to U.S. Application No. 17 / 589,504, filed on January 31, 2022, and entitled "Radar Object Classification Based on Radar Cross Section Data," the entire contents of which are hereby incorporated by reference into this specification in their entirety for all purposes.

[0003]

[0002] Radar generally measures the distance from the radar device to the surface of an object by transmitting radio waves and receiving reflections of the radio waves from the surface of the object, which may be read by a sensor of the radar device. The sensor may generate a signal based at least in part on the radio waves incident on the sensor. The radar reflection signal may be due to reflections of the object, but a portion of the radar signal may also be due to noise and / or other interfering signals (e.g., from the radar device itself or from external sources). In the context of an autonomous vehicle, a radar system may be used to detect objects in a driving environment, analyze the objects, and / or determine a route for the vehicle to safely and efficiently navigate the environment. For example, an autonomous vehicle may use radar data to detect and avoid obstacles, such as pedestrians, in a planned driving path. However, in some cases, radar noise and interference may cause errors in the analysis of radar data, such as false positive object detection. Such radar data analysis errors may present challenges to safely and comfortably navigating the environment. [Brief description of the drawings]

[0004] The detailed description will be set forth with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the drawing in which that reference number first appears. Use of the same reference number in different drawings indicates similar or identical items or features. [Figure 1] FIG. 1 is a pictorial flow diagram illustrating an example technique for determining a driving path over road surface features based on radar data, such as radar cross section (RCS) variance of an object, in accordance with one or more examples of the present disclosure. [Diagram 2]

[0005] FIG. 2 is another pictorial flow diagram illustrating an example technique for determining a driving path around a non-road object based on the object's radar data RCS variance, in accordance with one or more examples of this disclosure. [Diagram 3]

[0006] FIG. 3 illustrates an example computing system including an RCS classifier configured to classify objects based on RCS distribution data, in accordance with one or more examples of the present disclosure. [Figure 4A]

[0007] FIG. 4A illustrates three example radar data graphs showing RCS data associated with three different objects and / or combinations of objects within a driving environment, in accordance with one or more examples of the present disclosure. [Figure 4B]

[0007] FIG. 4B is a diagram illustrating three example radar data graphs showing RCS data associated with three different objects and / or combinations of objects within a driving environment, in accordance with one or more examples of the present disclosure. [Figure 4C]

[0007] FIG. 4C illustrates three example radar data graphs showing RCS data associated with three different objects and / or combinations of objects within a driving environment, in accordance with one or more examples of the present disclosure. [Figure 4D]

[0008] FIG. 4D illustrates another example radar data graph showing RCS data associated with objects in a driving environment, including several sliding windows for determining RCS variance within different time and / or distance windows, in accordance with one or more examples of the present disclosure. [Diagram 5]

[0009] FIG. 5 is a block diagram illustrating an example system for implementing various techniques described herein. [Figure 6]

[0010] FIG. 6 is a flow diagram illustrating an example process for classifying objects by a vehicle based on RCS data variance and determining a driving path for the vehicle relative to the detected objects based on the object classification, in accordance with one or more examples of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005]

[0011] The present disclosure describes techniques for an autonomous vehicle to detect and classify objects in a driving environment based on radar cross section (RCS) data associated with the object. In some examples, the variance of the RCS data associated with the object may be evaluated to determine signal interference from the object caused by multipath fading. The presence of multipath fading may indicate an object with height, such as a pedestrian, that cannot be safely driven by an autonomous vehicle. In contrast, the absence of multipath fading may indicate road surface features and / or objects without significant height profiles, such as manhole covers, storm drains, bridge expansion joints, that can be safely driven by the vehicle. In various examples, an RCS variance-based object classifier (or RCS classifier) ​​may compare the variance of the RCS data associated with the object to a variance threshold to determine whether the vehicle can safely drive over the object. Based on the classification of the object, the vehicle may determine a trajectory directly over the object or an alternate trajectory that avoids the object. In various examples, an autonomous vehicle may determine an object classification based on the variance of the RCS radar data using an RCS classifier and / or based on a combination of the RCS variance data with additional radar data elements (e.g., RCS data, distance data, azimuth data, speed data, etc.) and / or additional sensor data captured by the vehicle (e.g., image data, lidar data, sonar data, etc.). Prior to classifying an object as a road surface feature with a sufficient level of confidence to execute a driving path on the object, the autonomous vehicle may use the RCS variance data in combination with various other sensor-based techniques to determine the object's classification and attributes. As described in more detail below, the techniques described herein may improve vehicle safety and driving efficiency by using the RCS variance to more efficiently and accurately classify objects during navigation of a driving environment.

[0006]

[0012] When an autonomous vehicle is operating in a driving environment, the vehicle may use a radar device to capture radar data of the surrounding environment. The radar data may be analyzed by the autonomous vehicle to detect and classify various objects in the environment. The autonomous vehicle may encounter various types of objects in different driving environments, including mobile dynamic objects (e.g., vehicles, motorcycles, bicycles, pedestrians, animals, etc.) and / or static objects (e.g., buildings, road features, trees, signs, barriers, parked vehicles, etc.). To safely navigate the driving environment, the autonomous vehicle may include various components configured to detect objects and classify objects. In some examples, the perception component of the autonomous vehicle may include various models and / or subcomponents for detecting objects based on radar data and / or other sensor data, evaluating the radar data (e.g., radar cross section data), and classifying objects. For example, the perception component may analyze the radar data to detect objects near the vehicle and may analyze various components of the radar data (e.g., range, azimuth, RCS, speed, altitude, etc.) to classify objects. One or more prediction and / or planning components of an autonomous vehicle may use object detection and classification to determine a driving path for the vehicle relative to objects. In various examples, the perception component may use one or more machine learning (ML) models and / or heuristics-based components to efficiently detect objects, evaluate sensor data, and / or classify objects while the vehicle navigates a driving environment.

[0007]

[0013] The perception component may analyze radar data from one or more different radar devices and / or any number of other sensor types to detect objects in the driving environment and analyze attributes of the objects. In some examples, the radar device may generate multiple radar data parameters including, but not limited to, radar cross section (RCS) data, altitude data, azimuth data, velocity data, etc. In other examples, the radar device may be configured to generate additional and / or fewer radar parameters (or data elements).

[0008]

[0014] When analyzing radar data associated with objects in the environment, an autonomous vehicle may misclassify certain objects based on the radar data and / or misattribute radar signals from road features to non-existent obstacles. In some examples, a radar device may detect a road feature (e.g., a manhole cover, a storm drain, a road widening joint, a steel road plate used in a construction zone, or a road safety feature such as a speed bump, a raised pavement marker, or a rumble strip), and the radar reflection signal caused by the feature may resemble the radar reflection signal of an object having a height that prevents an autonomous vehicle from safely driving over the object based on the size, shape, texture, and material of the road feature. For example, a road feature such as a manhole cover may have a similar azimuth range and reflection strength as a pedestrian to generate similar RCS data. As a result, if a particular radar device does not provide certain additional radar data elements (e.g., altitude), the perception system may not be able to accurately determine the height of the object or distinguish the particular road feature from other objects. Thus, for radar devices that do not provide altitude data (and / or additional sensor data), the perception system may not be able to determine whether an object has a significant height that would prevent it from being driven over. For a particular road surface feature, the radar device may return radar data that erroneously indicates that the road surface feature has a significant height, which may cause the autonomous vehicle to perform dangerous or inefficient driving maneuvers (e.g., an emergency stop, an alternate route to avoid the object, etc.) when the autonomous vehicle could safely drive directly over the road surface feature.

[0009]

[0015] To address the technical challenges and inefficiencies of incorrectly evaluating radar data and incorrectly classifying objects in an environment, the techniques described herein include determining signal interference caused by multipath fading using an RCS variance-based classification system (which may also be referred to as an "RCS variance classifier" or "RCS classifier"). Initially, the RCS classifier may receive radar data captured by a radar sensor of an autonomous vehicle traveling in an environment. In some cases, the autonomous vehicle may include multiple radar devices configured to receive radar data of the traveling environment. Furthermore, each of the radar devices may provide unique radar data representing the radar device's perspective. Additionally, the autonomous vehicle may have various types of radar devices that simultaneously capture different types (or parameters) of radar data, including but not limited to distance data, azimuth data, RCS data, speed data, altitude data, etc.

[0010]

[0016] Based on the radar data, the RCS classifier may detect and classify various objects in the environment. In some examples, the RCS classifier may automatically detect objects based on the radar data using automated techniques such as machine learning models and / or heuristics-based techniques. The RCS classifier may determine an object classification associated with an object by evaluating the variance of the RCS data to determine signal interference caused by multipath fading. Multipath fading occurs when a radar sensor transmits radio waves and the reflected radio waves return to the radar sensor from multiple paths. Multipath fading may be more easily observed when a radar device detects an object with a significant height because the height of the object provides a greater difference in the length of the multiple return paths back to the radar sensor. RCS data illustrating multipath fading may include high variability and / or lack of consistency between RCS values. Thus, the RCS classifier may determine the presence or absence, or multipath fading, by evaluating the variance and / or consistency of the RCS data values. In various examples described herein, the variance and / or consistency of the RCS data values ​​may be determined by machine learning models and / or heuristics-based techniques. In some examples, the RCS classifier may determine an RCS variance and compare the RCS variance to a variance threshold. The variance threshold may be a predefined value associated with a type of radar device and / or an autonomous vehicle model. The RCS variance threshold may indicate a radar return signal in which multipath fading may be present. In some examples, if the RCS classifier determines that the variance of the RCS data associated with the object meets or exceeds the variance threshold, the RCS classifier may determine that the radar data indicates the presence of multipath fading. Conversely, if the determined RCS data variance is below the variance threshold, the RCS classifier may determine that the radar data indicates the absence of multipath fading.

[0011]

[0017] Based on the RCS classifier determining the absence of multipath fading associated with the object, the RCS classifier may classify the object as a road surface feature with no height or minimal height. For example, the RCS classifier may determine low RCS variance and / or the absence of multipath fading to determine that the object is a road surface feature such as a manhole cover, a storm drain, a bridge expansion joint, a steel road plate used in a construction zone, a speed bump, a dip, a pothole, or a road safety feature (e.g., a raised pavement marker or rumble strip used for lane identification, etc.) that can be safely driven by a vehicle. Alternatively, if the RCS classifier determines the presence of relatively high RCS variance and / or multipath fading for the object, the RCS classifier may determine the object classification as an object with significant height that cannot be safely driven by an autonomous vehicle.

[0012]

[0018] In some examples, an RCS classifier, which may be integrated into or otherwise associated with a perception system, may transmit the object classification to a planning system, which may determine a driving path for the autonomous vehicle based on the object classification. Other inputs to the perception system (e.g., image data, lidar data, etc.) may provide additional information regarding the presence, absence, and / or height of the object. For example, for an object classified as a road feature (and for which other sensor inputs to the perception system are consistent with the object being a road feature), the planning system may determine that the object can be safely driven over and generate a trajectory for the autonomous vehicle directly over the object. Alternatively, if the RCS classifier determines that the object has a significant height (e.g., a pedestrian, a traffic sign, road debris, etc.) and / or other sensor inputs to the perception system indicate that the object is present and has a significant height, the planning system may determine that it is not safe to drive over the object and generate a trajectory for the autonomous vehicle to stop and / or drive around the object. The trajectory generated by the planning system may be used to control the autonomous vehicle as it navigates through an environment relative to the object.

[0013]

[0019] As illustrated by these examples, the techniques described herein can improve the functionality, safety, and efficiency of autonomous and semi-autonomous vehicles traveling through a driving environment by using the variance of RCS data to more efficiently and accurately classify objects. The RCS classifier described herein can improve vehicle safety and driving efficiency by improving the accuracy of object classification based on the variance of RCS data to determine signal interference caused by multipath fading. Improved classification of objects by the RCS classifier can be used to more efficiently and accurately determine which objects may be safely driven over by a vehicle and which objects may not. Thus, the features and functions described herein improve vehicle safety by preventing potential collisions with pedestrians and other objects in the driving environment while avoiding dangerous driving maneuvers (e.g., emergency stops or hard turns) caused by false positive object detection from road surface features that can be safely driven over.

[0014]

[0020] The techniques described herein can be implemented in many ways. Exemplary implementations are provided below with reference to the following drawings. Although discussed in the context of an autonomous vehicle, the methods, apparatus, and systems described herein can be applied to a variety of systems (e.g., sensor systems or robotic platforms) and are not limited to autonomous vehicles. In one example, similar techniques can be utilized in driver-controlled vehicles where such systems can provide an indication of whether it is safe to perform various operations. In other examples, any or all of the techniques described herein can be implemented in security systems, object inspection and / or other machine vision systems, such as quality assurance systems, environmental monitoring systems, etc.

[0015]

[0021] FIG. 1 illustrates an example process 100 for classifying objects using variance of RCS radar data associated with objects in a driving environment and determining a driving path on the objects based on the object classification. In various examples, some or all of the operations in process 100 may be performed by a perception component, a planning component, and / or an RCS classifier component integrated within other components and systems in an autonomous vehicle. For example, as shown in this example, process 100 may be implemented using an RCS classifier 102 (or an RCS variance-based object classification system). As described in more detail below, RCS classifier 102 may include various components, such as an RCS variance analyzer and an object classifier, which may be configured to receive radar data and determine and use the variance of the RCS data to classify objects detected by the autonomous vehicle while traveling through a driving environment.

[0016]

[0022] In operation 104, the RCS classifier 102 may receive radar data collected by one or more radar devices of the autonomous vehicle. The RCS classifier 102 and / or other components in the vehicle's perception component may detect objects in the environment based on the radar data (and / or additional sensor data). For example, box 106 shows an autonomous vehicle 108 traveling through a driving environment and approaching an object 110. In this example, the object 110 may be a road surface feature. As shown in box 106, the road surface feature in this example may be a manhole cover. In other examples, the road surface feature may be any object without a significant height profile over which the vehicle 108 can safely travel, including, but not limited to, a storm drain, a road widening joint, a road safety feature object (e.g., a raised pavement marker or rumble strip), or any other road surface feature or object.

[0017]

[0023] In various examples, an autonomous vehicle 108 may be configured to receive radar data from a single radar device or any number of radar devices configured to capture radar data of a traveling environment. Each of the radar devices utilized by an autonomous vehicle 108 may provide unique radar data representing the radar device's perspective. In some examples, based on the position of the radar device on the vehicle relative to objects in the environment (e.g., mounted on the left, right, front, back, top, etc.), a first radar device at a first location may generate different radar data than a second radar device at a second location. The first radar device may be closer or farther away from the object compared to the second radar device and / or may be positioned at a different angle relative to the object, causing different radar data return signals based on the same object to be received.

[0018]

[0024] For any number of radar devices used to capture radar data, each of the radar devices may be of the same or different types (e.g., a monadic radar device, a continuous wave radar device, a Doppler radar device, a monopulse radar device, etc.). In some examples, the different types of radar devices may collect various radar data parameters (or elements) (e.g., RCS, altitude, azimuth, speed, range, etc.). Different radar devices may be configured to collect different combinations of radar data elements, and an autonomous vehicle may include one or a combination of different radar device types to capture different types of radar data.

[0019]

[0025] In operation 104, the radar data collected by the radar device may be used to detect objects within an environment. The RCS classifier 102 may automatically detect objects based on the radar data using various automated techniques, such as machine learning models and / or heuristics-based techniques. As shown in box 106, the vehicle 108 may provide the captured radar data to the RCS classifier 102, which may utilize automated techniques to detect and / or classify objects 110 within the driving environment.

[0020]

[0026] In operation 112, the RCS classifier 102 may determine an object classification for the object 110 by analyzing the variance of the RCS radar data associated with the object 110. Box 114 shows an example in which several RCS data values ​​are rendered on a graph. The RCS data may correspond to radar data captured by one or more radar devices of the vehicle 108. The RCS data may provide a measurement of the target object's ability to reflect a radar signal back to the radar device, and thus may indicate the object's reflectivity / detectability to the radar device. The radar reflectivity of an object detected in the environment may be based on any number of object attributes, such as the object's size, shape, and material, as well as the angle at which a radar beam from the radar device strikes and is reflected from the object. If the object's RCS data value is relatively high, the object is more radar detectable and / or reflective. Conversely, if the RCS data value is low, the object is less radar detectable and / or reflective.

[0021]

[0027] The RCS classifier 102 may use the RCS data to determine an object classification (and / or object type) of the object 110. To determine the classification of the object, the RCS classifier 102 may evaluate the variance and / or consistency of the RCS data values ​​associated with the object to determine signal interference caused by multipath fading. Based on the presence or absence of multipath fading and / or the magnitude of multipath fading, the RCS classifier 102 may determine the object classification of the object. Multipath fading may occur when a transmitter of a radar device transmits a radar beam (e.g., radio waves) and the reflected radar beam returns to the radar device from multiple paths. Thus, multipath fading may cause weakening of signal strength and / or significant RCS data fluctuations. Multipath fading frequently occurs when a radar device detects an object of significant height because the height of the object provides multiple paths for the radio waves to return to the radar receiver. RCS data that illustrates multipath fading may include high variability and / or lack of consistency between RCS values.

[0022]

[0028] Thus, the RCS classifier 102 may determine the presence or absence of multipath fading and / or the magnitude of multipath fading by evaluating the variance and / or consistency of the RCS data values. In any of the various examples described herein, the variance and / or consistency of the RCS data values ​​may be determined by a machine learning model and / or heuristics-based technique that may be performed by the RCS classifier 102 based on variance range, derivatives, and / or any other technique for measuring the smoothness or roughness of a graph or data plot. Additionally or alternatively, the RCS radar data associated with various objects may be analyzed to determine patterns associated with particular object classifications. For example, a machine learning model may be trained based on the RCS radar data associated with various objects detected in the environment to distinguish road surface features that can be safely driven over from objects of significant height that cannot be driven over. In other examples, RCS data patterns associated with various object classes may be stored as pattern or profile data (e.g., object classification profiles) and compared to RCS radar data received by the vehicle's radar device to determine matching object classes of objects detected by the vehicle.

[0023]

[0029] After the RCS classifier 102 determines the variance (e.g., a variance level or metric) of the RCS data associated with the object 110, the variance may be compared to a variance threshold to determine the extent of multipath fading. The variance threshold or thresholds may be determined as predefined values ​​indicating one or more thresholds above which multipath fading is determined to be present. As described in more detail below, the RCS classifier 302 may use a time-based and / or distance-based sliding window of RCS data reflected from the object to periodically determine an RCS variance metric as the vehicle approaches the object. In such an example, a subset of the RCS data values ​​received within the time or distance window (e.g., excluding any RCS data values ​​received outside the window) may be used to determine an RCS variance metric associated with the object for the time / distance window. The RCS variance metric determined based on the time / distance window may be used to determine an object classification for the object. As the vehicle continues to approach the object, the time / distance window may slide to cover subsequent time intervals and / or closer distance RCS data values, and the RCS classifier 302 may determine an updated RCS variance and / or an updated object classification for the object based on the updated time / distance window of RCS data.

[0024]

[0030] In some examples, the RCS variance metric of the object 110 may be expressed as the difference between the maximum RCS value and the minimum RCS value in the RCS data associated with the object 110. In such examples, the variance metric associated with the object 110 (e.g., the maximum RCS value minus the minimum RCS value in a sliding window) may be compared to a variance range threshold. In other examples, a metric representing the RCS variance may be determined by calculating the statistical variance of the distribution of the RCS data. In such examples, the statistical variance metric may be calculated over a sliding window of the RCS data (e.g., a distance of N meters of RCS values), and the statistical variance metric may be compared to a statistical variance threshold. Thus, the variance threshold described herein may be based on the type of RCS variance metric determined by the RCS classifier 312 (e.g., an RCS max-min metric for the statistical variance of the distribution, etc.). Additionally, in some examples, the RCS classifier 312 may determine a higher threshold or variance threshold based on situational data such as the number of radar devices used, the availability of other sensor data (e.g., lidar data, image or video data, sonar data, etc.), current driving conditions (e.g., ambient light, weather conditions such as rain or fog, road conditions, road gradient such as uphill or downhill on the road, etc.). In various other examples, the RCS variance metric may be determined as a derivative, smoothness or roughness metric determined based on the RCS data plot, etc., and the variance threshold may be a corresponding derivative threshold, smoothness or roughness threshold, etc. The variance threshold may be determined by a human or by an automated technique (e.g., a machine learning model, a heuristic-based technique, etc., which may be based on previously acquired data annotated with a corresponding classification label). In some examples, if the determined RCS variance associated with the object 110 exceeds the variance threshold, the degree of RCS variance may indicate a degree of multipath fading and may even indicate a significant height of the object 110.Conversely, if the determined RCS variance is below the variance threshold, the RCS data may indicate that multipath fading is not present and may further indicate that the object 110 does not have a significant height.

[0025]

[0031] In operation 112, the RCS classifier 102 may determine a classification (or type) of the object 110 by evaluating the variance of the RCS data to determine signal interference caused by multipath fading. As shown in box 114, the variance of the RCS data associated with the object 110 is relatively low throughout the range of distances in which the vehicle 108 is receiving radar signals representing the object 110. The RCS data in box 114 maintains relatively consistent values ​​throughout the range of distances as the vehicle 108 approaches the object 110. Thus, in this example, the RCS classifier 102 may determine a low variance of the RCS data for the object 110 and compare the variance to a variance threshold to determine that the variance is below the variance threshold. Thus, no significant amount of multipath fading is present in this example, indicating that the object 110 has a minimal height profile. As such, the RCS classifier 102 may classify the object 110 as a road surface feature.

[0026]

[0032] In some examples, the RCS classifier 102 may classify the object 110 into one of two broad categories corresponding to road surface features over which the vehicle 108 can safely drive (e.g., objects with no height or minimum height profile) and non-road surface features over which the vehicle 108 cannot safely drive (e.g., objects of non-trivial height). In other examples, the RCS classifier 102 may determine a more specific object classification representing a type of road surface feature (e.g., manhole cover, storm drain, road expansion joint, road safety feature, pothole, etc.) and / or a type of non-road surface feature (e.g., pedestrian, sign, traffic cone, road debris, etc.). In some cases, different types of road surface features may be determined based on a combination of RCS variances associated with the road surface features, additional radar data elements (e.g., RCS data, distance data, azimuth data, speed data, etc.), and / or additional sensor data captured by the vehicle 108 (e.g., image data, lidar data, sonar data, etc.).

[0027]

[0033] In operation 116, the object classification determined by the RCS classifier 102 may be used to determine a travel path of the vehicle 108 over the object 110. In some examples, the RCS classifier 102 may send the object classification of the object 110 to a trajectory planning component in the vehicle 108, which may determine a travel path directly over the object 110 based on a determination that the object is a road surface feature. Box 118 illustrates an example in which the planning component has generated a trajectory 120 in which the vehicle 108 travels directly over the object 110. The planning component (or "planner") of the vehicle 108 may be configured to generate one or more trajectories based on the intended destination of the vehicle and the output of the perception component of the vehicle 108. As described above, the perception component may include several subcomponents, including, but not limited to, the RCS classifier 102 configured to detect objects and determine object classification, location, size, trajectory, etc. The trajectory generated by the planning component may be referred to herein as a planned trajectory. The planner, together with other components of the autonomous vehicle (e.g., localization, perception, prediction, and map components), may determine one or more planned trajectories for guiding the vehicle to an intended destination. When determining the planned trajectory, the perception system may analyze the captured radar data to detect and identify one or more other objects in the environment near the vehicle. The prediction component may determine a predicted trajectory of dynamic objects, and the planning component may determine a planned trajectory for traveling through the environment based on the map data and the desired destination of the vehicle, as well as the classification, location, and predicted trajectory of other objects in the environment. Examples of various techniques for generating planned trajectories for autonomous vehicles can be found, for example, in U.S. Pat. No. 10,921,811, filed on January 22, 2018 and issued on February 16, 2021, and entitled "ADAPTABLE AUTONOMOUS VEHICLE PLANNING LOGIC," and U.S. Pat. No. 10,955,851, filed on February 14, 2018 and issued on March 23, 2021, and entitled "Blocking Object Detection," each of which is incorporated by reference in its entirety into this specification.

[0028]

[0034] As shown in box 118, in this example, the planning component may generate a trajectory 120 for the vehicle 108 to drive directly above the object 110. The planner may select the trajectory 120 based on a determination that the object 110 has been classified by the RCS classifier 102 as a road surface feature (e.g., an object that does not have a height or minimum height over which the vehicle 108 can safely drive). As such, the planner may determine that the object 110 may be safely driven over and may generate the trajectory 120 directly above the object 110.

[0029]

[0035] In operation 122, the planner may control the vehicle 108 using the trajectory 120 that travels directly above the object 110. Box 124 illustrates an example in which the vehicle 108 is shown traveling according to the trajectory 120 directly above the object 110 as a road surface feature over which it may be safely traveled. In various examples, the trajectory 120 may be utilized by a planning component of the vehicle 108, a vehicle safety system, and / or other system controller to determine vehicle control commands that may cause the vehicle 108 to track the trajectory 120.

[0030]

[0036] 2 illustrates another example process 200 for classifying objects using variance of RCS radar data associated with objects in a driving environment and determining alternative driving paths for the objects based on the object classification. As in the above examples, some or all of the operations in process 200 may be performed by RCS classifier 102 in the perception component, the planning component, and / or other components and systems in the autonomous vehicle. As described above, RCS classifier 102 may include an RCS variance analyzer and an object classifier that may be configured to receive radar data and determine and use the variance of the RCS data to classify objects detected by the autonomous vehicle while navigating the driving environment.

[0031]

[0037] In operation 204, the RCS classifier 102 may receive radar data collected by a radar device of the autonomous vehicle. As described above, the RCS classifier 102 and / or other components in the perception component may detect objects in the environment based on the radar data (and / or additional sensor data). For example, box 206 shows an autonomous vehicle 208 traveling in a driving environment and approaching a pedestrian 210. Although a pedestrian 210 is shown in this example, in other examples, the vehicle 208 may detect a variety of other static and dynamic objects that it cannot safely drive around, such as other vehicles, motorcycles, bicycles, animals, traffic signs, road debris, etc.

[0032]

[0038] In various examples, the autonomous vehicle 208 may be configured to receive radar data from a single radar device or any number of radar devices on the vehicle configured to capture radar data of the surrounding environment. Each of the radar devices may collect various radar data elements, such as RCS data, distance data, azimuth data, speed data, and altitude data. The RCS classifier 102 may receive and analyze the radar data along with various other sensor data (e.g., image data, lidar data, sonar data, etc.) and / or map data associated with the vehicle 208 to detect objects within the environment. The RCS classifier 102 may classify the detected objects based on the RCS variance of the radar data associated with the objects using various automated techniques, as described above. In box 206, the vehicle 208 may provide the RCS classifier 102 with various elements of the captured radar data associated with any objects detected within the driving environment.

[0033]

[0039] In operation 212, the RCS classifier 102 may determine an object classification for the pedestrian 210 by analyzing the variance of the RCS data associated with the pedestrian 210. Box 214 shows an example in which several RCS data values ​​are rendered on a graph. As described above, the RCS data may represent a measure of an object's reflectivity (and / or general radar detectability) based on the object's ability to reflect a radar signal back to the vehicle's 208 radar device. As such, the RCS classifier 102 may determine and use the variance of the RCS data to determine a classification for the pedestrian 210. To determine the classification for the pedestrian 210, the RCS classifier 102 may evaluate the variance and / or consistency of the RCS radar data associated with the pedestrian 210 to determine signal interference caused by multipath fading. As described above, the RCS classifier 102 may determine the presence or absence of multipath fading and / or the magnitude of multiple fading by evaluating the variance and / or consistency of the RCS data values. The variance and / or consistency of the RCS data values ​​may be determined by machine learning models and / or heuristics-based techniques that may be performed by the RCS classifier 102 based on a variance range, a derivative, and / or any other technique for measuring the smoothness or roughness of a graph or data plot. For example, after the RCS classifier 102 determines the variance (e.g., a variance metric or level) of the RCS data associated with the pedestrian 210, the variance may be compared to a variance threshold to determine the extent of multipath fading. As described above, the RCS data variance of an object (e.g., the pedestrian 210) may be expressed as a range between a maximum RCS value and a minimum RCS value in the RCS data associated with the object, and the variance threshold may be a predefined variance range. In other examples, the variance may be expressed as a derivative, smoothness or roughness metric determined based on the RCS data plot, etc., and the variance threshold may be a corresponding derivative threshold, smoothness or roughness threshold, etc.

[0034]

[0040] In operation 212, the RCS classifier 102 may determine a classification (or type) of the pedestrian 210 by evaluating the variance of the RCS data to determine signal interference caused by multipath fading. As shown in box 214, the variance of the RCS data associated with the pedestrian 210 is relatively high throughout the distance range in which the vehicle 208 is receiving radar signals representing the pedestrian 210. Compared to the RCS data in box 114 associated with the object 110 (e.g., a road feature object), the RCS data in box 214 is generally more variable and less consistent, especially as the vehicle 208 approaches the pedestrian 210. Thus, in this example, the RCS classifier 102 may determine a high variance of the RCS data for the pedestrian 210, compare the variance to a variance threshold, and determine that the variance meets or exceeds the variance threshold. Thus, in this example, there is a significant amount of multipath fading indicating that the pedestrian 210 is an object with a significant height. Thus, the RCS classifier 102 may classify the pedestrian 210 as a non-surface object (e.g., an object having height) over which the vehicle 208 cannot safely drive.

[0035]

[0041] In operation 216, the object classification determined by the RCS classifier 102 may be used to determine an alternative driving path for the vehicle 208 around the pedestrian 210. As described above, the RCS classifier 102 may transmit the object classification of the pedestrian 210 to a trajectory planning component in the vehicle 208 system to determine an alternative driving path for the pedestrian 210 based on a determination that the object is not a road surface feature and cannot be safely driven on. Box 218 illustrates an example in which the planning component has generated a trajectory 220 for the vehicle 208 to drive around the pedestrian 210. As described above, the planning component of the vehicle 208 may be configured to generate any number of alternative trajectories to avoid the pedestrian 210, including stopping trajectories and / or steering trajectories. The alternative trajectories generated by the vehicle 208 may be based on the intended destination of the vehicle or based on the output of the perception component of the vehicle 108. As mentioned above, the perception component may include several subcomponents, including, but not limited to, an RCS classifier 102 configured to detect objects and determine object classification, position, size, trajectory, and the like.

[0036]

[0042] As shown in box 218, the planning component may generate a trajectory 220 for the vehicle 208 to travel around the pedestrian 210, rather than over it. The planner may select the trajectory 120 based on a determination that the pedestrian 210 has been classified as a non-road feature object having significant height. As such, the planner may determine that the pedestrian 210 cannot be safely traveled over by the vehicle 208 and may generate a trajectory 220 around the pedestrian 210. In this example, the planner generated a trajectory around the pedestrian 210, but in other examples, the planner may generate various other trajectories to avoid the pedestrian 210 (e.g., a stopping trajectory).

[0037]

[0043] In operation 222, the planner may control the vehicle 208 using the trajectory 220 to navigate around the pedestrian 210. Box 224 shows an example in which the vehicle 208 follows the trajectory 220 around the pedestrian 210 based on a determination that the pedestrian is a non-surface feature that cannot be safely driven over. As described above, the trajectory 220 may be utilized by the planning component, vehicle safety systems, and / or other system controllers of the vehicle to determine vehicle control commands that may cause the vehicle 208 to track the trajectory 220.

[0038]

[0044] FIG. 3 illustrates a block diagram of a computing system 300 including an example RCS variance-based object classifier (or “RCS classifier 302”) configured to evaluate the variance of RCS radar data to classify objects detected by an autonomous vehicle traveling in an environment. In some examples, the RCS classifier 302 may be similar or identical to the RCS classifier 102 described above, or any other examples herein. As described above, in some cases, the RCS classifier 302 may be implemented within or otherwise associated with a perception component of an autonomous vehicle. The RCS classifier 302 may include various subcomponents, described below, configured to perform different functions of the RCS variance object classification technique. For example, the RCS classifier 302 may include an RCS variance analyzer 304 configured to determine and evaluate the variance of RCS radar data associated with objects detected in an environment to determine signal interference due to multipath fading. Additionally, the RCS classifier 302 may include an object classifier 306 configured to determine a classification or type of object based on the RCS variance and / or other radar data elements. As described above, the degree of multipath fading may be used to determine whether an object is a road surface feature or has significant height, and / or to classify an object as an object over which a vehicle may or may not safely drive.

[0039]

[0045] In some examples, the RCS classifier 302 may receive data from one or more radar devices 308 within (or otherwise associated with) the autonomous vehicle. The different radar devices 308 may be mounted or installed in different locations on the vehicle and may include different types of radar devices that provide various elements (or parameters) of radar data 310 to the RCS classifier 302. As shown in this example, the RCS classifier 302 may include a radar data component 312 configured to receive, store, and / or synchronize radar data from one or more radar devices 308. The radar data component 312 may include various subcomponents, described below, to receive, store, synchronize, and / or analyze the particular radar data received from the radar devices 308. A radar device may capture any number of parameters of radar data 310 from any number of radar devices 308. As shown in FIG. 3, the subcomponents shown are some of the possible radar data parameters that a radar device may capture. In some examples, a radar device may capture more or less radar data components than the illustrated radar data components shown in FIG. 3.

[0040]

[0046] In this example, the radar data component 312 may include an RCS radar data subcomponent 314 configured to store and / or synchronize RCS radar data received from the radar device 308. As described above, the RCS radar data may provide a measurement of an object's reflectivity and / or detectability to the radar device 308. As described above, a high RCS data value may indicate a more detectable and / or reflective object, and a low RCS data value may represent a less detectable and / or reflective object.

[0041]

[0047] The radar data component 312 may also include one or more additional subcomponents associated with different radar data parameters. As shown in FIG. 3, the radar device 308 may capture radar data 310 including speed radar data, altitude radar data, and azimuth radar data. In some examples, depending on the type of radar device, the radar device may capture additional or fewer radar data parameters. In this example, the radar speed data component 316 may be used to determine and / or synchronize the speed of a detected object based on the radar data 310. The radar altitude data component 318 may be used to determine and / or synchronize the height of a detected object based on the radar data 310. The radar azimuth data component 320 may be used to determine and / or synchronize the direction (or orientation) of a detected object relative to the radar device 308.

[0042]

[0048] As described above, the RCS classifier 302 may include an RCS variance analysis component (or “RCS variance analyzer 304”) configured to determine and evaluate the variance of the RCS radar data associated with the object classifier 306. In some examples, the RCS variance analyzer 304 may use the variance of the RCS data associated with the object to determine the extent to which the RCS data includes signal interference caused by multipath fading. The RCS variance analyzer 304 may be a subcomponent of the RCS classifier 302 and / or may be implemented in any other component in the autonomous vehicle perception system. In this example, the RCS variance analyzer 304 may receive RCS data from an RCS radar data subcomponent 314 associated with one or more objects. As described below, the RCS variance analyzer 304 may determine a variance level based on the RCS data points. Additionally or alternatively, the RCS variance analyzer 304 may calculate a derivative and / or a smoothness or roughness metric based on the RCS data points representing the object. The RCS variance analyzer 304 may include one or more trained machine learning models and / or heuristic-based techniques for automatically determining the variance of the RCS data, calculating derivatives of the RCS data, and / or determining a metric of smoothness or roughness of the RCS data. In at least some examples, the derivative of the RCS with respect to distance may be evaluated on a point-by-point basis. In such examples, the variance may be obtained for any number of sliding windows including distances of about 4 m across the data set. A resulting set of variances over distances may then be output by the analyzer 304.

[0043]

[0049] After determining a variance (or set of variances) based on the RCS data associated with the object, the RCS variance analyzer 304 may compare the determined RCS variance to one or more variance thresholds. In some examples, one or more variance thresholds may be defined corresponding to different degrees of multipath fading associated with the RCS data of the objects in the environment. The variance threshold used by the RCS variance analyzer 304 may be a predefined or predetermined value. In this example, the RCS variance analyzer 304 may compare the determined RCS variance determined for the object to the variance threshold. If the determined variance is below the variance threshold, the RCS variance analyzer 304 may determine that the object is a road feature and / or has a minimal height profile, and thus the RCS data associated with the object does not exhibit multipath fading. Alternatively, if the determined variance level exceeds the variance threshold, the RCS variance analyzer 304 may determine that the object has a significant height and is not a road feature, and thus the RCS data exhibits at least some degree of multipath fading. The RCS variance analyzer 304 may send a decision from the comparison to the object classifier 306 .

[0044]

[0050] The object classifier 306 may be configured to determine an object classification (or type) associated with an object detected in the environment based on the RCS variance determined by the RCS variance analyzer 304 and / or various additional radar parameters received from the radar data component 312. In some examples, the object classifier 306 may determine an object classification (e.g., road surface feature or non-road surface feature) based solely on the RCS variance of the object received from the RCS variance analyzer 304. Additionally or alternatively, the object classifier 306 may receive and use radar speed data from the radar speed data component 316, radar altitude data from the radar altitude data component 318, and / or radar azimuth data from the radar azimuth data component 320 to determine the object classification. In some cases, the object classifier 306 may use the RCS variance data in conjunction with the radar-based altitude data and / or to verify the accuracy of the radar-based altitude data received from the radar altitude data component 318. The object classifier 306 may receive additional or fewer radar data parameters depending on the type of radar device used.

[0045]

[0051] In various examples, the object classifier 306 may receive and analyze radar data captured by a single radar device or multiple radar devices 308. For example, different sets of RCS data may be captured by different radar devices 308 but associated with the same object. In such examples, the different sets of RCS data may be evaluated individually to determine different RCS variances, each of which may be compared to a variance threshold. In other examples, the different sets of RCS data from different radar devices 308 may be synchronized (e.g., merged and / or shifted based on distance and angle differences), and the RCS variance analyzer 304 may evaluate the synchronized RCS data. Techniques described herein that include determining and evaluating RCS variance based on radar data captured by multiple radar devices 308 and / or evaluating RCS variance along with additional radar data parameters (e.g., altitude data, speed data, etc.) may further increase the accuracy of object classification in some cases.

[0046]

[0052] As mentioned above, the vehicle may use any number of different object classifiers, each of which may receive any number of different inputs. In some examples, determining the object classification may be based on the variance of RCS radar data associated with the object using various techniques described herein in conjunction with object classification based on other types of sensor data (e.g., lidar data, image data, sonar data, etc.) and / or map data. For example, an object classification determined by the object classifier 306 based on RCS variance data from the RCS variance analyzer 304 may be used to validate another object classification performed by another perception subcomponent, or vice versa, to increase the accuracy and confidence level associated with the object classification.

[0047]

[0053] In some examples, the RCS classifier 302 may include an RCS distribution object profile repository (or “object profile data store 322”). The object profile data store 322 may be a subcomponent of the RCS classifier 302 or may be stored in a separate component within the autonomous vehicle perception system and / or an external computer system. In some examples, the object profile data store 322 may store RCS distribution thresholds and / or one or more RCS data profiles associated with different individual road features, individual non-road feature objects, and / or combinations of multiple road features and / or non-road feature objects. The data in the object profile data store 322 (e.g., the RCS distribution thresholds and / or the RCS data profiles) may be predetermined and / or updated by the object classifier 306 based on the RCS data associated with the objects and the corresponding classification results.

[0048]

[0054] Using any combination of the techniques described herein, the RCS classifier 302 may determine an object classification for an object in an environment based on a variance of RCS data associated with the object (and / or additional data). As shown in this example, the RCS classifier 302 may output an object classification 324, which may be utilized by one or more prediction and / or planning components 326 of the autonomous vehicle. For example, downstream object prediction systems, trajectory planning systems, vehicle safety systems, etc. may use the object classification 324 to determine a navigation maneuver to be performed by the autonomous vehicle. Such a navigation maneuver may include following a trajectory for the object, such as a trajectory that includes driving over a road feature object, or a trajectory that stops before or drives around a non-road feature object.

[0049]

[0055] 4A-4D show four example graphs illustrating various sets of RCS data that may be used to determine signal interference caused by multipath fading and classify associated objects based on the degree of multipath fading in the RCS data. Three different graphs of the RCS data are shown in FIG. 4A-4D. In these examples, the RCS data shown in FIG. 4A-4D may be similar or identical to the RCS data shown in FIG. 1 and FIG. 2.

[0050]

[0056] As described above, an autonomous vehicle may use a radar device to capture radar data of an environment surrounding the vehicle. The radar data, along with additional sensor data and / or map data, may be analyzed by the autonomous vehicle to detect and classify various objects in the environment. Such objects may include mobile, dynamic objects (e.g., vehicles, motorcycles, bicycles, pedestrians, animals, etc.) and / or static objects (e.g., buildings, road features, trees, signs, barriers, etc.). To safely navigate a driving environment, an autonomous vehicle may include various components configured to detect objects and classify detected objects. As described above, the RCS classifier 302 may be configured to classify objects, for example, as road features or non-road features and / or as more specific object types based on the RCS radar data. In various examples, the RCS classifier 302 may determine a particular object classification using patterns of RCS radar data (e.g., object profiles) associated with an object type and / or RCS data variance and variance thresholds. For example, the RCS classifier 302 may determine and evaluate a variance of the RCS data associated with an object to determine signal interference caused by multipath fading. Based on the degree of multipath fading, the RCS classifier 302 may determine an object classification for the object.

[0051]

[0057] 4A shows a first graph 400A illustrating a first example set of RCS data. As shown in this example, the first graph 400A may represent RCS radar data associated with a road feature detected by a radar device of an autonomous vehicle. The graph 400A may include data points representing RCS data values ​​of a radar beam reflected by the road feature and received by the radar device 308. In the graph 400A, the RCS data is displayed in a graph format where the X-axis represents the distance (range) of the vehicle from the detected object. The Y-axis of the graph 400A may represent the magnitude of energy reflected from the road feature object, which may be expressed in decibels per square meter (or "dBsm").

[0052]

[0058] In some cases, to determine the variance of the RCS radar data associated with an object, the RCS classifier 302 may determine the difference between the maximum RCS value and the minimum RCS value within a distance range. For example, the RCS classifier 302 may determine a sliding window over the range of the RCS data and determine the variance of the RCS values ​​within each window (e.g., using the equation Variance(RCS)). In some examples, the RCS classifier 302 may determine the variance for only a portion of the range of RCS values. For example, it may be observed that for distances closer than a distance threshold (e.g., 10-15 meters), the RCS data is relatively poor at distinguishing between road surface features and objects having height. In such examples, the RCS classifier 302 may determine and use the variance of the RCS data for the sliding distance window only for longer distances (e.g., greater than 13 meters).

[0053]

[0059] In the example shown in FIG. 4A, using a predetermined distance, the RCS classifier 302 may analyze the RCS values ​​between a distance of 15-20 meters to determine an approximate magnitude difference of +5 dBsm that determines high and low RCS values. The RCS classifier 302 may repeat this process for the remaining distances of the RCS data in the graph 400A. In various examples, the RCS classifier 302 may determine individual RCS variances for different distance ranges (e.g., 0-5 meters, 5-10 meters, 10-15 meters, etc.) and average the individual RCS variances to determine an overall RCS variance value associated with the object. For simplicity, a distance range of 5 meters is shown in this example, but in other examples, the RCS variance may be determined using a distance range of any size. Additionally or alternatively, the RCS classifier 302 may determine the RCS variance data using a derivative and / or a smoothness or roughness metric based on the RCS data of the graph 400A. The RCS classifier 302 may then compare the determined RCS variance value to a predefined variance threshold. In this example, the determined variance level of the RCS data may be less than the variance threshold, indicating that the RCS data shown in graph 400A does not exhibit multipath fading. Thus, the objects that reflected the RCS data in graph 400A may be classified as heightless road features or minimal height profiles.

[0054]

[0060] FIG. 4B illustrates a second graph 400B showing a second example set of RCS data. As shown in this example, the graph 400B may represent RCS radar data associated with a pedestrian detected by a radar device of an autonomous vehicle. The graph 400B includes a set of data points representing RCS data values ​​of a radar beam reflected by the pedestrian and received by the radar device 308. As in the above example, the RCS data shown in the graph 400B is displayed in a graphical format with an X-axis representing the vehicle's distance (or range) in meters from the detected object and a Y-axis representing the magnitude of energy reflected from the pedestrian object, which may be expressed in dBsm. As described above, the RCS classifier 302 may determine and evaluate the variance of the RCS data to determine signal interference caused by multipath fading. In this example, the RCS classifier 302 may determine high and low RCS values ​​within a particular distance range to determine the variance of the RCS data associated with the pedestrian. Additionally or alternatively, the RCS classifier 302 may determine RCS variance data based on the RCS data of the graph 400B using derivatives and / or smoothness or roughness metrics. The RCS classifier 302 may then compare the determined RCS variance value to a predefined variance threshold. In this example, the determined variance level of the RCS data may be greater than the variance threshold, indicating that the RCS data shown in the graph 400B exhibits multipath fading. Thus, the objects reflected in the RCS data of the graph 400B may be classified as non-road surface features (e.g., pedestrians) having significant (or non-trivial) height profiles.

[0055]

[0061] As described in the above examples, if the RCS classifier 302 determines that the object has a relatively low RCS data variance (e.g., indicating a lack of multipath fading), it may classify the object as a road feature or a minimum height object over which the vehicle may safely drive. In contrast, if the RCS classifier 302 determines that the object has a relatively high RCS data variance (e.g., indicating the presence of multipath fading), it may classify the object as a non-road feature object with significant height over which the vehicle may not safely drive. However, in some examples, the driving environment may include a non-road object with significant height (e.g., a pedestrian) on top of or near a road feature (e.g., a manhole cover). In these examples, the RCS data received from the road feature and the non-road object may be combined into the same radar return signal having the same range and azimuth angle relative to the vehicle. As shown in the following example, a set of RCS data based on road features and a separate non-road object at or near the same location may be observed to have an RCS variance between the variance observed for only the road features (e.g., low RCS variance) and the variance observed for the non-road object (e.g., high RCS variance). Thus, detecting RCS data with a variance in the intermediate range of RCS variance may cause the RCS classifier 302 to determine that the object associated with the RCS data includes both road features and a separate non-road object at or near the same location.

[0056]

[0062] FIG. 4C illustrates a third graph 400C showing a third exemplary set of RCS data. As shown in this example, the graph 400C may represent RCS radar data based on a combination of two objects: a road feature (e.g., a manhole cover) and a pedestrian at the same location or in close proximity to each other in the driving environment. As in the above example, the RCS data shown in the graph 400C is displayed in a graphical format with an X-axis representing the vehicle's distance (or range) in meters from the detected object and a Y-axis representing the magnitude of the energy reflected from the object, which may be expressed in dBsm. As described above, the RCS classifier 302 may determine and evaluate the variance of the RCS data to determine signal interference caused by multipath fading. In this example, the RCS classifier 302 may determine high and low RCS values ​​within a particular distance range in the graph 400C to determine the variance of the RCS data associated with the combination of objects. The RCS classifier 302 may then compare the determined RCS variance to a predefined variance threshold. In this example, the determined variance of the RCS data in graph 400C may meet or exceed a first variance threshold (e.g., a threshold associated with a road feature classification) but may be less than a second variance threshold (e.g., a threshold associated with a non-road feature object classification). Thus, RCS classifier 302 may classify the RCS data in graph 400C as indicative of a combination of multiple objects including a road feature (e.g., a manhole cover) with a minimal height profile and a non-road feature object (e.g., a pedestrian) with a significant height profile.

[0057]

[0063] 4D shows a fourth graph 400D illustrating another example set of RCS data. The RCS data shown in this example may be similar or identical to the RCS data shown in graph 400B and represents RCS data associated with a pedestrian detected by a radar device of an approaching vehicle. As described above, the RCS classifier 302 may repeatedly (e.g., periodically or continuously) determine an RCS variance metric associated with an object as the vehicle approaches the object. As shown in this example, the RCS classifier 302 may determine a sliding time window and / or a sliding distance window of RCS data for the object as the vehicle approaches the object, and may use the sliding time / distance window to determine an updated RCS variance metric for the object.

[0058]

[0064] In this example, as the vehicle approaches the object, at an initial time T0 (e.g., t=0 seconds), the vehicle is at a distance of about 25 meters from the object. When the vehicle reaches a subsequent time T1 (e.g., t=1 second), the vehicle is at a distance of about 21 meters from the object. At or about time T1, the vehicle may determine a first sliding window 402 of RCS data that includes any RCS data values ​​received by the vehicle's radar device between time T0 and time T1 that correspond to a distance of 21 to 25 meters from the object. The RCS classifier 302 may use the RCS data defined by the first sliding window 402 to determine a first RCS dispersion metric for the object and classify the object based on the first RCS dispersion metric. As the vehicle continues to approach the object, at a subsequent time T2 (e.g., t=2 seconds), the vehicle may be at a distance of about 17 meters from the object. At or about time T2, the vehicle may determine a second sliding window 404 of RCS data that includes any RCS data values ​​received by the vehicle's radar device between time T1 and time T2 that correspond to a distance of approximately 17 to 21 meters from the object. The RCS classifier 302 may use the RCS data defined by the second sliding window 404 to determine an updated RCS dispersion metric for the object and reclassify the object based on the second RCS dispersion metric. Similarly, at a subsequent time T3 (e.g., t=3 seconds), the vehicle may be at a distance of approximately 13 meters from the object. At or about time T3, the vehicle may determine a third sliding window 406 of RCS data that includes any RCS data values ​​received by the vehicle's radar device between time T2 and time T3 that correspond to a distance of approximately 13 to 17 meters from the object. The RCS classifier 302 may use the RCS data defined by the third sliding window 406 to determine an updated RCS variance metric for the object and re-classify the object based on the second RCS variance metric.

[0059]

[0065] As shown in this example, the sliding window of RCS data used by the RCS classifier 302 may be defined based on time (e.g., increasing as the vehicle approaches the object) and / or based on distance (e.g., decreasing as the vehicle approaches the object). Additionally, while the sliding windows 402-406 are shown as non-overlapping for clarity in this example, it should be understood that the RCS classifier 302 may determine any number of overlapping RCS data windows as the vehicle approaches the object. For example, the RCS classifier 302 may determine updated RCS data windows at periodic distance thresholds (e.g., every 0.1 meters,..., 0.5 meters, 1 meter, etc.) or at periodic distance thresholds (e.g., every 0.1 seconds,..., 0.5 seconds, 1 second, etc.).

[0060]

[0066] As the vehicle approaches the object, the RCS classifier 302 may determine any number of RCS variance metrics for the object based on different sliding RCS data windows and may determine an object classification based on each RCS variance metric. In some cases, the RCS classifier 302 may determine an initial object classification for the object (e.g., a road object) based on a first sliding window of RCS data, and then determine an updated object classification for the object (e.g., a pedestrian) based on a second sliding window of RCS data as the vehicle approaches the object. In such an example, if the RCS classifier 302 reclassifies the object as a non-road feature object, the vehicle may determine an updated driving path (e.g., an alternative drive to avoid the object), or vice versa.

[0061]

[0067] In some examples, the RCS classifier 302 may use different sized sliding RCS data windows and / or dynamically adjust the size of the sliding RCS data window used to determine the RCS dispersion metric as the vehicle approaches an object. As an example, if the vehicle is moving faster and / or approaching an object faster, the RCS classifier 302 may use a smaller sized sliding RCS data window so that the RCS dispersion metric and object classification can be performed faster. In other examples, the size of the sliding RCS data window used by the RCS classifier 302 may be adjusted up or down based on the number of radar devices used, the availability of other sensor data (e.g., lidar data, image or video data, sonar data, etc.), current driving conditions (e.g., ambient light, weather conditions such as rain or fog, road conditions, road gradient such as uphill / downhill, etc.).

[0062]

[0068] As described in the examples above, the RCS classifier 302 may determine an object classification based on the variance of RCS data associated with an object detected in the environment. For example, the RCS classifier 302 may determine an RCS variance value from RCS radar data reflected by an object (or objects) and compare the RCS variance value to any number of variance thresholds to determine an object classification. The RCS variance metric associated with the object may include an RCS delta (e.g., a maximum observed RCS value minus a minimum observed RCS value) of the object over a time interval and / or distance window of the RCS data associated with the object. Additionally or alternatively, the RCS variance metric may include a statistical variance calculation of the distribution for the RCS data within a particular time interval and / or distance window. In yet another example, the RCS classifier 302 may generate a curve based on the RCS data points and derive the curve at one or more locations to determine an RCS derivative value associated with the object. In such an example, the RCS derivative value may be compared to an RCS derivative threshold to determine an object classification for the object. In yet another example, the RCS classifier 302 may analyze RCS data points reflected from an object to determine a metric corresponding to the smoothness or roughness of the RCS data. The metric of the smoothness or roughness of the RCS data may be compared to a threshold to determine an object classification for the object. In other cases, the RCS classifier 302 may classify the RCS data associated with the object using one or more trained ML models in addition to or in lieu of a threshold. For example, RCS data associated with an object region in an environment may be provided as input to an ML model trained to classify the RCS data into one of a predetermined number of RCS data patterns. Each RCS data pattern may be associated with one or more object classifications or types, and the output of the ML model may be used to classify the object (or multiple objects).

[0063]

[0069] FIG. 5 illustrates a block diagram of an example system 500 for implementing various techniques described herein. The system 500 may include a vehicle 502, which may correspond to an autonomous or semi-autonomous vehicle configured to implement various techniques described herein for classifying detected objects in a driving environment based on the variance of RCS data associated with the object. In this example, the vehicle 502 may include components configured to detect objects using radar data and / or additional sensor data, classify the objects based on the variance of RCS data associated with the object, and determine a driving path for the vehicle based on the object classification. The vehicle 502 in this example may be a driverless vehicle, such as an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions for an entire trip without the driver (or passenger) being expected to control the vehicle at all times. In such an example, the vehicle 502 may be configured to control all functions from the start to the completion of a trip, including all parking functions, and may or may not include a driver and / or controls for driving the vehicle 502, such as a steering wheel, accelerator pedal, and / or brake pedal. This is by way of example only, and the systems and methods described herein may be incorporated into any land-based, air-based, or water-based vehicle, including vehicles that require manual control by a driver at all times, to vehicles that are partially or fully autonomously controlled.

[0064]

[0070] The vehicle 502 may include one or more vehicle computing devices 504, one or more sensor systems 506, one or more emitters 508, one or more communication connections 510, at least one direct connection 512, and one or more drive systems 514.

[0065]

[0071] The vehicle computing device 504 may include one or more processors 516 and a memory 518 communicatively coupled to the one or more processors 516. In the illustrated example, the vehicle 502 is an autonomous vehicle, although the vehicle 502 may be other types of vehicles or robotic platforms. In the illustrated example, the memory 518 of the vehicle computing device 504 stores a localization component 520, a perception component 522 including one or more RCS classifiers 524, a prediction component 526, a planning component 528, one or more maps 530, and one or more system controllers 532. Although the perception component 522 is shown in this example storing the RCS classifier 524, in other examples, the one or more RCS classifiers 524 may be stored within any other component of the vehicle 502. Further, while illustrated in FIG. 5 as residing within memory 518 for illustrative purposes, one or more of the localization component 520, the perception component 522, the prediction component 526, the planning component 528, the map 530, and / or the system controller 532 may additionally or alternatively be accessible to the vehicle 502 (e.g., stored on or otherwise accessible by memory separate from the vehicle 502).

[0066]

[0072] The vehicle computing device 504 may generally perform processing to control how the vehicle operates within an environment. The vehicle computing device 504 may implement various artificial intelligence (AI) techniques, such as machine learning, to analyze and understand the environment around the vehicle 502 and / or direct the vehicle 502 to move within the environment. Various components of the vehicle computing device 504, such as a localization component 520, a perception component 522, a prediction component 526, and / or a planning component 528, may implement various AI techniques to localize the vehicle, detect objects around the vehicle, segment sensor data, determine object classifications, predict object tracking, generate trajectories for the vehicle 502 and objects around the vehicle, etc. In some examples, the vehicle computing device 504 may process data from multiple types of sensors on the vehicle, such as light detection and ranging (lidar) sensors, radar sensors, image sensors, depth sensors (time of flight, structured light, etc.), cameras, etc., within the sensor system 506.

[0067]

[0073] While illustrated in FIG. 5 as residing in memory 518 for illustrative purposes, it is contemplated that the localization component 520, the perception component 522, the prediction component 526, the planning component 528, the map 530, and / or the system controller 532 may additionally or alternatively be accessible to the vehicle 502 (e.g., may be stored in or otherwise accessible by memory remote from the vehicle 502, such as memory 540 of a remote computing device 536).

[0068]

[0074] In at least one example, the localization component 520 may include functionality to receive data from the sensor system 506 to determine the position and / or orientation of the vehicle 502 (e.g., one or more of x-, y-, z-position, roll, pitch, or yaw). For example, the localization component 520 may include and / or request / receive a map of the environment and continually determine the position and / or orientation of the autonomous vehicle within the map. In some examples, the localization component 520 may receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc., and accurately determine the position of the autonomous vehicle using simultaneous localization and mapping (SLAM), calibration, localization and mapping, simultaneously (CLAMS), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc. In some examples, the localization component 520 may provide data to various components of the vehicle 502 to determine an initial position and / or trajectory of the vehicle 502, as described herein.

[0069]

[0075] In some examples, and generally, the perception component 522 may include functionality to perform object detection, segmentation, and / or classification. In some examples, the perception component 522 may provide processed sensor data indicative of the presence of an object in proximity to the vehicle 502 and / or the classification of the object as a type of object (e.g., car, pedestrian, bicycle, animal, building, tree, road surface, curb, sidewalk, stop light, stop sign, unknown, etc.). In additional or alternative examples, the perception component 522 may provide processed sensor data indicative of one or more characteristics associated with a detected object or entity (e.g., a tracked object) and / or the environment in which the entity is located. In some examples, the characteristics associated with the object or entity may include, but are not limited to, x-position (global and / or local position), y-position (global and / or local position), z-position (global and / or local position), orientation (e.g., roll, pitch, yaw), type of object or entity (e.g., classification), velocity of the entity, acceleration of the entity, range (size) of the entity, etc. Characteristics associated with an environment may include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, the time of day, the day of the week, the season, weather conditions, darkness / light indications, etc.

[0070]

[0076] The RCS classifier 524, which may be implemented within the perception component 522, may include any components described herein configured to perform one or more object detection and / or classification functions. In some examples, the RCS classifier 524 may be similar or identical to the RCS classifier 302 described above. For example, the RCS classifier 524 may include one or more of the RCS variance analyzer 304, the object classifier 306, and / or the object profile data store 322. These components may be used in combination to analyze received radar data associated with an object, determine the variance of the RCS data associated with the object, and classify the object as a road feature or non-road feature object. In some examples, the RCS classifier 524 may include a trained ML model configured to classify the object using the RCS data patterns, and / or additional components. After determining the object classification, based in whole or in part on the RCS classifier 524 (e.g., using the RCS variance data), the perception component 522 may transmit the object classification to the planning component 528, which may generate a vehicle trajectory based on the object classification.

[0071]

[0077] In general, the prediction component 526 may include functionality for generating predictive information associated with objects in the environment. As an example, the prediction component 526 may be implemented to predict the location of an area of ​​pedestrians proximate a crosswalk area in the environment (or an area or location otherwise associated with pedestrians crossing a road) as they cross or prepare to cross the crosswalk area. As another example, the techniques discussed herein may be implemented to predict the location of other objects (e.g., vehicles, bicycles, pedestrians, etc.) as the vehicle 502 travels through the environment. In some examples, the prediction component 526 may generate one or more predicted positions, predicted speeds, predicted trajectories, etc. for target objects based on attributes of such target objects and / or other objects proximate the target objects.

[0072]

[0078] In general, the planning component 528 can determine a path to be followed by the vehicle 502 traveling through the environment. The planning component 528 can include functionality to determine various routes, trajectories, and various levels of detail. For example, the planning component 528 can determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the route can be a sequence of waypoints for traveling between the two locations. As non-limiting examples, the waypoints include roads, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 528 can generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 528 can determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instructions can be a trajectory, or a portion of a trajectory. In some examples, multiple trajectories may be generated substantially simultaneously (e.g., within technical tolerances) according to a receding horizon methodology, and one of the multiple trajectories is selected for travel by the vehicle 502. In some examples, the planning component 528 may generate one or more trajectories for the vehicle 502 based at least in part on predicted positions associated with objects in the environment. In some examples, the planning component 528 may evaluate one or more trajectories for the vehicle 502 using temporal logic, such as linear temporal logic and / or signal temporal logic.

[0073]

[0079] In at least one example, the vehicle computing device 504 may include one or more system controllers 532 that may be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems of the vehicle 502. These system controllers 532 may communicate with and / or control corresponding systems of the drive system 514 and / or other components of the vehicle 502. For example, the planning component 528 may generate instructions based at least in part on the sensory data generated by the perception component 522 and send instructions to the system controller 532, which may control operation of the vehicle 502 based at least in part on the instructions. In some examples, if the planning component 528 receives a notification that tracking of an object has been “lost” (e.g., the object is no longer visible in the sensory data and is not occluded by any other objects), the planning component 528 may generate instructions to safely stop the vehicle 502 and / or send a request for teleoperator assistance.

[0074]

[0080] The memory 518 may further include one or more maps 530 that may be used by the vehicle 502 traveling within the environment. For purposes of this disclosure, a map may be any number of data structures modeled in two, three, or N dimensions that may provide information about the environment, such as, but not limited to, topology (such as intersections), streets, mountains, roads, terrain, and the environment in general. In some examples, the map may include, but is not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., lidar information, radar information, etc.), spatial information (e.g., vectorized information about features of the environment, image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and / or intensities), reflectance information (e.g., specular reflectance information, retroreflectance information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional mesh of the environment. In some examples, the maps may be stored in a tiled format, such that individual tiles of the map represent discrete portions of the environment, and may be loaded into the working memory as needed. In at least one example, the one or more maps 530 may include at least one map (e.g., an image and / or a mesh).

[0075]

[0081] In some examples, the vehicle 502 can be controlled based at least in part on the map 530. That is, the map 530 can be used in conjunction with the localization component 520, the perception component 522, the prediction component 526, and / or the planning component 528 to determine a position of the vehicle 502, identify objects in the environment, and / or generate a path and / or trajectory for traveling through the environment. In some examples, one or more maps 530 can be stored on a remote computing device, such as in a memory 540 of the computing device 536, and can be accessed by the vehicle 502 over the network 534. In some examples, multiple maps 530 can be retrieved from the memory 540 and stored, for example, based on characteristics (e.g., type of entity, time of day, day of the week, season, etc.). Storing multiple maps 530 can have similar memory requirements but can improve the speed at which data in the maps can be accessed.

[0076]

[0082] As can be understood, the components discussed herein (e.g., the localization component 520, the perception component 522, the prediction component 526, the planning component 528, the map 530, and / or the system controller 532) are described separately for purposes of illustration. However, operations performed by various components may be combined or performed in any other component, and vice versa. Furthermore, any component discussed as being implemented in software may also be implemented in hardware, and vice versa. Furthermore, any functionality implemented in the vehicle 502 may be implemented in the remote computing device 536 and / or other components, and vice versa.

[0077]

[0083] In at least one example, the sensor system 506 can include time-of-flight sensors, lidar sensors, radar devices, and / or radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), etc. The sensor system 506 can include multiple instances of each of these or other types of sensors. For example, the time-of-flight sensors can include individual time-of-flight sensors located at the corners, front, back, sides, and / or top of the vehicle 502. As another example, the camera sensors can include multiple cameras located at various locations about the exterior and / or interior of the vehicle 502. The sensor system 506 can provide input to the vehicle computing device 504. Additionally or alternatively, the sensor system 506 may transmit sensor data via one or more networks 534 to one or more computing devices 536 at a particular frequency, after a predetermined period of time, in near real-time, etc.

[0078]

[0084] The vehicle 502 may include one or more emitters 508 that emit light and / or sound, as described above. In this example, the emitters 508 include interior acoustic and visual emitters for communicating with occupants of the vehicle 502. By way of example and not by way of limitation, the interior emitters may include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. The emitters in this example also include exterior emitters. By way of example and not by way of limitation, the exterior emitters in this example include lights that signal direction of travel or other indicators of vehicle actions (e.g., indicator lights, signs, light arrays, etc.), as well as one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) that acoustically communicate with pedestrians or one or more other nearby vehicles including acoustic beam steering technology.

[0079]

[0085] Vehicle 502 may also include one or more communications connections 510 that enable communication between vehicle 502 and one or more other local or remote computing devices. For example, communications connections 510 may facilitate communication with other local computing devices on vehicle 502 and / or drive system 514. Communications connections 510 may also allow the vehicle to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.). Communications connections 510 may also enable vehicle 502 to communicate with remotely operated computing devices or other remote services.

[0080]

[0086] The communication connections 510 may include physical and / or logical interfaces for connecting the vehicle computing device 504 to other computing devices or networks, such as the network 534. For example, the communication connections 510 may enable Wi-Fi® based communications, such as over frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Bluetooth®, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol that enables individual computing devices to interface with other computing devices.

[0081]

[0087] In at least one example, the vehicle 502 may include one or more drive systems 514. The vehicle 502 may have a single drive system 514 or may have multiple drive systems 514. In at least one example, when the vehicle 502 has multiple drive systems 514, the individual drive systems 514 may be located at opposite ends (e.g., front and rear, etc.) of the vehicle 502. In at least one example, the drive system 514 may include one or more sensor systems for detecting conditions surrounding the drive system 514 and / or the vehicle 502. By way of example and not limitation, the sensor systems may include one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the wheels of the drive module, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of the drive module, cameras or other image sensors, ultrasonic sensors for acoustically detecting objects in the vicinity of the drive system, lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be intrinsic to the drive system 514. In some cases, the sensor systems of the drive system 514 may overlap or complement corresponding systems of the vehicle 502 (eg, the sensor system 506).

[0082]

[0088] The drive system 514 can include many vehicle systems including a high voltage battery, a motor to propel the vehicle, an inverter to convert direct current from the battery to alternating current for use in other vehicle systems, a steering system including a steering motor and a steering rack (which can be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for braking force distribution to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights to illuminate the vehicle's exterior environment), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electronic components such as DC / DC converters, high voltage junctions, high voltage cables, charging systems, charging ports, etc.). Additionally, the drive system 514 can include a drive system controller that can receive and pre-process data from the sensor systems to control the operation of various vehicle systems. In some examples, the drive system controller can include one or more processors and a memory communicatively coupled to the one or more processors. The memory can store one or more components that perform various functions of the drive system 514. Additionally, drive system 514 also includes one or more communication connections that enable each drive system to communicate with one or more other local or remote computing devices.

[0083]

[0089] In at least one example, the direct connection 512 can provide a physical interface for coupling one or more drive systems 514 with the body of the vehicle 502. For example, the direct connection 512 can allow for the transfer of energy, fluid, air, data, etc. between the drive systems 514 and the vehicle. In some examples, the direct connection 512 can further removably secure the drive systems 514 to the body of the vehicle 502.

[0084]

[0090] In at least one example, the localization component 520, the perception component 522, the prediction component 526, the planning component 528, the map 530, and / or the system controller 532 can process the sensor data as described above and can transmit their respective outputs to one or more computing devices 536 via one or more networks 534. In at least one example, the respective outputs of the components can be transmitted to one or more computing devices 536 at a particular frequency, after a predetermined period of time, in near real-time, etc. Additionally or alternatively, the vehicle 502 can transmit driving log data including raw sensor data, processed sensor data, and / or representations of the sensor data to one or more computing devices 536 via the network 534. Such driving log data (or sensor data) can be transmitted as multiple log files to one or more computing devices 536 at a particular frequency, after a predetermined period of time, in near real-time, etc.

[0085]

[0091] The computing device 536 may include a processor 538, a memory 540, and various components that may be received from and / or transmitted to the vehicle 502 and additional autonomous vehicles in the vehicle fleet. For example, the memory 540 of the computing device 536 may store one or more RCS variance analyzers 542, object classifiers 544, and object classification profiles 546. The RCS variance analyzers 542, object classifiers 544, and object classification profiles 546 may be stored by the computing device 536 and transmitted to different autonomous vehicles based on characteristics of the different autonomous vehicles. For example, based on the number, type, and location of radar devices on the vehicle 502, the computing device 536 may select and provide a particular version of the RCS variance analyzer 542, object classifier 544, and object classification profile 546 to the vehicle 502.

[0086]

[0092] The processor 516 of the vehicle 502 and the processor 538 of the computing device 536 may be any suitable processor capable of processing data and executing instructions to perform operations as described herein. By way of example and not limitation, the processors 516 and 538 may comprise one or more central processing units (CPUs), graphic processing units (GPUs), or any device or part of a device that processes electronic data and converts it to other electronic data that can be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors so long as they are configured to implement encoded instructions.

[0087]

[0093] The memories 518 and 540 are examples of non-transitory computer-readable media. The memories 518 and 540 can store an operating system, as well as one or more software applications, instructions, programs, and / or data for implementing the methods and functions attributed to the various systems described herein. In various examples, the memories 518 and 540 can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein can include many other logical, programmatic, and physical components, of which those shown in the accompanying drawings are merely examples relevant to the description herein.

[0088]

[0094] Although FIG. 5 is shown as a distributed system, in alternative examples, components of the vehicle 502 may be associated with the computing device 536 and / or components of the computing device 536 may be associated with the vehicle 502, or vice versa.

[0089]

[0095] 6 is a flow diagram illustrating an example process 600 for classifying an object based on RCS data variance and determining a travel path of an autonomous vehicle relative to the object based on the object classification. As described below, the process 600 may be performed by one or more computer-based components configured to implement various functions described herein. For example, some or all of the operations of the process 600 may be performed by an RCS classifier 302 configured to classify objects in a travel environment by analyzing the variance of RCS data received from one or more radar devices. As mentioned above, the RCS classifier 302 may be integrated as an in-vehicle system in some examples.

[0090]

[0096] The process 600 is illustrated as a collection of blocks in a logic flow diagram that represent sequences of operations, some or all of which may be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, encryption, decryption, compression, recording, data structures, and the like that perform particular functions or implement particular abstract data types. The order of operations described should not be construed as limiting. Any number of the described blocks may be combined in any order and / or in parallel to perform a process or alternative processes, and not all blocks need to be performed in all examples. For purposes of discussion, the processes herein are described with reference to the frameworks, architectures, and environments described in the examples herein, although the processes may be implemented in a wide variety of other frameworks, architectures, or environments.

[0091]

[0097] In operation 602, the RCS classifier 302 may receive radar data from one or more radar devices of an autonomous vehicle traveling in an environment. As described above, an autonomous vehicle may obtain various types of radar data from any number of radar devices. Furthermore, each of the radar devices may be configured to collect a variety of different radar data parameters. For example, some radar devices may capture radar data parameters including RCS data, azimuth data, speed data, altitude data, etc.

[0092]

[0098] At operation 604, a perception component of the autonomous vehicle may detect objects within the driving environment. In some examples, the perception component may be configured to evaluate radar data to detect objects. Additionally or alternatively, the perception component may use various other types of sensor data (e.g., lidar data, image data, sonar data, etc.) and / or localization and map data to determine objects within the driving environment. As described above, the perception component may detect objects based on various input data using various automated techniques, such as machine learning models and / or heuristics-based techniques.

[0093]

[0099] In operation 606, the RCS classifier 302 may determine an object classification for the object based on the variance of the RCS data associated with the object. As described above, the RCS classifier 302 may classify the object by evaluating the variance of the RCS data associated with the radar data to determine the degree of signal interference caused by multipath fading. Multipath fading in the RCS data may indicate that the object has significant height. Thus, RCS data that illustrates multipath fading may include high variability and / or lack of consistency between RCS values ​​within a particular distance window. The RCS classifier 302 may determine the degree of multipath fading by evaluating the variance and / or consistency of the RCS data values. As described above, after the variance of the RCS data associated with the object is determined, the RCS classifier 302 may compare the variance to one or more variance thresholds. For example, the RCS classifier 302 may determine the variance in a set of RCS data values, and if the variance is below the variance threshold, the RCS classifier 302 may determine that the RCS data may indicate that multipath fading is not present. Conversely, if the determined RCS data variance exceeds the variance threshold, the RCS classifier 302 may determine that the RCS data may indicate the presence of multipath fading.

[0094]

[0100] After the RCS classifier 302 determines the extent of multipath fading in the RCS data associated with the object, the RCS classifier 302 may classify the object based on the extent of multipath fading (e.g., variance of the RCS data). If the RCS data associated with the object indicates a relatively high amount of interference from multipath fading, the RCS classifier 302 may classify the object as an object with a significant height that cannot be safely driven by a vehicle. Alternatively, if the RCS data associated with the object indicates a relatively low amount of interference from multipath fading, the RCS classifier 302 may classify the object as an object without road surface features and / or a significant height profile over which a vehicle can be safely driven.

[0095]

[0101] In operation 608, the autonomous vehicle's planning component may determine whether the object is classified as a road surface feature. As described above, the RCS classifier 302 may determine the object classification based on the RCS data variance associated with the object. After determining the object classification, the RCS classifier 302 may send the object classification to the vehicle's trajectory planning / travel path determination component (e.g., planning component 528) and may determine a trajectory based at least in part on whether the object is a road surface feature. In this example, if the object is classified as a road surface feature (608: Yes), then the planning system may determine and generate candidate trajectories directly above the object in operations 610 and 612. For example, in operation 610, the planning component may determine and generate candidate trajectories directly above the object. As described above, after the object is determined to be a road surface feature (e.g., a manhole cover, a storm drain, a bridge expansion joint, or a road safety feature), the planner may determine that the autonomous vehicle may safely drive directly above the object. The planning component, along with other components of the autonomous vehicle, may determine candidate trajectories for guiding the vehicle to its intended destination. In operation 612, the planning component may control the autonomous vehicle based at least in part on the candidate trajectories determined in operation 610. For example, the planner may cause a system controller of the autonomous vehicle to execute a trajectory directly over a road surface feature.

[0096]

[0102] In contrast, if the object is classified as a non-road feature (608: No), the planning system may determine and generate alternative candidate trajectories. For example, in operation 614, the planning component may determine and generate alternative candidate trajectories. As described above, based on a determination that the object is not a road feature, the planner may determine that the autonomous vehicle may not safely drive directly over the object. Thus, the planning component may generate an alternative trajectory for the object, such as stopping the vehicle before the object or driving around the object. The planning component, together with other components of the autonomous vehicle, may determine an alternative trajectory for guiding the vehicle to its intended destination. In operation 616, the planning component may control the autonomous vehicle based at least in part on the trajectory determined in operation 614. For example, the planner may cause a system controller of the autonomous vehicle to execute a trajectory that avoids the object.

[0097] Example clause

[0103] A. A system having one or more processors and one or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations including receiving radar data from a radar device associated with a vehicle operating in an environment; detecting an object in the environment based at least in part on the radar data; determining a variance of the radar cross section data based at least in part on radar cross section data associated with the radar data; determining an object classification associated with the object based at least in part on a determination that the variance is less than or equal to a variance threshold; determining a travel path for the vehicle based at least in part on the object classification; and controlling the vehicle in the environment based at least in part on the travel path.

[0098]

[0104] B. The system of paragraph A, wherein the operations further include determining a second variance of the radar cross section data, the variance of the radar cross section data being a first variance associated with a first range window of the radar data and the second variance being associated with a second range window of the radar data that is different from the first range window, and determining a second object classification associated with the object based at least in part on the second variance, the second object classification being different from the object classification.

[0099]

[0105] C. The system described in paragraph B, wherein determining the driving path of the vehicle includes determining an alternative driving path for the vehicle based at least in part on the second object classification, and controlling the vehicle within the environment based at least in part on the alternative driving path.

[0100]

[0106] D. The system of paragraph A, wherein the operations further include determining a confidence metric associated with the object classification based at least in part on the variance.

[0101]

[0107] E. The system of paragraph A, wherein determining the object classification includes determining that the object is associated with at least one of a manhole cover object classification, a storm drain cover object classification, a road expansion joint object classification, a steel road plate object classification, a pothole object classification, or a road safety feature object classification.

[0102]

[0108] F. A method comprising: receiving radar data associated with a vehicle operating in an environment from a radar device; determining a variance of the radar cross section data based at least in part on radar cross section data associated with the radar data; determining an object classification associated with an object represented in the radar data based at least in part on the variance; determining a travel path for the vehicle based at least in part on the object classification; and controlling the vehicle in the environment based at least in part on the travel path.

[0103]

[0109] G. The method of paragraph F, further including determining a second variance of the radar cross section data, the variance of the radar cross section data being a first variance associated with a first range window of the radar data and the second variance associated with a second range window of the radar data that is different from the first range window, and determining a second object classification associated with the object based at least in part on the second variance, the second object classification being different from the object classification.

[0104]

[0110] H. The method of paragraph G, wherein determining the driving path of the vehicle includes determining an alternative driving path for the vehicle based at least in part on the second object classification, and controlling the vehicle within the environment based at least in part on the alternative driving path.

[0105]

[0111] I. The method of paragraph F, wherein determining the variance includes determining a set of radar cross section values ​​associated with at least one of a time window or a range window, and determining at least one of a statistical variance for a distribution of the set of radar cross section values, a difference between a maximum and a minimum value in the set of radar cross section values, or an output of a machine learning model trained to determine the object classification based on the set of radar cross section values.

[0106]

[0112] J. The method of paragraph F, wherein the radar data includes output from a plurality of radar devices.

[0107]

[0113] K. The method of paragraph F, wherein the radar data includes velocity data associated with the object, and wherein determining the object classification is further based at least in part on the velocity data associated with the object.

[0108]

[0114] L. The method of paragraph F, wherein the radar data includes altitude data associated with the object, and wherein determining the object classification is further based at least in part on the altitude data associated with the object.

[0109]

[0115] M. The method of paragraph F, wherein determining the object classification includes determining that the object is associated with at least one of a manhole cover object classification, a storm drain cover object classification, a road expansion joint object classification, a steel road plate object classification, a pothole object classification, or a road safety feature object classification.

[0110]

[0116] N. One or more non-transitory computer-readable media storing instructions executable by a processor that, when executed, cause the processor to perform operations including receiving radar data from a radar device associated with a vehicle operating in an environment; determining a variance of the radar cross section data based at least in part on radar cross section data associated with the radar data; determining an object classification associated with an object represented in the radar data based at least in part on the variance; determining a travel path for the vehicle based at least in part on the object classification; and controlling the vehicle within the environment based at least in part on the travel path.

[0111]

[0117] O. The one or more non-transitory computer-readable media of paragraph N, wherein the operations further include determining a second variance of the radar cross section data, the variance of the radar cross section data being a first variance associated with a first range window of the radar data and the second variance being associated with a second range window of the radar data that is different from the first range window; and determining a second object classification associated with the object based at least in part on the second variance, the second object classification being different from the object classification.

[0112]

[0118] P. One or more non-transitory computer-readable media as described in paragraph O, wherein determining the driving path of the vehicle includes determining an alternative driving path for the vehicle based at least in part on the second object classification, and controlling the vehicle within the environment based at least in part on the alternative driving path.

[0113]

[0119] Q. The one or more non-transitory computer readable media of paragraph N, comprising determining a set of radar cross section values ​​associated with at least one of a time window or a range window, and determining at least one of a statistical variance for a distribution of the set of radar cross section values, a difference between a maximum and a minimum value in the set of radar cross section values, or an output of a machine learning model trained to determine the object classification based on the set of radar cross section values.

[0114]

[0120] R. The one or more non-transitory computer-readable media of paragraph N, wherein the radar data includes output from a plurality of radar devices.

[0115]

[0121] S. The one or more non-transitory computer-readable media of paragraph N, wherein the radar data includes velocity data associated with the object, and wherein determining the object classification is further based at least in part on the velocity data associated with the object.

[0116]

[0122] T. The one or more non-transitory computer-readable media of paragraph N, wherein the radar data includes altitude data associated with the object, and determining the object classification is further based at least in part on the altitude data associated with the object.

[0117]

[0123] Although the example sections above are described with respect to particular implementations, it should be understood that in the context of this specification, the contents of the example sections may be implemented via methods, devices, systems, computer-readable media, and / or other implementations. Additionally, any example AT may be implemented alone or in combination with one or more other example ATs.

[0118] Summary

[0124] While one or more examples of the technology described herein have been described, various modifications, additions, permutations, and equivalents thereof fall within the scope of the technology described herein.

[0119]

[0125] In describing the examples, reference is made to the accompanying drawings, which form a part of this specification, which show by way of illustration specific examples of the claimed subject matter. It should be understood that other examples may be used and that modifications or substitutions, such as structural changes, may be made. Such examples, modifications, or variations do not necessarily depart from the intended scope of the claimed subject matter. Although steps herein may be presented in a particular order, in some cases the order may be changed such that certain inputs are provided at different times or in a different order without changing the functionality of the described systems and methods. The disclosed procedures may also be performed in different orders. Additionally, the various calculations herein need not be performed in the order disclosed, and other examples using alternative calculation orders could be readily implemented. In addition to reordering, calculations may also be decomposed into sub-calculations that produce the same results.

[0120]

[0126] Although the subject matter has been described in language specific to structural features and / or method acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

[0121]

[0127] The components described herein represent instructions that may be stored on any type of computer-readable medium and implemented in software and / or hardware. All of the methods and processes described above may be embodied and fully automated via software code and / or computer-executable instructions executed by one or more computers or processors, hardware, or some combination thereof. Some or all of the methods may alternatively be embodied in specialized computer hardware.

[0122]

[0128] Conditional language such as "may," "could," "may," or "might" is understood to indicate that, in the context, a particular example includes a particular feature, element, and / or step, while other examples do not include the particular feature, element, and / or step, unless specifically stated otherwise. Thus, such conditional language does not generally imply that a particular feature, element, and / or step is in any way required by one or more examples, or that one or more examples necessarily include logic for determining, with or without user input or prompting, whether a particular feature, element, and / or step is included in or performed in any particular embodiment.

[0123]

[0129] Connecting phrases such as "at least one of X, Y, or Z," unless specifically stated otherwise, should be understood to mean that the item, term, etc. can be either X, Y, or Z, or any combination thereof, including collections of individual elements. Unless expressly described as singular, "a" means singular and plural.

[0124]

[0130] It should be understood that any routine description, element or block in the flow diagrams described herein and / or illustrated in the accompanying figures may represent a module, segment, or portion of code that includes one or more computer-executable instructions for implementing particular logical functions or elements within the routine. Alternative implementations are included within the scope of the examples described herein, where elements or functions are performed in an order different from that shown or described, including removing elements or functions, performing substantially synchronized, reversed, additional operations, or omitting operations, depending on the functionality involved, as would be understood by one of ordinary skill in the art.

[0125]

[0131] It should be understood that many variations and modifications can be made to the above-described examples, elements of which are within the scope of other acceptable examples, and all such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.

Claims

1. one or more processors; one or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations, the operations including: receiving radar data associated with a vehicle operating in an environment from a radar device; determining a variance of the radar cross section data based at least in part on radar cross section data associated with the radar data; determining an object classification associated with an object represented in the radar data based at least in part on the variance; and determining a driving path for the vehicle based at least in part on the object classification; and controlling the vehicle within the environment based at least in part on the travel path.

2. The operation is determining a second variance of the radar cross section data, the variance of the radar cross section data being a first variance associated with a first range window of the radar data, and the second variance being associated with a second range window of the radar data that is different from the first range window; 10. The system of claim 1, further comprising: determining a second object classification associated with the object based at least in part on the second variance, the second object classification being different from the object classification.

3. Determining the travel route of the vehicle includes: determining an alternative driving route for the vehicle based at least in part on the second object classification; and controlling the vehicle within the environment based at least in part on the alternative driving path.

4. Determining the variance comprises: determining a set of radar cross section values ​​associated with at least one of the time window or the range window; a statistical variance for the distribution of said set of radar cross section values; the difference between the maximum and minimum values ​​in the set of radar cross section values, or and determining at least one output of a machine learning model trained to determine the object classification based on the set of radar cross section values.

5. The system of claim 1 , wherein the radar data includes outputs from multiple radar devices.

6. 4. The system of claim 1, wherein the radar data includes velocity data associated with the object, and wherein determining the object classification is further based at least in part on the velocity data associated with the object.

7. 4. The system of claim 1, wherein the radar data includes altitude data associated with the object, and wherein determining the object classification is further based at least in part on the altitude data associated with the object.

8. receiving radar data associated with a vehicle operating in an environment from a radar device; determining a variance of the radar cross section data based at least in part on radar cross section data associated with the radar data; determining an object classification associated with an object represented in the radar data based at least in part on the variance; and determining a driving path for the vehicle based at least in part on the object classification; and controlling the vehicle within the environment based at least in part on the travel path.

9. determining a second variance of the radar cross section data, the variance of the radar cross section data being a first variance associated with a first range window of the radar data and the second variance being associated with a second range window of the radar data that is different from the first range window; 9. The method of claim 8, further comprising: determining a second object classification associated with the object based at least in part on the second variance, the second object classification being different from the object classification.

10. Determining the travel route of the vehicle includes: determining an alternative driving route for the vehicle based at least in part on the second object classification; and controlling the vehicle within the environment based at least in part on the alternative driving path.

11. Determining the variance comprises: determining a set of radar cross section values ​​associated with at least one of the time window or the range window; a statistical variance for the distribution of said set of radar cross section values; the difference between the maximum and minimum values ​​in the set of radar cross section values, or and determining at least one output of a machine learning model trained to determine the object classification based on the set of radar cross section values.

12. 11. The method of claim 8, wherein the radar data comprises outputs from a plurality of radar devices.

13. 11. The method of claim 8, wherein the radar data includes velocity data associated with the object, and determining the object classification is further based at least in part on the velocity data associated with the object.

14. 11. The method of claim 8, wherein the radar data includes altitude data associated with the object, and determining the object classification is further based at least in part on the altitude data associated with the object.

15. 11. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 8 to 10.

Images