Identifying radar reflections using speed and position information

Abstract

Techniques for determining a reflected return in radar sensor data are discussed. In some instances, radar returns can be compared to one another. For example, a velocity associated with a first radar return can be projected into a radial direction associated with a second radar return to determine a projected velocity. In some examples, the second radar return can be a reflected return if a magnitude of the projected velocity corresponds to a magnitude of the second radar return. In other instances, a reflection point can be determined using position data, and the second radar return can be a reflected return if an object is located at the reflection point. In some instances, a vehicle, such as an autonomous vehicle, can be controlled with information from reflected returns excluded.

Description

[0001] Cross-reference to related applications

[0002] This PCT international patent application claims the benefit and priority of U.S. Patent Application No. 16 / 288,990, filed on February 28, 2019, and U.S. Patent Application No. 16 / 289,068, filed on February 28, 2019, the disclosure of each of which is incorporated herein by reference. Background Technology

[0003] Autonomous vehicles utilize various methods, devices, and systems to traverse environments that include obstacles. For example, autonomous vehicles can use route planning methods, devices, and systems to navigate through areas that may include other vehicles, buildings, pedestrians, etc. These planning systems may rely on sensor data, including radar data, lidar data, image data, and / or other data. However, in some examples, the presence of vehicles, buildings, and / or objects in the environment can cause reflections, resulting in inaccurate or incorrect sensor data, including, for example, false alarms. This inaccurate and / or incorrect sensor data can pose challenges to safely and comfortably traversing the environment. Summary of the Invention

[0004] According to a first aspect of the invention, an autonomous vehicle is provided, comprising: a radar sensor on the autonomous vehicle; one or more processors; and a memory storing processor-executable instructions, which, when executed by the one or more processors, cause the autonomous vehicle to perform actions including: receiving radar data of an environment from the radar sensor, the radar data including: a first radar echo including: a first velocity and a first range along a first radial direction from the radar sensor; a second radar echo including: a second velocity and a second range along a second radial direction from the radar sensor; identifying the first radar echo as an object echo related to an object, at least in part based on the radar data; projecting the first velocity onto a point corresponding to the second radar echo as a projection velocity; determining, at least in part based on a pairwise comparison of the second velocity and the projection velocity, that the second radar echo corresponds to a reflected radar echo reflected from an intermediate surface; and controlling the autonomous vehicle within the environment, excluding the second radar echo.

[0005] According to a second aspect of the invention, a method is provided, comprising: capturing radar data of an environment by radar sensors on an autonomous vehicle, the radar data including a plurality of radar echoes; identifying a first radar echo among the plurality of radar echoes as an object-related object echo based at least in part on the radar data; projecting a first velocity included in the first radar echo onto a point corresponding to a second radar echo among the plurality of radar echoes as a projection velocity; determining, at least in part on a pairwise comparison of a second velocity included in the second radar echo with the projection velocity, that the second radar echo corresponds to a reflected radar echo reflected from an intermediate surface; and controlling the autonomous vehicle within the environment, excluding the second radar echo.

[0006] According to a third aspect of the invention, one or more non-transitory computer-readable media are provided that store instructions, when executed, cause one or more processors to perform the method described above. Attached Figure Description

[0007] Specific embodiments are described with reference to the accompanying drawings. In the drawings, the leftmost numeral of the reference numeral indicates the drawing in which that numeral first appears. The use of the same reference numerals in different drawings indicates similar or identical components or features.

[0008] Figure 1 This is a schematic view illustrating an example vehicle and the environment in which the vehicle operates according to an embodiment of the present disclosure, the vehicle including radar sensors for sensing objects in the environment.

[0009] Figure 2 yes Figure 1 A schematic view of the environment, illustrating an example technique according to embodiments of the present disclosure for using velocity information to distinguish between radar echoes associated with objects in the environment and reflected echoes reflected from intermediate objects.

[0010] Figure 3 yes Figure 1 Another schematic view of the environment illustrates an example technique, according to embodiments of the present disclosure, for using location information to distinguish between radar echoes associated with objects in the environment and reflected echoes reflected from intermediate objects.

[0011] Figure 4 This is a schematic block diagram of an example system according to an embodiment of the present disclosure, the example system including a vehicle and a computing device that can be used to implement the radar reflection identification technology described herein.

[0012] Figure 5 A flowchart depicts an example process for implementing radar reflection identification technology using speed information according to embodiments of the present disclosure.

[0013] Figure 6 A flowchart depicts an example process for implementing radar reflection identification technology using location information according to embodiments of the present disclosure. Detailed Implementation

[0014] As discussed above, certain types of sensor data (e.g., radar data) can be susceptible to reflections from, for example, vehicles, buildings, and other objects in the environment. These reflected echoes can pose challenges to the safe and / or comfortable passage of autonomous vehicles through their environment. For instance, reflected echoes can represent approaching objects that do not actually exist. Planning in response to these non-existent or phantom "objects" may cause the vehicle to take unnecessary actions, such as braking, steering, etc.

[0015] This application describes techniques for identifying reflected echoes in radar sensor data captured by a radar system. Typically, radar sensors emit radio energy that is reflected (or bounced off) objects in the environment before returning to the sensor. When the emitted energy returns directly from the object to the radar sensor, the radar sensor can capture object echoes, which include accurate data about the object (e.g., distance, location, speed, etc.). However, in some instances, the radio energy may reflect from multiple objects in the environment before returning to the radar sensor. In these instances, the radar sensor can capture reflected echoes. Reflected echoes do not accurately represent objects in the environment. At least in some examples, safety-critical path planning for autonomous vehicles may be affected without knowing whether these echoes originate from actual objects or are reflections.

[0016] In some examples, the techniques described herein can use pairwise comparisons of echoes (e.g., pairing the considered echo with an echo of a known object) to determine whether a radar echo is a reflected echo. More specifically, radar echoes unrelated to a known object can be compared with object echoes (e.g., echoes associated with a known object) to determine whether the radar echoes generally correspond to the theoretical reflection of a physical echo from a hypothetical object. In some examples, the object echo can be determined from all radar echoes based on information that associates the echo with a trajectory or other previously obtained information. By way of non-limiting example, theoretical reflection can be determined by projecting the object velocity (which can be provided by the magnitude and direction from the object echo) into a radial direction extending from the vehicle to the considered echo. The radial component of the projected velocity (e.g., extending along the radial direction) represents the expected velocity of the object echo reflected along that radial direction. Thus, an echo can be identified as a reflected echo when the magnitude of the radial component of the projected velocity corresponds to the velocity associated with the echo in question.

[0017] In some examples, the techniques described herein can determine that an echo was reflected using pairwise comparisons of radar echoes without any available prior knowledge. For example, any two echoes can be compared to determine whether one of the echoes (e.g., the more distant echo) can theoretically be an echo of the other echo. In some examples, this comparison can be based on projecting the velocity of the closer echo onto the location of the more distant echo. In other examples, the location of the echo can be used to determine the theoretical reflection point on the line between the sensor and the more distant echo.

[0018] In some examples, the techniques described herein can also be used to confirm that an identified reflected echo is indeed a reflection. For example, the techniques described herein can determine a hypothetical reflection point (e.g., a point along the radial direction of the considered echo) where radio energy would be reflected if the considered echo were a reflected echo. In aspects of this disclosure, additional sensor information about the environment (e.g., lidar data, additional radar data, time-of-flight data, sonar data, image data, etc.) can be used to confirm the presence of an object near the reflection point. In other words, the presence of an object at the hypothetical or calculated reflection point may further indicate (and / or confirm) that the echo is a reflected echo of another echo.

[0019] Furthermore, in some implementations, the techniques described herein can identify a subset of all radar echoes as potential reflected echoes for investigation purposes. For example, radar echoes can be filtered to include only those that are likely reflected echoes and / or may have a non-negligible impact on vehicle operation. In some examples, echoes associated with a trajectory or other previously obtained information can be designated as object echoes, and therefore these echoes are not potential reflected echoes. Additionally, echoes that are relatively closer to the vehicle (e.g., radially closer) than object echoes will not be reflected echoes, and therefore can be excluded from consideration using the techniques described herein. Furthermore, such radar echoes can also be ignored, for example, because the vehicle's planning system does not consider radar echoes with speeds equal to or below a threshold speed. In some examples, the threshold speed may vary with distance from the vehicle. In at least some examples, filtering such radar echoes can reduce the amount of time, processing, and / or memory required to determine whether an echo is a reflection or an actual object.

[0020] In some examples, information about reflected radar echoes can be output for use by the autonomous vehicle's vehicle computing equipment to control the vehicle's safe passage through the environment. For example, the vehicle computing equipment can exclude reflected echoes from route and / or trajectory planning. In this way, the autonomous vehicle will not brake, steer, or otherwise act in response to phantom "objects." Furthermore, the vehicle computing equipment may not track or otherwise follow echoes identified as reflected echoes, which can, for example, reduce processing load.

[0021] The techniques discussed in this paper can improve the functionality of computing devices in a variety of ways. For example, in determining control for a vehicle, the amount of data to be considered can be reduced, for instance, by excluding reflected echoes, thereby reducing excessive resources dedicated to unnecessary determinations about the environment. Improved trajectory generation can improve safety outcomes and enhance the driver experience (e.g., by reducing unnecessary braking in response to phantom objects, steering to avoid such objects, etc.). These and other improvements to computer functionality and / or user experience are discussed in this paper.

[0022] The techniques described herein can be implemented in a variety of ways. Example implementations are provided below with reference to the accompanying figures. Although discussed in the context of autonomous vehicles, the methods, apparatus, and systems described herein can be applied to a wide range of systems (e.g., robotic platforms) and are not limited to autonomous vehicles. In another example, these techniques can be utilized in an aviation or maritime context, or in any system using machine vision.

[0023] Figure 1 This is a schematic view of the environment 100 in which vehicle 102 operates. In the example shown, vehicle 102 is moving within the environment, but in other examples, vehicle 102 may be stationary and / or parked within environment 100. Vehicle 102 includes one or more radar sensor systems 104 that capture data representing environment 100. By way of example and not limitation, vehicle 102 may be an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration, which describes vehicles capable of performing all safety-critical functions throughout a journey where a driver (or occupant) is not expected to control the vehicle at any time. In such an example, because vehicle 102 may be configured to control all functions from start to stop (including all parking functions), the vehicle may be driverless. This is merely an example, and the systems and methods described herein can be incorporated into any land, air, or water vehicle, ranging from vehicles requiring manual control by a driver at all times to vehicles with partially or fully autonomous control. Further details associated with vehicle 102 are described below.

[0024] Vehicle 102 may travel through environment 100 generally in the direction indicated by arrow 106 relative to one or more other objects. For example, environment 100 may include dynamic objects such as additional vehicle 108 (traveling generally in the direction indicated by arrow 110) as well as static objects such as first parked vehicle 112(1), second parked vehicle 112(2), third parked vehicle 112(3) (collectively referred to as “parked vehicle 112”), first building 114(1), second vehicle 114(2), and third vehicle 114(3) (collectively referred to as “building 114”). Additional vehicle 106, parked vehicle 112, and building 114 are merely examples of objects that may be in environment 100; additional and / or different objects may also be located or alternatively located in environment 100, including but not limited to vehicles, pedestrians, cyclists, trees, street signs, fixed objects, etc.

[0025] In at least one example, and as noted above, vehicle 102 may be associated with and may be mounted on radar sensor(s) ...

[0026] exist Figure 1In the example, radio energy emitted by (multiple) radar sensors 104 can generally reach the attached vehicle 108 along the direction of arrow 116, and is reflected back to (multiple) radar sensors 104, for example, generally along the direction of arrow 118. In this example, the radio energy is generally emitted and returned along the same path, which can be object echo path 120. In the example shown, object echo path 120 is essentially a straight line. Radio energy reflected along object echo path 120 is captured by (multiple) radar sensors 104, for example, as object echo 122. Object echo 122 is shown as a square at a location determined based on the captured data. For example, information associated with object echo 122 may include information indicating location in the environment, such as the location of attached vehicle 108. Location information may include distance and orientation relative to vehicle 102, or location in a local or global coordinate system. Moreover, in an implementation, object echo 122 may include signal strength information. For example, signal strength information may indicate the type of object. More specifically, radio waves can be strongly reflected by objects having certain shapes and / or components. For example, broad, flat surfaces and / or sharp edges have higher reflectivity than circular surfaces, and metals have higher reflectivity than people. In some instances, signal strength can include radar cross-section (RCS) measurements. The object echo 122 can also include speed information. For example, the speed of the auxiliary vehicle 108 can be based on the frequency of the radio energy reflected by the auxiliary vehicle 108 and / or the time when the reflected radio energy was detected.

[0027] The object echo 122 can be an example of accurate data captured by (e.g., corresponding to) the additional vehicle 108 by (multiple) radar sensors 104. However, (multiple) radar sensors 104 can also capture echoes that may be less reliable. For example, Figure 1It is also shown that radio energy emitted by the radar sensors 104 generally in the direction of arrow 124 can be reflected from the first building 114(1) to travel generally in the direction of arrow 126. In the example shown, the radio energy reflected from the first building 114(1) can then be reflected by the attached vehicle 108 back to the first building 114(1) generally in the direction of arrow 128 (i.e., opposite to the direction of arrow 126). Finally, the radio energy can then be reflected again from the first building 114(1) and return generally in the direction of arrow 130 to the radar sensors 104. Thus, the radio energy reflected from the attached vehicle 108 can return to the radar sensors 104 along a first reflection echo path 132, which includes a first branch 134 between the attached vehicle 108 and the first building 114(1), and a second branch 136 between the first building 114(1) and the vehicle 102 (e.g., the radar sensors 104). The reflected energy can be captured by (multiple) radar sensors 104 as a first reflected echo 138. For example... Figure 1 As shown, the first reflected echo 138 is received along the direction of the second branch 136 (e.g., along arrows 124, 130), but has a range equal to the distance of the first reflected echo path 132 (e.g., the sum of the distances of the first branch 134 and the second branch 136). Figure 1 As will be appreciated from the foregoing description, the first reflected echo 138 indicates the presence of an object at the location corresponding to the first reflected echo 138 (i.e., along the direction of the second branch 136). However, as described herein, such an "object" is a phantom "object" that does not exist.

[0028] Figure 1It is shown that the radar sensors(s) 104 can also capture a second reflected echo 140 corresponding to the radio energy reflected from the first parked vehicle 112(1) and the auxiliary vehicle 108. More specifically, the radio energy emitted by the radar sensors(s) 104 generally in the direction of arrow 142 can be reflected from the first parked vehicle 112(1) to travel generally in the direction of arrow 144. The radio energy can then contact and be reflected from the auxiliary vehicle 108, thus returning to the first parked vehicle 112(1) generally in the direction of arrow 146 (i.e., opposite to the direction of arrow 144). Finally, the radio energy can then again be reflected generally from the first parked vehicle 116(1) in the direction of arrow 148. Therefore, the radio energy corresponding to the second reflected echo 140 can travel along the second reflected echo path 154, which includes a first branch 156 between the auxiliary vehicle 108 and the first parked vehicle 112(1), and a second branch 158 between the first parked vehicle 112(1) and vehicle 102 (e.g., (a plurality of) radar sensors 104). Figure 1 As shown, the second reflected echo includes information about the radio energy received along the second branch 158 (e.g., along the direction of arrows 142, 148), but has a range equal to the distance of the second reflected echo path 154 (e.g., the sum of the distance of the first branch 156 and the second distance of the second branch 158). Figure 1 As will be appreciated from the foregoing description, the second reflected echo 140 indicates the presence of an object at the location corresponding to the second reflected echo 140 (i.e., along the direction of the second branch 158). However, as described herein, such an "object" is a phantom "object" that does not exist.

[0029] In addition to including Figure 1In addition to the location information shown (e.g., from distance and orientation), the object echo 122, the first reflected echo 138, and the second reflected echo 140 (hereinafter referred to as "reflected echoes 138, 140") may also include velocity information. More specifically, the object echo 122 may include information about the object velocity 160, for example, its rate (of the attached vehicle 108) along the direction of arrow 118. Similarly, the first reflected echo 138 may include information about the first reflected echo velocity 162 (e.g., the rate along the direction of the second branch 136 of the first reflected echo path 132), and the second reflected echo 140 may include information about the second reflected echo velocity 164 (e.g., the rate along the direction of the second branch 158 of the second reflected echo path 154). Therefore, object echo 122 provides information about the attached vehicle 108 (e.g., position, speed), while first reflected echo 138 indicates that an object (phantom object) is approaching from a location associated with that echo at a first reflected echo speed 162, and second reflected echo 140 indicates that an object (phantom object) is approaching from a location associated with that echo at a second reflected echo speed 164. As described herein, vehicle 102 may include a planning system and other functions that can determine a route, trajectory, and / or control relative to an object in environment 100 based on received sensor data. However, the planning system using reflected echoes 138, 140 can plan responses to objects that do not actually exist.

[0030] The techniques described in this paper can improve the accuracy and performance of planning systems by identifying echoes (e.g., echoes 138, 140) as reflected echoes. For example, as Figure 1 As further shown, the radar sensor 104 may be one of a plurality of sensor systems 164 associated with the vehicle 102. In some examples, the sensor system 164 may also include one or more additional sensors 166, which may include additional radar sensors, light detection and ranging (LiDAR) sensors, ultrasonic transducers, sound navigation and ranging (sonar) sensors, positioning sensors (e.g., Global Positioning System (GPS), compass, etc.), inertial sensors (e.g., inertial measurement units), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), wheel encoders, microphones, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), etc.

[0031] For example Figure 1As shown, radar sensor(s) 104 can generate radar data 168, and additional sensors(s) 166 can generate sensor data 170. Radar data 168 may include information about object echo 122, first reflected echo 138, and second reflected echo 140, including but not limited to position, velocity, and / or other information associated therewith. For some examples, radar data 168 may also include signal strength information, which may include RCS measurements. Radar data may also include sensor-specific information, including but not limited to sensor orientation (e.g., sensor posture relative to the vehicle), pulse repetition frequency (PRF) or pulse repetition interval (PRI) for the sensor, field of view, or detection arc, etc. In some implementations, certain aspects of radar data 168 (e.g., field of view, orientation, PRF, or PRI, etc.) may be pre-configured, in which case the data may be available (e.g., in a storage device) and may not be transmittable with the radar echo, for example. Sensor data 170 may include any information about environment 100 and / or additional sensors(s) 166. By way of non-limiting example, when the additional sensors(s) 166 include a lidar sensor, sensor data 170 may include point cloud data, and when the additional sensors(s) 166 include a camera, sensor data 170 may include image data. The lidar sensors(s) 104 and the additional sensors(s) 166 may generate radar data 168 and sensor data 170 respectively at predetermined intervals, which may be the same or different for different sensors in the sensor system 164. For example, the lidar sensors(s) 104 may have a scanning frequency at which they capture echoes and generate radar data 168.

[0032] Radar data 168 and sensor data 170 can be received at one or more vehicle computing devices 172 and utilized by the vehicle computing devices(s) 172 to perform planning, for example, using a planning system (not shown). In the example shown, sensor system 164 and the vehicle computing devices(s) 172 are part of vehicle 102 (e.g., mounted on vehicle 102). However, in other examples, some or all of sensor system 164 and / or the vehicle computing devices(s) 172 may be located separately from and / or remotely from vehicle 102. In such an arrangement, data acquisition, processing, command, and / or control can be transmitted to / from vehicle 102 by one or more remote computing devices via wired and / or wireless networks.

[0033] In at least one example, the vehicle computing device 172 can utilize radar data 168 and sensor data 170 captured by the sensor system 164 in the reflection recognition component 174. For example, the reflection recognition component 174 can receive radar data 168 including object echo 122, a first reflected echo 138, and a second reflected echo 140, and the reflection recognition component 174 can determine that reflected echoes 138 and 140 are reflected echoes, not echoes associated with actual objects in the environment. By making this distinction, the reflection recognition component 174 can send only object echo 122 (instead of reflected echoes 138 and 140) to the planning system and / or use additional echoes to further refine the estimated position and / or velocity of the object. Therefore, control planning can be implemented without considering reflected echoes 138 and 140. See below for reference. Figure 2 In more detail, the reflection identification component 174 can determine whether an echo (e.g., a candidate echo or an unconfirmed echo) is a reflected echo by comparing two echoes (e.g., the object echo 122 and an additional echo (e.g., echoes 138 and / or 140)). For example, the reflection identification component 174 can “project” a known object echo, such as the object echo 122, onto a line extending along the direction associated with the unconfirmed echo (e.g., the direction extending radially from (multiple) radar sensors 104 and through the unconfirmed echo). See below for reference. Figure 2 The technique for projecting object echo 122 is described in more detail, but the projection of the echo will have a velocity component that has the same direction as the velocity associated with the unidentified echo (e.g., radial direction relative to the radar sensor(s)104). The reflection identification component 174 can also determine the magnitude of the velocity associated with the projection of object echo 122. In some implementations, the reflection identification component 174 can determine that the unidentified echo is (or is very likely to be) a reflected echo when this magnitude corresponds to the magnitude of the velocity of the unidentified echo. As used herein, magnitudes can correspond when they are substantially equal (e.g., within a certain threshold range or error margin).

[0034] The reflection recognition component 174 may utilize some prior knowledge of objects in the environment and / or the environment itself. For example, the reflection recognition component 174 may project only the echoes of known objects. In some examples, this knowledge may come from tracking objects and / or previously identified objects, such as the attached vehicle 108. For example, such tracking and / or identification may be based on previously captured radar data 168 and / or previously captured sensor data 170 used to sense objects. For example, objects such as the attached vehicle 108 may be tracked for a few seconds and / or over an extended distance. On the other hand, since the reflected echo requires alignment of vehicle 102, attached vehicle 108, and intermediate reflective objects / surfaces (e.g., the first parked vehicle 112(1) and the first building 114(1)), the reflected echo may be more transient. As one or more of these objects move, the conditions that allow the reflected echo to be generated (e.g., relative alignment and / or orientation) may disappear, causing the reflected echo to disappear from subsequent scans of the radar sensors(s) 104. Alternatively or concurrently, the map data available to vehicle 102 (e.g., map data that can be downloaded at any time based on the location of vehicle 102 or that can be otherwise accessed by the vehicle) may include a three-dimensional representation of the environment, such as a mesh of a local three-dimensional environment. In such an example, reflection recognition component 174 may determine that the first reflected echo 138 is a non-physical echo based on knowledge of the corresponding building 114(1).

[0035] The reflection identification component 174 may also include the function of confirming that it has determined a candidate echo to be a reflected echo. In some instances, the reflection identification component 174 may, for example, attempt to track the echo received as radar data 168 via subsequent scans. As noted above, relative movement between vehicle 102 and the attached vehicle 108 can cause reflection conditions to weaken, resulting in the inability to receive subsequent echoes corresponding to the “object” associated with the reflected echo. However, ignoring an object before its trajectory can be verified through continuously collected radar scans could slow reaction time, which could be unsafe. Therefore, in other implementations, the reflection identification component 174 may, for example, use sensor data 170 and / or radar data 168 to determine whether a reflecting object is positioned along the direction of the reflected echo. Reference Figure 1In some examples, lidar data, image data, etc., can confirm the presence of the first parked vehicle 112(1) and / or the first building 114(1). Therefore, the reflection recognition component 174 can identify the first parked vehicle 112(1) and / or the first building 114(1) and confirm that the echo is a previously determined reflection echo 138 based on the location of these objects (e.g., at an intersection between branches of the corresponding reflected echo paths 132, 154). Map data can help identify static, fixed objects, such as building 114, ground topology, street signs, public fixtures, etc. However, real-time or near-real-time sensor data may be required to identify movable objects, whether static or dynamic. For example, parked vehicle 112, pedestrians, cyclists, other moving vehicles, etc., may not be available through map data.

[0036] In some implementations, the reflection identification component 174 can process radar echoes to determine whether they are real-time or near-real-time reflected echoes. For example, the reflection identification component 174 can compare, for example, multiple echoes in parallel with a known object echo, such as object echo 122. In one example, all echoes in a radar scan that are not known object echoes can be compared with known object echoes to determine whether such an echo is a reflection. In some other implementations, the reflection identification component 174 can filter echoes to, for example, exclude points that are unlikely to be reflections or are physically impossible to be reflections. By way of a non-limiting example, an echo that is relatively closer than a known object echo will not be a reflection of the object echo. In other examples, the reflection identification component 174 can exclude echoes indicating velocities below a certain threshold or velocities of zero. Other filtering techniques may also be used.

[0037] While the examples described herein compare a point or echo to a known object echo 122, in other examples, such as when it is unknown whether one of the echoes is a known object echo, the reflection identification component 174 may alternatively compare echo pairs. More specifically, the techniques described herein can determine that two echoes may be reflections of each other, regardless of whether one of the echoes is an object echo. For example, when considering echo pairs, a more distant echo may be flagged or otherwise designated as a potential reflection. Further processing can serve as a basis for determining whether a closer echo is associated with an object in the environment (or a further echo is associated with a reflection). The reflection identification component 174 may perform advanced filtering as described herein to identify a subset of points used for pairwise comparisons. By way of non-limiting examples, echoes with velocities below a threshold may be ignored, and echo pairs that differ excessively in position and / or velocity (e.g., equal to or above a threshold difference) may not be compared. In at least some examples, various techniques can be combined to improve the level of certainty regarding whether an echo is a reflection.

[0038] Figure 2 This is another schematic view illustrating environment 100, an additional aspect of this disclosure. To avoid confusion, refer to... Figure 2 To be discussed in detail Figure 1 Elements in (e.g., environment 100) in Figure 1 and Figure 2 The same reference numerals are used in the accompanying drawings. Furthermore, for clarity, Figure 2 Some elements in environment 100 are shown in gray.

[0039] More specifically, Figure 2 Vehicle 102, auxiliary vehicle 108, first parked vehicle 112(1) and first building 114(1) are shown. Figure 2 The object echo 122, the first reflected echo 138, and the second reflected echo 140 are also shown. (As shown above in conjunction with...) Figure 1 The object echo 122 and reflected echoes 138, 140 discussed herein may include position and velocity information based on properties of radio energy received at radar sensor(s) 104. For example, an object echo 122 spaced a distance from vehicle 102 along line 202 passing through both vehicle 102 and auxiliary vehicle 108 may indicate both the position of the object echo (and thus the position of auxiliary vehicle 108) and the velocity of the object echo 160. Note that since the object echo 122 is known (e.g., known based on previously obtained data (e.g., trajectory) about environment 100 and / or auxiliary vehicle 108) as a direct reflection from auxiliary vehicle 108 (e.g., along line 202), the object echo can be considered an accurate representation of part of auxiliary vehicle 108. As noted above, the presence of an object (e.g., auxiliary vehicle 108) or knowledge that the echo is an object echo (e.g., object echo 122) may not be required in all instances. The techniques described herein can be applied to pairwise comparisons of echoes to determine the possibility that the echoes are reflections of each other.

[0040] The reflected echoes 138 and 140 also include at least position and velocity information associated with the radio energy received at the radar sensor(s) 104. As discussed above, the first reflected echo 138 includes information about the velocity of the first reflected echo along line 204 at the illustrated location. The distance of the first reflected echo 138 along line 204 is determined by the radar sensor(s) 104 and corresponds to a distance or range that the received radio energy has traveled (e.g., the distance corresponds to...). Figure 1The first reflected echo path 132 is shown in the diagram. Similarly, the second reflected echo 140 includes information about the velocity of the second reflected echo along line 206 at the illustrated location. The distance of the second reflected echo 140 along line 206 is determined by the radar(s) ... Figure 1 The second reflected echo path 154 is shown in the figure.

[0041] The techniques described herein can utilize the geometry in environment 100 to determine whether a radar echo is a reflected echo. For example, when captured at radar sensor(s) 104, object echo 122 and reflected echoes 138, 140 are simply echoes. Object echo 122 can only be definitively attributed to the auxiliary vehicle 108 through some prior knowledge about environment 100 and / or objects in environment 100. In some instances, a tracker or other component associated with vehicle 102 may have been tracking the auxiliary vehicle 108, and the echo is identified as object echo 122 when it matches the expectation associated with the tracking. In other examples, as described herein, object echo 122 can be considered, for example, one echo in an echo pair, regardless of whether it is attributed to the auxiliary vehicle 108. Although object echo 122 is shown as a single point at a central location in front of the auxiliary vehicle 108, object echo 122 may also correspond to one or more other echoes and / or one or more other locations on the auxiliary vehicle 108. By way of non-limiting example, the properties of the object echo (e.g., object echo speed 160) can be determined based on multiple echoes associated with the additional vehicle 108. For example, object echo 122 could be the average of multiple echoes from the additional vehicle 108.

[0042] As noted above, unlike object echo 122, reflected echoes can be transient, occurring only under certain conditions (e.g., geometric conditions, which may cause the reflected echo to move irregularly, appear, or disappear). Therefore, there is no prior knowledge about the trajectory or other aspects of the reflected echo. However, the techniques described herein determine whether an echo is a reflection or can directly correspond to an actual object in environment 100.

[0043] As noted above, the first reflected echo 138 corresponds to an echo located along line 204. To characterize the first reflected echo 138 as a reflected echo, the techniques described herein can treat the echo together with the object echo 122 as a pair. For example, since the direction of line 204 is known and the position of the first reflected echo 138 on line 204 relative to the radar sensor is known, the object echo 122 can be projected onto the position of the first reflected echo. Conceptually, projecting the object echo 122 may include defining a line 208 that connects the position associated with (e.g., along radial line 202) the object echo 122 and the position associated with (e.g., along radial line 204) the first reflected echo 138. Again, as shown, line 210, perpendicular to and bisects line 208, intersects line 204 at reflection point 212; therefore, the distance of line segment 214 between reflection point 212 and object echo 122 is equal to the distance of line segment 216 between reflection point 212 and first reflected echo 138. Furthermore, the object echo velocity 160 can be reflected around line 210 as a first reflection velocity 218. Once the first reflection velocity 218 is determined, it can be resolved into two velocity components—a radial velocity component 220 generally along line 204 and a tangential velocity component 222 perpendicular to the radial velocity component 220. The radial velocity component 220 is the projection velocity of the object echo 122 along the first direction 204. In other words, the first projection velocity is the radial velocity component 220 of the first reflection velocity 218, and the first reflection velocity 218 is a mirror image of the vector representing the object echo velocity 160 about line 210.

[0044] exist Figure 2 In the example, if the radio energy reflected from the auxiliary vehicle 108 is also reflected at the reflection point 212, then the radial velocity component 220 (projection velocity) is the velocity (i.e., magnitude and direction) that will be sensed by the radar sensors 104(multiple). As will be appreciated, the radial velocity component 220 of the first reflection velocity 218 is along the same path as the first reflected echo velocity 162( Figure 1 As shown in the image, not Figure 2 (as shown in the diagram) in the same direction (e.g., along line 204). Therefore, if the radial velocity component 220 (first projection velocity) and the first reflected echo velocity 162 are substantially similar in magnitude (e.g., within a predetermined threshold or within each other), the first reflected echo 138 can be flagged, tagged, or otherwise identified as a possible reflection.

[0045] Figure 2A similar conceptualization for determining whether the second reflected echo 140 is a reflected echo is illustrated. For example, the second reflection velocity 224 at the location of the second reflected echo 140 could be a mirror image of the object echo velocity 160 about line 226. Line 226 is perpendicular to and bisects line 228 extending between the location of the object echo 122 and the location of the second reflected echo 140. The second reflection velocity 224 includes a radial velocity component 230 (i.e., along the radial line 206) and a tangential velocity component 232 (i.e., perpendicular to the radial line 206).

[0046] For example Figure 2 As shown, line 226 contacts radial line 206 at reflection point 234. As discussed above, the radial velocity component 230 of the second projected velocity 224 is the projected (e.g., desired) velocity (i.e., direction and magnitude) of the echo associated with the radio energy that bounces off the auxiliary vehicle 108 and is reflected back to radar sensor(s)(s) 104 at reflection point 234. In other words, the radial velocity component 230 is the projected velocity and corresponds to the desired echo associated with the reflection at reflection point 234. Like the first projected echo, the direction of the radial velocity component 230 is substantially the same as the direction of the second reflected echo velocity 164. That is, both are along line 206. Therefore, if the magnitude of the radial velocity component 230 (the second projected echo) is substantially the same as the magnitude of the second reflected echo velocity 164, the second reflected echo 140 can be labeled, calibrated, or otherwise identified as a possible reflection.

[0047] In the implementation of this disclosure, and as further described herein, when it is determined that reflected echoes 138 and 140 are reflected echoes (e.g., because the corresponding radial velocity components 220 and 230 are substantially the same as the velocities of the first and second reflected echoes 162 and 164), vehicle 102 may ignore (e.g., exclude from planning) reflected echoes 138 and 140. Vehicle 102 may also confirm reflections, for example, by determining the presence of an object at each of the reflection points 212 and 234. For example, vehicle 102 may use sensor data, map data, etc., to confirm the presence of an object, as described herein. In at least some examples, the echoes may be marked so that other components may know that there may be no corresponding object associated with them.

[0048] Figure 2Another example implementation is also shown, in which pedestrian 236 leaves building 238 and enters environment 100. As shown, pedestrian 236 may enter the environment at a location near the position associated with the second reflected echo 140. Multiple radar sensors 104 may generate radar data including information about the newly arrived pedestrian. Because pedestrian 236 has not been previously tracked (i.e., because pedestrian 236 has just appeared), the implementation described herein can use the techniques described herein to compare the echo associated with pedestrian 236 with a known object echo (e.g., object echo 122). For example, object echo velocity 160 may be projected onto the location associated with pedestrian 236. Because pedestrian 236 is farther from the multiple radar sensors 104 than the attached vehicle 108, the reflection point can also be identified. In the example shown, the reflection point would be near reflection point 234. As in other examples, the radial component of the projected object echo velocity at the location of the pedestrian echo can also be determined. However, since pedestrian echoes are not reflected echoes, the magnitude of the radial velocity component of the projected object echo velocity may differ. For example, unless the pedestrian walks in a manner in which the component of the pedestrian's velocity toward (multiple) radar sensors 104 matches the component of the reflected echo, the pedestrian echo will not be (correctly) identified as a reflection. When it is determined that the pedestrian echo is not a reflection, the pedestrian's movement can be tracked and / or information 236 about the pedestrian can be used to generate control for the vehicle.

[0049] This disclosure is not limited to Figure 2 The example implementation is shown below. For example, the techniques described herein can be used to identify reflections from other objects in the environment (including reflections from vehicle 102). In some examples, radio energy reflected by the attached vehicle 108 can be reflected (or bounced) from vehicle 102, travel back to the attached vehicle 108 (or other object), and be reflected again from the attached vehicle before being captured by the radar sensor(s) 104. In other words, the detected radio energy can traverse the path along line 202 four times (twice in each direction) before being captured by the radar sensor(s). This “double bounce” can result in an echo along direction 202, but at twice the distance from vehicle 102. In some examples, the magnitude of the velocity will differ from the magnitude of the object echo velocity (e.g., half its velocity), which can result in identifying the reflected echo.

[0050] In addition, although Figure 2 While the intermediate reflecting object is presented as a stationary object, in other implementations, the techniques described herein can also determine whether an echo is reflected from a moving object. For example, the velocity of the intermediate object would be known (e.g., known based on other radar echoes), and this velocity could be used to modify the radial component of the projection velocity. Other modifications are also envisioned.

[0051] Figure 3 Another schematic view of environment 100 is shown, and Figure 3 This is used to illustrate alternative methods for determining whether an echo is (or could be) a reflection of another echo. More specifically, Figure 3 A technique for using geometry to determine a reflection point can be described, and that reflection point can then be compared with environmental information to, for example, determine whether the reflection point corresponds to an object in the environment.

[0052] More specifically, Figure 3 The object echo 122 (leaving the attached vehicle 108) and the first reflected echo 138 are shown. For clarity, the second reflected echo 140 is not shown in the figure, although the techniques described with reference to echo pairs including the first reflected echo 138 and the object echo 122 can be applied to any radar echo pair (e.g., these radar echo pairs include: object echo 122 and second reflected echo 138, first reflected echo 138 and second reflected echo 140, and / or any other echo pair). As described herein, although prior knowledge of the environment 100 may allow the object echo 122 to be associated with the attached vehicle 108, the implementation of this disclosure is equally applicable to any radar echo pair, regardless of whether it is associated with a known object and / or regardless of the availability of prior knowledge of the environment 100. Therefore, although the echoes are labeled as “object” echo and “reflected” echo, one or both of these labels may not be known until after the attachment process. In at least some examples, such labels can be used to help guide autonomous vehicles. For example, although the importance of "reflected" echoes is reduced, "reflected" echoes can still be considered for navigation or in other ways during planning.

[0053] As described herein, the radar sensors 104 may receive only information about the range, position, and / or velocity (rate) of the echo. Therefore, for example, the radar sensors 104 may generate sensor data indicating the position of the object echo 122 and the position of the first reflected echo 138. For example, in Figure 3 middle, Figure 3Line 302 is the line between the location of object echo 122 and the location of radar sensor(s) 104, and line 304 is the line between the location of the first reflected echo 138 and the location of radar sensor(s) 104. Based on the echoes 122, 138 (i.e., the locations of echoes 122, 138), the lengths of lines 302, 304 and the angle 314 between lines 302, 304 are known (or can be easily determined). The technique described herein can use lines 302, 304 and angle 314 to determine a potential or theoretical reflection point 308 along line 304, at which radio energy initially reflected from an object (e.g., an auxiliary vehicle 108) associated with object echo 122 will be reflected before being received at radar sensor(s) 104. Once the location of theoretical reflection point 308 is determined, the technique described herein can determine whether an object is present at theoretical reflection point 308, thereby confirming (or at least indicating) that the first reflected echo 138 is a reflection of the object echo.

[0054] More specifically, the distance between the first reflected echo 138 and the object echo 122 can be determined using the distance between lines 302 and 304 and angle 306 (e.g., Figure 3 (The distance of line 310 shown). For example, the length of line 310 can be determined using the law of cosines, but this is not necessary. The theoretical reflection point 308 can then be determined as the intersection of lines 304 and 312, with line 312 bisecting and perpendicular to line 310 extending between echoes 122 and 138. For example, the angle 314 between line 304 and line 310, extending between sensor 304 and the first reflected echo 138, and line 310 extending between the first reflected echo 138 and the object echo 122, can be determined using simple geometry. The length of line segment 316 between the first reflected echo 138 and the reflection angle 316 bisected by line 312 can then be easily determined to provide the position of the reflected echo 308 along line 304. Furthermore, if the first reflected echo 138 is a true reflection, the distance of line segment 316 will be equal to the distance from the object echo 122 to the reflection point 308. This additional information can be used (alternatively or alternatively) to calculate the location 308 of the reflection point.

[0055] In the implementation described above, the potential reflection point 308 can be determined using only geometry. For example, velocity information about the echo is not required. However, the potential reflection point can be determined for any pair of echoes at different distances. Therefore, the techniques described herein can also determine whether an object is present at the potential reflection point to determine whether one of the echoes (e.g., a more distant echo) is a reflection. For example, sensor data (including but not limited to lidar data, image data, additional radar data, etc.) can be used to determine whether an object is present near the location of the theoretical reflection point 308. In the example shown, image data, lidar data, and / or other sensor data captured by vehicle 102 can be used to identify building 114 (1). In other embodiments, map data can confirm the presence of an object at reflection point 308. In some examples, velocity information can be used at least in part to determine the potential reflection based on projecting the velocity from the echo onto a unit vector associated with another echo. This projected velocity can then be directly compared to the velocity of the other echo.

[0056] Figure 2 and Figure 3 (and below) Figure 5 and Figure 6 (Refer to) Figure 1 The components shown are illustrated through examples. Figure 1 The environment described in 100. However, refer to Figure 2 , Figure 3 , Figure 5 and Figure 6 The examples described are not limited to execution or use in environment 100. Figure 1 The component execution. For example, see reference. Figure 2 , Figure 3 , Figure 5 and Figure 6 Some or all of the examples described can be derived from Figure 4 It may be executed by one or more components as described herein, or by one or more other systems or components.

[0057] Figure 4 A block diagram depicts an example system 400 for implementing the techniques described herein. In at least one example, system 400 may include a vehicle 402, which can be coupled with… Figure 1 The vehicle 102 shown may be the same as or different from the one shown.

[0058] Vehicle 402 may include vehicle computing devices 404, one or more sensor systems 406, one or more transmitters 408, one or more communication connections 410, one or more drive modules 412, and at least one direct connection 414.

[0059] Vehicle computing device 404 may include one or more processors 416, and a memory 418 communicatively coupled to the one or more processors 416. In the illustrated example, vehicle 402 is an autonomous vehicle. However, vehicle 402 may be any other type of vehicle, or any other system with at least one sensor (e.g., a camera-enabled smartphone). In the illustrated example, the memory 418 of vehicle computing device 404 stores a positioning component 420, a perception component 422, a prediction component 424, a planning component 426, a reflection recognition component 428, one or more system controllers 430, one or more maps 432, and a tracker component 434. Although for illustrative purposes, Figure 4 The positioning component 420, sensing component 422, prediction component 424, planning component 426, reflection recognition component 428, (multiple) system controllers 430, (multiple) maps 432, and / or tracker component 434 are depicted as residing in memory 418; however, it is conceivable that these components may additionally or alternatively be accessible to vehicle 402 (e.g., stored in memory remote from vehicle 402 or otherwise accessible by memory remote from vehicle 402). In some instances, (multiple) vehicle computing devices 404 may correspond to Figure 1 The vehicle computing system 172 (or multiple vehicle computing systems 172) may be an example of such a system.

[0060] In at least one example, the localization component 420 may include the function of receiving data from the sensor system(s) 406 to determine the position and / or orientation (e.g., one or more of x-position, y-position, z-position, roll, pitch, or yaw) of the vehicle 402. For example, the localization component 420 may include and / or request / receive a map of the environment and may continuously determine the autonomous vehicle's localization and / or orientation within the map. In some instances, the localization component 420 may utilize SLAM (Simultaneous Localization and Mapping), CLAMS (Simultaneous Calibration, Localization and Mapping), relative SLAM, beamforming, nonlinear least squares optimization, etc., to receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc., to accurately determine the autonomous vehicle's localization. In some instances, the localization component 420 may provide data to various components of the vehicle 402 to determine the initial position of the autonomous vehicle 402, thereby generating a trajectory.

[0061] In some instances, perception component 522 may include functionality for performing object detection, segmentation, and / or classification. In some examples, perception component 422 may provide processed sensor data indicating the presence of an object approaching vehicle 402 and / or classifying that object into an object type (e.g., car, pedestrian, cyclist, animal, building, tree, road surface, curb, sidewalk, unknown object, etc.). In additional and / or alternative examples, perception component 422 may provide processed sensor data indicating one or more characteristics associated with a detected object (e.g., a tracked object) and / or the environment in which that object is located. In some examples, characteristics associated with an object may include, but are not limited to, x-position (global and / or local position), y-position (global and / or local position), z-position (global and / or local position), orientation (e.g., roll, pitch, or yaw), object type (e.g., classification), object velocity, object acceleration, object range (size), etc. Characteristics associated with the environment may include, but are not limited to, the presence of another object in the environment, the state of another object in the environment, time of day, day of week, season, weather conditions, darkness / light indication, etc. In some examples, the sensing component 422 can use radar data to identify objects and can receive information about reflected echoes, for example, to include / exclude sensor data, as described herein.

[0062] In some examples, prediction component 424 may include functionality for generating predicted trajectories of objects in the environment. For example, prediction component 424 may generate one or more predicted trajectories for vehicles, pedestrians, animals, etc., within a threshold distance of vehicle 402. In some instances, prediction component 424 may measure the path of an object and generate the object's trajectory. In some instances, prediction component 424 may cooperate with tracker 434 to track an object as it moves through the environment. In some examples, information from prediction component 424 may be used when determining whether a radar echo originates from a known object.

[0063] Typically, the planning component 426 can determine the path that vehicle 402 will follow through the environment. For example, the planning component 426 can determine various routes and trajectories, as well as various levels of detail. For example, the planning component 426 can determine a route from a first location (e.g., the current location) to a second location (e.g., the target location). For the purposes of this discussion, the route can be a sequence of landmarks for traveling between two locations. As a non-limiting example, landmarks include streets, intersections, Global Positioning System (GPS) coordinates, etc. Furthermore, the planning component 426 can generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 426 can determine how to guide the autonomous vehicle from a first landmark in the landmark sequence to a second landmark in the landmark sequence. In some examples, the instructions can be a trajectory or a portion of a trajectory. Furthermore, in some implementations, multiple trajectories can be generated substantially simultaneously (e.g., within technical tolerances) according to a rolling time-domain technique, wherein one of the multiple trajectories is selected for navigation of vehicle 402. In some instances, the planning component 426 may generate one or more trajectories for the vehicle 402 based at least in part on sensor data (e.g., radar echoes). For example, the planning component 426 may exclude echoes that are identified as reflected echoes.

[0064] Typically, the reflection recognition component 428 may include functionality for identifying whether sensor data (e.g., radar echo) corresponds to an actual object or to a reflection of the object from some intermediate object. In some instances, the reflection recognition component 428 may correspond to... Figure 1 The reflection recognition component 424 is included in the design. As discussed herein, the reflection recognition component 428 can receive radar data, lidar data, image data, map data, etc., to determine whether the sensed object is an actual object or merely a reflection of an actual object. In some instances, the reflection recognition component 428 can provide sensor information determined not to correspond to a reflection to the planning component 426 to determine when to control the vehicle 402 to traverse the environment. In some instances, the reflection recognition component 428 can exclude sensor data determined to correspond to a reflection from the planning component 426, for example, causing the planning component 426 to determine control while excluding sensor data (e.g., radar data determined to be associated with a reflection from an intermediate object). The reflection recognition component 428 can also provide sensor information to the tracker 434, for example, causing the tracker to track dynamic objects in the environment.

[0065] The (multiple) system controllers 430 can be configured to control, for example, the steering system, propulsion system, braking system, safety system, transmitter system, communication system, and other systems of the vehicle 402 based on control generated by the planning component 426 and / or based on information from the planning component 426. The (multiple) system controllers 430 can communicate with and / or control corresponding systems of the (multiple) drive modules 414 and / or other components of the vehicle 402.

[0066] Multiple maps 432 can be used by vehicle 402 for navigation within the environment. For the purposes of this discussion, maps can be any number of data structures modeled in two, three, or N dimensions, capable of providing information about the environment, such as, but not limited to, topology (e.g., intersections), streets, mountains, roads, terrain, and the general environment. In some instances, maps can include, but are not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), intensity information (e.g., LiDAR information, radar information, etc.), spatial information (e.g., image data projected onto a grid, individual "surfels" (e.g., polygons associated with a single color and / or intensity)), and reflectivity information (e.g., specular reflectivity information, retroreflectivity information, BRDF information, BSSRDF information, etc.). In one example, a map may include a three-dimensional mesh of the environment. In some instances, maps can be stored in a tile format, such that individual tiles of the map represent discrete portions of the environment and can be loaded into working memory as needed. In some examples, the (multiple) maps 432 may include at least one map (e.g., an image and / or a grid). Vehicle 402 may be controlled at least in part based on the (multiple) maps 432. That is, the (multiple) maps 432 may be used in conjunction with the positioning component 420, the perception component 422, the prediction component 424, the planning component 426, and / or the reflection recognition component 432 to determine the location of vehicle 402, identify objects in the environment, and / or generate routes and / or trajectories for navigation within the environment. Furthermore, and as described herein, the (multiple) maps 432 may be used to verify the presence of objects (e.g., intermediate objects that can be reflected from them before radio energy is received at a radar sensor).

[0067] In some examples, maps(s) 432 may be stored on one or more remote computing devices (e.g., one or more computing devices 438) accessible via one or more networks 436. In some examples, maps(s) 432 may include multiple similar maps stored based on characteristics such as entity type, time of day, day of week, season of year, etc. Storing multiple maps 432 in this way can have similar memory requirements but increases the speed at which data can be accessed in the maps.

[0068] Tracker 434 may include functionality for tracking (e.g., following) the motion of an object. For example, tracker 434 may receive sensor data representing dynamic objects in the vehicle's environment from one or more of sensor systems 406. For example, an image sensor on vehicle 402 may capture image sensor data, a lidar sensor may capture point cloud data, and a radar sensor may obtain echoes indicating the position, pose, etc., of an object in the environment at multiple times. Based on this data, tracker 434 may determine tracking information for the object. For example, trajectory information may provide the associated historical position, velocity, acceleration, etc., of the object. Furthermore, in some instances, tracker 434 may track objects in occluded areas of the environment. U.S. Patent Application No. 16 / 147,177, filed September 28, 2018, entitled "Radar Spatial Estimation" (the entire disclosure of which is incorporated herein by reference), describes techniques for tracking objects in occluded areas. As described herein, information from tracker 434 can be used to verify that a radar echo corresponds to a tracked object. In at least some examples, tracker 434 may be associated with sensing component 422, causing sensing component 422 to perform data association to determine whether a newly identified object should be associated with a previously identified object.

[0069] As will be understood, for illustrative purposes, the components discussed herein (e.g., localization component 420, perception component 422, prediction component 424, planning component 426, reflection recognition component 428, (multiple) system controllers 430, (multiple) maps 432, and trackers 434) are described separately. However, the operations performed by the various components can be combined or performed in any other component. By way of example, the reflection recognition function can be performed by perception component 422 and / or planning system 426 (e.g., instead of reflection recognition component 428) to reduce the amount of data transmitted by the system.

[0070] In some instances, some or all of the aspects of the components discussed herein may include models, algorithms, and / or machine learning algorithms. For example, in some instances, the components in memory 418 (and / or memory 442 discussed below) may be implemented as neural networks.

[0071] As described herein, the example neural network is a biologically inspired algorithm that passes input data through a sequence of connected layers to produce an output. Each layer in a neural network may also include another neural network, or may include any number of layers (whether convolutional or not). As will be understood in the context of this disclosure, neural networks can utilize machine learning, which can refer to a large class of such algorithms that generate outputs based on learned parameters.

[0072] Although discussed in the context of neural networks, any type of machine learning can be used in accordance with this disclosure. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimation scatter smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, minimum absolute shrinkage and selection operator (LASSO), elastic net, minimum angle regression (LARS)), decision tree algorithms (e.g., classification and regression tree (CART), iterative bisectioner 4 (ID3), chi-square automatic interaction detection (CHAID), decision stump, conditional decision tree), Bayesian algorithms (e.g., Naive Bayes, Gaussian Naive Bayes, multinomial Naive Bayes, average one-dependency estimator (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k-means, k-median, expectation maximization (EM), hierarchical clustering), and association rule learning algorithms. (e.g., perceptron, backpropagation, hopfield network, radial basis function network (RBFN)), deep learning algorithms (e.g., deep Boltzmann machine (DBM), deep belief network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projective pursuit, linear discriminant analysis (LDA), mixture discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting, bootstrap aggregation (bagging), AdaBoost, stacked generalization (mixture), gradient boosting machine (GBM), gradient boosting regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc.).

[0073] Additional examples of architectures include neural networks, such as ResNet70, ResNet101, VGG, DenseNet, PointNet, etc.

[0074] In at least one example, the sensor system 406 may include lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, positioning sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time-of-flight, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), etc. The sensor system 406 may include multiple instances of each of these or other types of sensors. For example, lidar sensors may include individual lidar sensors located at corners, front, rear, sides, and / or top of vehicle 402. As another example, camera sensors may include multiple cameras disposed at various locations around the exterior and / or interior of vehicle 402. As another example, radar sensors may include multiple instances of the same or different radar sensors disposed at various locations around vehicle 402. The sensor system 406 may provide input to vehicle computing device 404. Alternatively or concurrently, the (multiple) sensor systems 406 may transmit sensor data to the (multiple) computing devices 438 via one or more networks 436 at a specific frequency, after a predetermined time period, or in near real-time. In some instances, the (multiple) sensor systems 406 may correspond to Figure 1 The sensor system 164 includes multiple radar sensors 104 and / or multiple additional sensors 166.

[0075] Multiple transmitters 408 may be configured to emit light and / or sound. In this example, multiple transmitters 408 may include internal audio and visual transmitters for communication with passengers of vehicle 402. By way of example, and not limitation, internal transmitters may include: speakers, lights, signs, displays, touchscreens, haptic transmitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seatbelt pretensioners, seat positioners, headrest positioners, etc.), etc. In this example, transmitter 408 may also include external transmitters. By way of example, and not limitation, external transmitters in this example include lights or other indicators (e.g., indicator lights, signs, light arrays, etc.) for signaling direction of travel, and one or more audio transmitters (e.g., speakers, speaker arrays, horns, etc.) for audible communication with pedestrians or other nearby vehicles, one or more of these transmitters incorporating beam steering technology.

[0076] Multiple communication connections 410 enable communication between vehicle 402 and one or more other local or remote computing devices. For example, multiple communication connections 410 can facilitate communication with multiple other local computing devices and / or multiple drive modules 414 on vehicle 402. Furthermore, multiple communication connections 410 can allow the vehicle to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.). Multiple communication connections 410 also enable vehicle 402 to communicate with remotely operated computing devices or other remote services.

[0077] Multiple communication connections 410 may include physical and / or logical interfaces for connecting vehicle computing device 404 to another computing device or network (e.g., multiple networks 436). For example, multiple communication connections 410 may enable Wi-Fi-based communication, such as via frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies (e.g., ...). ), cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.) or any suitable wired or wireless communication protocol that enables the corresponding computing device to connect with (multiple) other computing devices.

[0078] In at least one example, vehicle 402 may include multiple drive modules 412. In some examples, vehicle 402 may have a single drive module 412. In at least one example, vehicle 402 may have multiple drive modules 412, wherein a single drive module among the drive modules 412 is positioned at opposite ends of vehicle 402 (e.g., front and rear, etc.). In at least one example, the multiple drive modules 412 may include one or more sensor systems to detect the conditions of the multiple drive modules 412 and / or the surrounding environment of vehicle 402. By way of example and not limitation, the multiple sensor systems associated with the multiple drive modules 412 may include: one or more wheel encoders (e.g., rotary encoders) to sense the rotation of the wheels of the drive modules; inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) to measure the orientation and acceleration of the drive modules; cameras or other image sensors; ultrasonic sensors to acoustically detect objects in the environment surrounding the drive modules; lidar sensors; radar sensors, etc. For the multiple drive modules 412, some sensors such as wheel encoders may be unique. In some cases, the multiple sensor systems on the drive module 412 may overlap or complement the corresponding systems of the vehicle 402 (e.g., multiple sensor systems 406).

[0079] The (multiple) drive modules 412 may include a number of vehicle systems within the vehicle system, including: a high-voltage battery, an electric motor propelling the vehicle, an inverter converting direct current from the battery into alternating current for use by other vehicle systems, a steering system including a steering motor and a steering frame (which may be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate traction loss and maintain control, an HVAC system, lighting (e.g., headlights / taillights for illuminating the vehicle's exterior environment), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC / DC converters, high-voltage junctions, high-voltage cables, charging systems, charging ports, etc.). Additionally, the (multiple) drive modules 412 may include a drive module controller that can receive and preprocess data from the (multiple) sensor systems and control the operation of various vehicle systems. In some examples, the drive module controller may include one or more processors and a memory communicatively coupled to the one or more processors. The memory may store one or more modules to perform various functions of the (multiple) drive modules 412. In addition, the (multiple) drive modules 412 may also include one or more communication connections that enable the respective drive module to communicate with one or more other local or remote computing devices.

[0080] In at least one example, the direct connection 414 can provide a physical interface for coupling the drive module(s) 412 to the body of the vehicle 402. For example, the direct connection 414 can allow the transfer of energy, fluid, air, data, etc., between the drive module(s) 412 and the vehicle. In some instances, the direct connection 414 can further releasably secure the drive module(s) 412 to the body of the vehicle 402.

[0081] In at least one example, the localization component 420, sensing component 422, prediction component 424, planning component 426, reflection recognition component 428, multiple system controllers 430, multiple maps 432, and / or trackers 434 can process sensor data as described above and can transmit their respective outputs to one or more computing devices 438 via one or more networks 436. In at least one example, the localization component 420, sensing component 422, prediction component 424, planning component 426, reflection recognition component 428, multiple system controllers 430, multiple maps 432, and / or trackers 434 can transmit their respective outputs to one or more computing devices 438 at a specific frequency, after a predetermined time period, or in near real-time.

[0082] In some examples, vehicle 402 may transmit sensor data to computing devices 438, for example, via network(s) 436. In some examples, vehicle 402 may transmit raw sensor data to computing devices 438. In other examples, vehicle 402 may transmit processed sensor data and / or a representation of the sensor data (e.g., spatial grid data) to computing devices 438. In some examples, vehicle 402 may transmit sensor data to computing devices 438 at a specific frequency, after a predetermined time period, or near real-time. In some cases, vehicle 402 may transmit sensor data (raw or processed) to computing devices 438 as one or more log files.

[0083] Multiple computing devices 438 may include multiple processors 440 and memory 442, which stores one or more maps 344 and / or reflection recognition components 346.

[0084] In some instances, map(s) 344 may resemble map(s) 432. In addition to performing functions at vehicle computing devices(s) 404 or as an alternative to performing functions at vehicle computing devices(s) 404, reflection recognition component 448 may perform substantially the same functions as those described for reflection recognition component 428.

[0085] The processor(s) 416 of vehicle 402 and the processor(s) 440 of computing device(s) 438 can be any suitable processor capable of executing instructions to process data and perform the operations described herein. By way of example, and not limitation, processors 416 and 440 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or part of a device that processes electronic data to convert that electronic data into other electronic data that can be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices can be considered processors as long as they are configured to implement coded instructions.

[0086] Memory 418 and memory 442 are examples of non-transitory computer-readable media. Memory 418 and memory 442 may store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and the functions subordinate to various systems. In various implementations, the memory may be implemented using any suitable storage technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmable, and physical components, the ones shown in the figures being merely examples relevant to the discussion herein.

[0087] It should be noted that, although Figure 4 While illustrated as a distributed system, in alternative examples, components of vehicle 402 may be associated with computing devices 438 and / or components of computing devices 438 may be associated with vehicle 402. That is, vehicle 402 may perform one or more of the functions associated with computing devices 438, and vice versa. Furthermore, aspects of the reflection recognition components 428, 346, and / or tracker 434 may be performed on any of the devices discussed herein.

[0088] Figure 5 This is a flowchart of an example process 500 for determining whether a radar echo is a reflected echo according to embodiments of the present disclosure. Although discussed in the context of radar data, example process 500 can also be used and / or combined with lidar data, sonar data, time-of-flight image data and / or other types of data.

[0089] At operation 502, process 500 may include receiving radar data for the environment. For example, the radar data may include radar echoes that include location, velocity, and / or intensity information. In some examples, the radar system may include one or more radar sensors and may be a sensor system for an autonomous vehicle (e.g., vehicle 102 described above). The radar data may include multiple echoes, including static echoes corresponding to objects with location and zero velocity, and dynamic echoes or radar trajectories corresponding to moving objects with location and non-zero velocity.

[0090] At operation 504, process 500 may include determining a first velocity along a first radial direction at a first location based on a first radar echo in the radar data. The first radar echo may include a velocity oriented in a first direction between the object and the vehicle, for example, a velocity along a radial direction extending from the vehicle (on which the radar sensor) and passing through the object. In some instances, the first radar echo may be an object echo, for example, corresponding to a known object in the environment. In some examples, the correspondence between the first radar echo and the object echo may be determined based on tracking information generated prior to receiving the radar data. However, in other instances, the techniques described herein may operate without any prior knowledge of the echo. In other words, the first echo (and additional echoes including the second echo discussed below) may not be associated with an object in the environment, and in some instances such an association may be unnecessary. The location of the first radar echo may be a distance along the first radial direction.

[0091] At operation 506, process 500 may include determining a second velocity along a second radial direction at a second location based on a second radar echo in the radar data. For example, like the first radar echo, the second radar echo (and other radar echoes) may not be associated with a known object in the environment. The second radar echo may be an echo from another object in the environment (e.g., a newly detected object) and / or an echo reflected from some intermediate object in the environment. In the latter case, the second radar echo may identify the phantom "object" at the location along the second radial direction.

[0092] At operation 508, process 500 can determine the projection velocity as the projection of a first velocity in the second radial direction. For example, operation 508 can project one echo (e.g., the first echo) in an echo pair onto a direction associated with the other echo in the echo pair (e.g., a unit vector associated with the second echo). As will be appreciated, since a reflected echo cannot be closer than the object echo (e.g., a direct echo), operation 508 can project the velocity associated with the closer echo in the first and second echoes onto the direction associated with the more distant echo. For example, and referring to... Figure 2 For example, the first echo could be object echo 122, and the second echo could be first reflected echo 138. The projection velocity determined by operation 508 could be the radial component of the object echo velocity 160 reflected on line 210, where line 210 bisects and is perpendicular to line 208 connecting object echo 122 and first reflected echo 138. Therefore, the projection velocity could be the velocity generated by radio energy first reflected from the object, and then reflected from an intermediate object positioned along a second direction.

[0093] At operation 510, process 500 may determine whether the projection velocity corresponds to the second velocity. For example, at operation 510, the magnitude of the radial component of the projection velocity (e.g., the component along the second radial direction) may be compared with the magnitude of the second velocity (which is measured by the radar sensor along the second radial direction). For example, this comparison may determine whether the magnitude of the radial component is substantially equal to the magnitude of the second velocity. As used herein, the term "substantially equal to" and similar terms may indicate that two values ​​are equal or within a certain threshold margin of each other. For example, the margin may be an absolute value (e.g., 0.1 m / s, 0.5 m / s, 2 m / s, etc.), a percentage (e.g., 0.5%, 1%, 10%), or some other measure. In some examples, the margin may be based on the fidelity of the radar sensor, the distance to the object, the distance associated with the echo, the velocity of the object, the velocity associated with the echo, and / or other characteristics and / or factors.

[0094] If it is determined at operation 510 that the projection speed corresponds to the second speed, then at operation 512, process 500 can identify the second radar echo as a reflected radar echo (or possibly a reflected radar echo). For example, because the echo closely corresponds to the (theoretical) reflection of the first echo, the vehicle computing device can determine that the echo is (possibly) a reflected echo.

[0095] At operation 514, process 500 may optionally receive additional sensor data, and at operation 516, process 500 may optionally confirm the presence of an intermediate object based on the additional sensor data. For example, operations 514 and 516 can confirm the second radar echo as a reflected echo by confirming the presence of an intermediate object (which would have reflected radio energy) that caused the reflection. As described herein, the additional sensor data can be any type of sensor data from one or more sensor modes located on or otherwise associated with the vehicle. In some examples, the presence of an intermediate object can also be confirmed based on map data (e.g., when the intermediate object is a stationary object, terrain feature, etc.). In other implementations, the additional sensor data received at operation 514 can be subsequently received data, including but not limited to subsequent radar data. By way of non-limiting example, subsequently received data can be used to track the second echo over time. As noted herein, reflected echoes may be transient, and attempts to track the second echo using previous and / or subsequent additional sensor data may be futile.

[0096] At operation 518, process 500 can control the vehicle while excluding radar echoes. For example, because the radar echoes have been determined to be reflected echoes and do not represent actual objects in the environment, vehicle control may not depend on the radar echoes. As described herein, a conventional planning system may already control the vehicle (e.g., through braking, steering, etc.) to react to phantom “objects” represented by radar echoes. However, by identifying and excluding reflected echoes, the techniques described herein can provide improved control, for example, responding only to actual objects in the environment.

[0097] Conversely, if it is determined at operation 512 that the projection velocity does not correspond to the second velocity, then at operation 520, process 500 can identify the radar echo as a potential additional object in the environment. For example, when the magnitude of the second velocity does not correspond to the radial component of the projection velocity, the second radar echo may not be a reflected echo. Instead, the second echo may correspond to an actual object in the environment. For example, the second echo could originate from a newly detected object, such as... Figure 2 The example shows pedestrians emerging from a building.

[0098] At operation 522, process 500 may optionally confirm the presence of an additional object. For example, operation 522 may include receiving additional sensor information and determining, at least in part, based on that additional sensor information, that an object exists at a location associated with the second radar echo. The additional sensor information may include one or more of lidar data, image data, additional radar data, time-of-flight data, etc. In some examples, operation 522 may be substantially the same as operation 516, but will confirm the presence of an object at a second location rather than at some intermediate point.

[0099] At operation 524, process 500 may include controlling the vehicle at least in part based on the second radar echo. For example, because the second radar echo can be associated with an actual dynamic object, the techniques described herein can control the vehicle relative to that object. In some instances, the vehicle's trajectory may be determined at least in part based on the second radar echo. By way of non-limiting example, a prediction system (e.g., prediction component 324) may determine a predicted trajectory of an additional object, and a planning system (e.g., planning component 326) may generate a trajectory of the vehicle relative to the predicted trajectory of the additional object. In some implementations, operation 524 may also, or alternatively, include tracking the additional object, e.g., such that subsequent echoes from the additional object can be considered object echoes. Such object echoes can be used to determine reflected echoes caused by the additional object.

[0100] Figure 6This is a flowchart of another example process 600 for determining whether a radar echo is a reflected echo according to embodiments of the present disclosure. Process 600 may be used in place of or in addition to process 500 discussed above. Although process 600 is not limited to... Figure 3 The environment shown is different, but aspects of process 600 can correspond to the above reference. Figure 3 The techniques discussed. Furthermore, although discussed in the context of radar data, example procedure 600 can be used and / or combined with lidar data, sonar data, time-of-flight image data, and / or other types of data.

[0101] At operation 602, process 600 may include receiving radar data for the environment. For example, the radar data may include radar echoes that include location, velocity, and / or intensity information. In some examples, the radar system may include one or more radar sensors and may be a sensor system of an autonomous vehicle (e.g., vehicles 102, 104 described above). The radar data may include multiple echoes, including static echoes corresponding to objects with location and zero velocity, and dynamic echoes or radar trajectories corresponding to moving objects with location and non-zero velocity.

[0102] At operation 604, process 600 may include a first location identifying a first echo of radar data. The first radar echo may identify the depth or range of the echo, the position (e.g., angular position) of the echo relative to the sensor, and / or velocity. In some instances, the first radar echo may be an object echo, for example, corresponding to a known object in the environment. In some examples, the correspondence between the first radar echo and an object echo may be determined based on tracking information generated prior to receiving the radar data. However, in other instances, the techniques described herein can operate without any prior knowledge of the echo. In other words, the first echo (and additional echoes including the second echo discussed below) may not be associated with an object in the environment, and in some instances such an association may be unnecessary. The location of the first radar echo may be a position along a first radial direction, xy coordinates in a coordinate system, or some other location information.

[0103] At operation 606, process 600 may include a second location identifying a second echo of radar data. The second radar echo may identify the depth or range of the echo, the position (e.g., angular position) of the echo relative to the sensor, and / or velocity. In other implementations, the location of the second radar echo may be xy coordinates in a coordinate system or some other location information. The second radar echo may be an echo from another object in the environment (e.g., a newly detected object) and / or an echo reflected from some intermediate object in the environment. In the latter case, the radar echo may identify a phantom "object" at a location along the radial direction. The techniques described herein can be used to determine whether an echo is a reflected echo.

[0104] At operation 608, process 600 can determine the location of the reflection point based on the first and second positions. For example, the techniques described herein can determine the reflection point as a point in space along a second radial direction, at which radio energy reflected from an object associated with the first echo is reflected back to the sensor to generate the second echo. In some examples, operation 610 can use geometry associated with the positions of the first and second echoes to determine the reflection point. Figure 3 An example technique is illustrated, in which the reflection point can be determined as the intersection of a first line and a second line, the first line extending between the sensor and the second echo, and the second line perpendicular to and bisects the line extending between the first and second echoes. Alternatively or additionally, the assumption that the line segments from the reflection point to the two points are equal can be used to determine the position of the reflection point along the lines. As will be appreciated, for any given pair of points, the reflection point must lie on the line between the sensor and the echo that is radially farther from the sensor; therefore, in Figure 6 In the example, the second echo will be farther than the first echo.

[0105] At operation 610, process 600 may receive data about objects in the environment. For example, operation 610 may include receiving additional sensor data about the environment. This additional sensor data may be any type of sensor data from one or more sensor modes located on or otherwise associated with the vehicle. In some examples, operation 610 may include receiving map data about the environment. In the examples described herein, the data about objects may come from any source capable of providing information about whether objects in the environment are static or dynamic.

[0106] At operation 612, process 600 determines whether an object is located at the reflection point. As described above, the geometry of the echo allows for the determination of a theoretical reflection point, i.e., a point at which radio energy initially reflected from an object associated with the first echo will subsequently be reflected to give an echo at the location of the second echo. This reflection is a ghost reflection (or phantom), as opposed to (direct) reflection from an object in the environment. Therefore, while such a theoretical point can be determined for any pair of echoes, the data received at 610 regarding the object can be used to determine whether an object actually exists at the theoretical reflection point.

[0107] At operation 612, if it is determined that an object exists at the reflection point, then at operation 614, process 600 can identify the second radar echo as a reflected radar echo. For example, because the position of the second echo closely corresponds to the (theoretical) reflection position of the first echo at the reflection point, and an object exists at the theoretical reflection point, the vehicle computing device can determine that the echo is a reflected echo. For example, the vehicle computing device can mark the second echo as a potential reflected echo and / or send information to other components of the vehicle.

[0108] At operation 616, process 600 can control the vehicle while excluding the second radar echo or otherwise control the vehicle. For example, because the second radar echo has been identified as a reflected echo and does not represent an actual object in the environment, vehicle control may not depend on the second radar echo. As described herein, a conventional planning system may already control the vehicle (e.g., by braking, steering, etc.) to react to a phantom “object” represented by the radar echo. However, by identifying and excluding reflected echoes, the techniques described herein can provide improved control, for example, responding only to actual objects in the environment. In some implementations, the techniques described herein can also track the second echo to confirm that it is a reflection point. As noted above, environmental geometry will create conditions that allow reflected echoes, but this geometry is constantly changing. Therefore, while a reflected echo may exist in one radar scan, it is unlikely to exist in subsequent (or previous) scans, and thus attempting to track a fleeting echo may be impossible.

[0109] Conversely, if it is determined at operation 612 that no object exists at the location of the reflection point, then at operation 518, process 600 can identify the second radar echo as a potential additional object in the environment. For example, when no object exists at the reflection point, the radar echo is likely not a reflected echo. Instead, the second echo can correspond to an actual object in the environment. For example, the echo could originate from a newly detected object, such as in... Figure 3 The example shows pedestrians emerging from a building.

[0110] At operation 620, process 600 may optionally confirm the presence of an additional object. For example, operation 624 may include receiving additional sensor information and determining, at least in part, that an object is present at a location associated with the second radar echo based on that additional sensor information. The additional sensor information may include one or more of lidar data, image data, additional radar data, time-of-flight data, etc. In some implementations, operation 620 may be substantially the same as operations 610, 612, but instead of determining that an object is present at the reflection point, process 600 may determine that an object is present at a second location (i.e., the second echo).

[0111] At operation 622, process 600 may include controlling the vehicle at least in part based on the second radar echo. For example, because the second radar echo can be associated with an actual dynamic object, the techniques described herein can control the vehicle relative to that object. In some instances, the vehicle's trajectory may be determined at least in part based on the second radar echo. By way of non-limiting example, a prediction system (e.g., prediction component 424) may determine a predicted trajectory of an additional object, and a planning system (e.g., planning component 426) may generate a trajectory of the vehicle relative to the predicted trajectory of the additional object. In some implementations, operation 622 may also, or alternatively, include tracking the additional object, e.g., such that subsequent echoes from the additional object can be considered object echoes. Such object echoes can be used to determine reflected echoes caused by the additional object.

[0112] The operations of processes 500 and 600 can be executed serially and / or in parallel. By way of non-limiting example, the radar data received at operations 502, 504, and 602 may include multiple radar echoes. In some implementations, operations 506, 508, 510, 512, 608, 610, 612, etc., can be performed, for example, in parallel for each of the radar echoes. Therefore, all reflected echoes from the radar data can be identified relatively quickly and removed from consideration, as described herein. Furthermore, although in Figure 5 and Figure 6Only a single echo pair is mentioned here, but multiple echo pairs can also be compared using the techniques described herein. By way of non-limiting examples, the same echo can be compared with multiple other echoes (which may or may not be known to be associated with objects in the environment). In some examples, multiple radar echoes can be received for each known object, and these echoes can be compared (e.g., using process 400 or process 500) with other echoes not associated with objects. Furthermore, echo pairs can be processed according to both process 400 and process 500 to, for example, obtain further confirmation that the echo is a reflection. In other implementations, multiple radar echoes can be filtered, for example, such that only a subset of the multiple radar echoes is processed. For example, a radar echo that is relatively closer to vehicle 102 than a known object cannot be a reflected echo and therefore can be excluded from investigation according to aspects of process 500. Furthermore, radar echoes with speeds below a minimum (e.g., non-zero speeds) may also be excluded from investigation, for example, because these echoes (even reflected echoes) are likely to have minimal impact on the vehicle. Furthermore, the angle of the echo (relative to the (multiple) radar sensors 104 / vehicle 102) is equal to or greater than a threshold angle (e.g., 45 degrees, 60 degrees, 90 degrees, etc.). Other filtering technologies and standards may also be used.

[0113] Figure 5 and Figure 6 An example process according to an embodiment of this disclosure is shown. Figure 5 and Figure 6 The process can, but is not required, be executed as multiple sub-processes, for example, by different components of vehicles 102 and 402. Processes 500 and 600 are shown as logic flowcharts, where each operation represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, an operation represents a computer-executable instruction stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, causes the computer or autonomous vehicle to perform the invoked operation. Typically, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a specific function or implement a specific abstract data type. The order in which operations are described should not be construed as limiting, and any number of described operations can be combined in any order and / or in parallel to implement these processes.

[0114] Example Terms

[0115] A: An example autonomous vehicle includes: a radar sensor on the autonomous vehicle; one or more processors; and a memory storing processor-executable instructions that, when executed by the one or more processors, cause the autonomous vehicle to perform actions including: receiving radar data of the environment from the radar sensor, the radar data including: a first radar echo including: a first velocity and a first range along a first radial direction from the radar sensor; a second radar echo including: a second velocity and a second range along a second radial direction from the radar sensor; projecting the first velocity onto a point corresponding to the second radar echo as a projection velocity; determining, at least in part, that the second radar echo corresponds to a reflected radar echo reflected from an intermediate surface based on the second velocity and the projection velocity; and controlling the autonomous vehicle within the environment, excluding the second radar echo.

[0116] B: An autonomous vehicle of Example A, wherein projecting a first velocity includes: determining a reflection ray based at least in part on a first range in a first radial direction and a second range in a second radial direction; regarding the reflection ray reflecting the first velocity from a first position to a second position; and determining the projected velocity as a component of the reflection velocity along the second radial direction.

[0117] C: An autonomous vehicle of Example A or Example B, wherein determining that the second radar echo corresponds to the reflected radar echo includes: comparing a first magnitude of the projected speed with a second magnitude of the second speed; and determining that the first magnitude is substantially equal to the second magnitude.

[0118] D: An autonomous vehicle of any of Examples A through C, further comprising: receiving a previous radar echo associated with an object before receiving radar data; and identifying a first radar echo as an object echo associated with the object, at least in part based on the previous radar echo.

[0119] E: An autonomous vehicle of any of Examples A through D, further comprising: at least one additional sensor on the vehicle, the action further comprising: receiving additional sensor data from the at least one additional sensor; and verifying the existence of an intermediate object based at least in part on the additional sensor data.

[0120] F: An autonomous vehicle of any of Examples A through E, the action further includes: selecting a position associated with a first echo or a speed associated with a first echo or a second echo, which are derived from a plurality of candidate radar echoes, based at least in part on one or more of the distances associated with the first echo and the second echo.

[0121] G: An example method comprising: capturing radar data of an environment by radar sensors on a vehicle, the radar data including a plurality of radar echoes; determining a first velocity along a first radial direction extending from the vehicle, based at least on a first radar echo among the plurality of radar echoes; determining a second velocity along a second direction extending from the vehicle, based at least in part on a second radar echo among the plurality of radar echoes; determining a projection velocity of the first velocity along the second direction; and determining, based at least in part on a comparison of the projection velocity and the second velocity, that the second radar echo corresponds to a reflected radar echo reflected from an object and an intermediate object between the object and the radar sensor.

[0122] H: The method of example G further includes: receiving sensor data from at least one of a radar sensor or an additional sensor; and identifying the first echo as associated with an object in the environment based at least in part on the sensor data.

[0123] I: The method of Example G or Example H further includes: tracking objects in the environment based at least in part on sensor data and before capturing the first radar echo and the second radar echo, wherein determining the second echo as a reflected echo is also based at least in part on the tracking.

[0124] J: A method of any one of Examples G to I, wherein the projection velocity includes the velocity projected onto the position corresponding to the second echo along the second direction.

[0125] K: A method of any one of Examples G to J, wherein determining the projection velocity comprises: determining a reflected ray based at least in part on a first range associated with a first echo and a second range associated with a second echo; determining a reflection velocity with respect to the reflected ray at a first velocity; and determining the projection velocity as a component of the reflection velocity along a second radial direction.

[0126] L: The method of any one of Examples G to K further includes: determining the reflection point as the intersection of a reflection line and a line extending between the radar sensor and the second location, the reflection point being the location associated with the intermediate object.

[0127] M: The method according to any one of Examples G to L further includes: receiving at least one of additional sensor data or map data; and identifying an intermediate object at the reflection point based at least in part on the additional sensor data or map data.

[0128] N: A method of any one of Examples G to M, wherein the first echo and the second echo are selected based at least in part on at least one of the following: the distance associated with the first echo and the second echo, the position associated with the first echo and the second echo, or the speed associated with the first echo and the second echo.

[0129] O: A method of any one of Examples G to N, wherein a first echo is associated with an object in the environment and a second echo is determined at least in part based on at least one of a second radar echo having a second distance or a second velocity, the second distance being greater than a first distance associated with the first radar echo, and the second velocity being equal to or greater than a threshold velocity.

[0130] P: One or more example non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations including: capturing radar data of the environment by radar sensors, the radar data including: a first radar echo including a first velocity, a first direction, and a first range; and a second radar echo including a second velocity, a second direction, and a second range; determining a projection velocity of the first velocity relative to the second direction; and determining, at least in part, based on a comparison of the projection velocity and the second velocity, that the second radar echo corresponds to a reflected radar echo.

[0131] Q: One or more non-transitory computer-readable media of example P, wherein determining the projection velocity comprises: determining the reflected ray based at least in part on a first echo and a second echo; determining the reflection velocity with respect to the reflected ray reflecting a first velocity from a first position to a second position; and determining the projection velocity as a component of the reflection velocity along a second direction.

[0132] R: The method of example P or example Q further includes: determining the reflection point as a point along a second direction such that a line from the reflection point bisects the line connecting the first point to the second point; receiving additional sensor data; determining the presence of a surface at the reflection point based at least in part on the additional sensor data; and verifying that the second radar echo corresponds to the reflected radar echo.

[0133] S: One or more non-transitory computer-readable media of any one of Examples P to R, the operation further comprising: receiving sensor data from at least one of a radar sensor or an additional sensor; and determining, at least in part, that a first reflected echo is associated with an object in the environment based on the sensor data.

[0134] T: One or more non-transitory computer-readable media of any one of Examples P to S, the operation further comprising: identifying one or more candidate reflected echoes from a plurality of reflected echoes, the one or more candidate reflected echoes including a second radar echo, and the identification being based at least in part on at least one of a range associated with a single reflected echo among the one or more candidate reflected echoes or a velocity associated with a single reflected echo among the one or more candidate reflected echoes.

[0135] U: Example autonomous vehicle, comprising: one or more sensors on the autonomous vehicle, including at least one radar sensor; one or more processors; and a memory storing processor-executable instructions that, when executed by the one or more processors, cause the autonomous vehicle to perform actions including: receiving radar data of the environment from the radar sensor, the radar data including: a first radar echo having an associated first location, comprising a first range along a first radial direction from the radar sensor; a second radar echo having an associated second location, comprising a second range along a second radial direction from the radar sensor; determining a reflection point along the second radial direction based at least in part on the first location and the second location; receiving additional sensor data from the one or more sensors; identifying an object in the environment based at least in part on the additional sensor data; determining that the second radar echo is a reflected echo based at least in part on the object being positioned at the reflection point; and controlling the autonomous vehicle within the environment, excluding the second radar echo.

[0136] V: An autonomous vehicle of example U, wherein determining the reflection point includes: determining a reflection ray based at least in part on a first position and a second position; and determining the reflection point as the intersection of the reflection ray and a line extending along a second direction.

[0137] W: An autonomous vehicle of example U or example V, the action further includes: receiving prior sensor data associated with a second object from a radar sensor and prior to receiving radar data; and identifying a first radar echo as an object echo associated with the second object based at least in part on the prior sensor data.

[0138] X: An autonomous vehicle of any of Examples U to W, the action further includes: selecting a second echo from a plurality of echoes based at least in part on the fact that the second range is greater than the first range.

[0139] Y: An autonomous vehicle of any of Examples U to X, the action further includes: receiving an additional radar echo associated with the environment from a radar sensor and after receiving radar data; and verifying, at least in part, that a second radar echo is a reflected echo based on the additional radar echo.

[0140] Z: An example method includes: receiving radar data of the environment from a radar sensor on a vehicle, the radar data including a plurality of radar echoes; determining a first position in a first radial direction extending from the radar sensor, based at least on a first radar echo of the plurality of radar echoes; determining a second position in a second radial direction extending from the radar sensor, based at least in part on a second radar echo of the plurality of radar echoes; determining a reflection point along the second radial direction and between the radar sensor and the second position; and determining, at least in part, that the second radar echo is a reflected echo based on additional data about the environment.

[0141] AA: The method of Example Z also includes: controlling vehicles in the environment while excluding second radar echoes.

[0142] BB: The method of example Z or example AA, wherein determining the reflection point includes: determining the reflection ray based at least in part on a first position and a second position.

[0143] CC: A method of any one of Examples Z to BB, wherein the additional data includes at least one of sensor data or map data, and the sensor data includes one or more of LiDAR data, additional radar data, or image data.

[0144] DD: The method of any one of Examples Z to CC further includes: receiving an additional radar echo associated with the environment from a radar sensor; and verifying, at least based on the additional radar echo, that the second radar echo is a reflected echo.

[0145] EE: The method of any one of Examples Z to DD further includes: receiving from a radar sensor and prior to receiving radar data a previous radar echo associated with a tracked object in the environment; and identifying a first radar echo as an object echo associated with the tracked object, based at least in part on the previous radar echo.

[0146] FF: A method of any one of Examples Z to Example EE, wherein the first echo and the second echo are selected based at least in part on at least one of the following: the range associated with the first echo and the second echo, the first position and the second position, or the velocity associated with the first echo and the second echo.

[0147] GG: A method of any one of Examples Z to FF, wherein a first echo is associated with an object in the environment and a second echo is determined at least in part based on at least one of a second radar echo having a second range or a second velocity of the second radar echo, the second range being greater than a first range associated with the first radar echo, the second velocity being equal to or greater than a threshold velocity.

[0148] HH: The method of any one of Examples Z to GG further includes: receiving additional sensor data from additional sensors on the vehicle; and identifying an object associated with the first echo based at least in part on the additional sensor data.

[0149] II: One or more example non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations including: receiving radar data of the environment from a radar sensor on a vehicle, the radar data including a plurality of radar echoes; determining a first position in a first radial direction extending from the radar sensor, based at least on a first radar echo of the plurality of radar echoes; determining a second position in a second radial direction extending from the radar sensor, based at least in part on a second radar echo of the plurality of radar echoes; determining a reflection point along the second radial direction and between the radar sensor and the second position; determining, at least in part on additional data about the environment, the presence of an object at the position corresponding to the reflection point; and determining, at least in part on the presence of the object at that position, that the second radar echo is a reflected echo.

[0150] JJ: One or more non-transitory computer-readable media of Example II, wherein determining the reflection point includes: determining the reflection ray based at least in part on a first position and a second position.

[0151] KK: One or more non-transitory computer-readable media of Example II or Example JJ, wherein the additional data includes at least one of sensor data or map data, and the sensor data includes one or more of lidar data or image data.

[0152] LL: One or more non-transitory computer-readable media of any one of Examples II to KK, the operation further comprising: receiving an additional radar echo associated with the environment from a radar sensor; and verifying, at least in part, that a second radar echo is a reflected echo based on the additional radar echo.

[0153] MM: One or more non-transitory computer-readable media of any one of Examples II to LL, the operation further comprising: receiving from a radar sensor and prior to receiving radar data a previous radar echo associated with a tracked object in the environment; and identifying a first radar echo as an object echo associated with the tracked object, at least in part based on the previous radar echo.

[0154] NN: One or more non-transitory computer-readable media of any one of Examples II to MM, wherein the first echo and the second echo are selected based at least in part on at least one of the following: the range associated with the first echo and the second echo, the first position and the second position, or the velocity associated with the first echo and the second echo.

[0155] Although the above example clauses have been described with respect to a particular implementation, it should be understood that, in the context of this document, the content of the example clauses may also be implemented via methods, devices, systems, computer-readable media, and / or other implementations.

[0156] Although the above example clauses have been described with respect to a particular implementation, it should be understood that, in the context of this document, the content of the example clauses may also be implemented via methods, devices, systems, computer-readable media, and / or other implementations.

[0157] in conclusion

[0158] Although one or more examples of the techniques described herein have been described, various modifications, additions, substitutions, and equivalents thereof are also included within the scope of the techniques described herein.

[0159] In the description of the examples, reference is made to the accompanying drawings, which form part of the description, illustrating specific examples of the claimed subject matter by way of illustration. It should be understood that other examples may be used, and changes or alterations such as structural modifications may be made. Such examples, changes, or alterations do not necessarily deviate from the scope of the claimed subject matter. Although the steps herein may be presented in a certain order, in some cases the order may be changed so that certain inputs are provided at different times or in a different order without altering the function of the described system and method. The disclosed processes may also be performed in a different order. Furthermore, it is not necessary to perform the various calculations described herein in the disclosed order, and other examples using alternative orders of calculations can be readily implemented. In addition to being reordered, in some instances these calculations may also be decomposed into sub-computations with the same results.

Claims

1. An autonomous vehicle comprising: a radar sensor on the autonomous vehicle; one or more processors; and memory storing processor-executable instructions that, when executed by the one or more processors, cause the autonomous vehicle to perform acts comprising: receiving, from the radar sensor, radar data of an environment, the radar data comprising: a first radar return comprising a first velocity and a first range along a first radial direction from the radar sensor; a second radar return comprising a second velocity and a second range along a second radial direction from the radar sensor; identifying, based at least in part on the radar data, the first radar return as an object return related to an object; projecting the first velocity onto a point corresponding to the second radar return as a projected velocity; determining, based at least in part on a pairwise comparison of the second velocity and the projected velocity, that the second radar return corresponds to a reflected radar return reflected from an intermediate surface; and controlling the autonomous vehicle within the environment excluding the second radar return. projecting the first velocity comprises:

2. The autonomous vehicle of claim 1, wherein, determining a reflection line based at least in part on the first range in the first radial direction and the second range in the second radial direction; reflecting the first velocity from a first location to a second location with respect to the reflection line; and determining the projected velocity as a component of the reflected velocity along the second radial direction. determining that the second radar return corresponds to the reflected radar return comprises:

3. The autonomous vehicle of claim 2, wherein, determining a reflection point along the second radial direction based at least in part on the first location and the second location; receiving additional sensor data from one or more additional sensors; identifying an object in the environment based at least in part on the additional sensor data; and determining that the object is disposed at the reflection point. determining the reflection point comprises:

4. The autonomous vehicle of claim 3, wherein, determining a reflection line based at least in part on the first location and the second location; and determining the reflection point as an intersection of the reflection line and a line extending along the second radial direction.

5. The autonomous vehicle of any one of claims 1-4, the acts further comprising: prior to receiving the radar data, receiving a previous radar return associated with an object; and identifying, based at least in part on the previous radar return, the first radar return as an object return associated with the object. at least one additional sensor on the autonomous vehicle, the acts further comprising: receiving additional sensor data from the at least one additional sensor; and 6. The autonomous vehicle of any one of claims 1-4, further comprising: verifying, based at least in part on the additional sensor data, a presence of the intermediate surface.

7. A method of identifying radar reflections, comprising: capturing, by a radar sensor on an autonomous vehicle, radar data of an environment, the radar data comprising a plurality of radar returns; identifying, based at least in part on the radar data, a first radar return of the plurality of radar returns as an object return related to an object, the first radar return comprising a first velocity and a first range along a first radial direction from the radar sensor; ​ ​ projecting a first velocity included in the first radar return onto a point corresponding to a second radar return of the plurality of radar returns as a projected velocity, the second radar return including a second velocity along a second radial direction from the radar sensor and a second range; determine, based at least in part on a pairwise comparison of the second velocity included in the second radar return and the projected velocity, that the second radar return corresponds to a reflected radar return reflected from an intermediate surface; and control the autonomous vehicle within the environment excluding the second radar return.

8. The method of claim 7, wherein, projecting the first velocity includes: determining a reflection line based at least in part on the first range in the first radial direction and the second range in the second radial direction; reflecting the first velocity from a first location to a second location with respect to the reflection line; and determining the projected velocity as a component of a reflected velocity along the second radial direction.

9. The method of claim 8, further comprising: determining a reflection point as an intersection of the reflection line and a line extending between the radar sensor and the second location, the reflection point being a location associated with the intermediate surface.

10. The method of claim 8 or 9, further comprising: determining a reflection point along the second radial direction based at least in part on the first location and the second location; receiving additional sensor data from one or more additional sensors; and identifying an additional object in the environment based at least in part on the additional sensor data, wherein determining the second radar return as a reflected return is based at least in part on the additional object being disposed at the reflection point. determining the reflection point includes:

11. The method of claim 10, wherein, determining a reflection line based at least in part on the first location and the second location; and determining the reflection point as an intersection of the reflection line and a line extending along the second radial direction.

12. The method of claim 8 or 9, further comprising: selecting the first radar return and the second radar return based at least in part on at least one of: the first range and the second range, the first location and the second location, or the first velocity and the second velocity.

13. One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the method of any of claims 7-12. ​

Images