Method and system for filtering vehicle self-reflections in radar

Abstract

Example embodiments relate to self-reflection filtering techniques within radar data. A computing device can use radar data to determine a first radar representation conveying information about surfaces in an environment of a vehicle. The computing device can use a predefined model to generate a second radar representation assigning predicted self-reflection values to respective locations of the environment based on the information about the surfaces conveyed by the first radar representation. The predefined model enables a predefined self-reflection value to be assigned to a first location based on information about a surface located at a second location and a relationship between the first location and the second location. The computing device can then modify the first radar representation based on the predicted self-reflection values in the second radar representation and provide instructions to a control system of the vehicle based on the modified first radar representation.

Description

Technical Field

[0001] This disclosure relates to a method and system for filtering vehicle self-reflections in radar. Background Technology

[0002] Radio detection and ranging systems (“radar systems”) are used to estimate the distance to environmental features by transmitting radio signals and detecting the returned reflected signals. The distance to radio-reflected features in the environment can be determined based on the time delay between transmission and reception. Radar systems can transmit signals whose frequencies vary over time, such as signals with a time-varying frequency ramp, and then correlate the frequency difference between the transmitted and reflected signals with the range estimate. Some radar systems can also estimate the relative motion of the reflecting object based on the Doppler shift in the received reflected signal.

[0003] Directional antennas can be used to transmit and / or receive signals to correlate each range estimate with a bearing angle. More generally, directional antennas can also be used to focus radiated energy onto a given field of interest. Combining measured range and orientation information allows for mapping of surrounding environmental features. Summary of the Invention

[0004] The example implementation describes a technique for filtering vehicle autoreflections that can interfere with the accuracy of radar and other types of sensor data. The example filtering technique involves using a predefined model to infer how likely vehicle autoreflections are to appear in radar data captured by the vehicle's radar system, so that these autoreflections can be removed, thereby improving the accuracy of radar measurements.

[0005] In one aspect, an example method is provided. The method includes receiving radar data indicating an environment at a computing device and from a radar unit coupled to a vehicle, and determining a first radar representation based on the radar data that conveys information about multiple surfaces in the environment. The method also includes generating a second radar representation by the computing device and using a predefined model, the second radar representation assigning predicted autoreflection values ​​to corresponding locations in the environment based on the information about the multiple surfaces conveyed by the first radar representation. The predefined model is capable of assigning predefined autoreflection values ​​to a first location based on information about a surface located at a second location and the relationship between the first and second locations. The method further includes modifying the first radar representation based on the predicted autoreflection values ​​in the second radar representation, and providing instructions to the vehicle's control system based on the modified first radar representation.

[0006] On the other hand, an example system is provided. The system includes a radar unit coupled to a vehicle and a computing device. The computing device is configured to receive radar data indicating the environment from the radar unit and is also configured to determine a first radar representation based on the radar data, conveying information about multiple surfaces in the environment. The computing device is further configured to generate a second radar representation using a predefined model, the second radar representation assigning predicted autoreflection values ​​to corresponding locations in the environment based on the information about the multiple surfaces conveyed by the first radar representation. The predefined model is capable of assigning predefined autoreflection values ​​to a first location based on information about a surface located at a second location and the relationship between the first and second locations. The computing device is also configured to modify the first radar representation based on the predicted autoreflection values ​​in the second radar representation and is further configured to provide instructions to the vehicle's control system based on the modified first radar representation.

[0007] In yet another example, an exemplary non-transitory computer-readable medium is provided, storing program instructions executable by a computing system to cause the computing system to perform functions. The functions may involve receiving radar data indicating the environment from a radar unit coupled to a vehicle, and determining a first radar representation based on the radar data that conveys information about multiple surfaces in the environment. The functions also involve generating a second radar representation using a predefined model, the second radar representation assigning predicted autoreflectance values ​​to corresponding locations in the environment based on the information about the multiple surfaces conveyed by the first radar representation. The predefined model is capable of assigning predefined autoreflectance values ​​to a first location based on information about a surface located at a second location and the relationship between the first and second locations. The functions also involve modifying the first radar representation based on the predicted autoreflectance values ​​in the second radar representation, and providing instructions to the vehicle's control system based on the modified first radar representation.

[0008] The foregoing overview is illustrative only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0009] Figure 1 This is a functional block diagram illustrating a vehicle according to one or more example embodiments.

[0010] Figure 2A The illustration shows a side view of a vehicle according to one or more example embodiments.

[0011] Figure 2B The illustration shows a top view of a vehicle according to one or more example embodiments.

[0012] Figure 2C The illustration shows a front view of a vehicle according to one or more example embodiments.

[0013] Figure 2D The illustration shows a rear view of a vehicle according to one or more example embodiments.

[0014] Figure 2E Additional views of a vehicle according to one or more example embodiments are illustrated.

[0015] Figure 3 It is a simplified block diagram of a computing system according to one or more example embodiments.

[0016] Figure 4 It is a system for filtering vehicle self-reflections in sensor data according to one or more example embodiments.

[0017] Figure 5A A top view depicts a vehicle performing a self-reflective filtering technique according to one or more example embodiments.

[0018] Figure 5B Another view depicts a vehicle performing a self-reflective filtering technique according to one or more example embodiments.

[0019] Figure 6 This is a flowchart of a method for performing vehicle self-reflection technology according to one or more example embodiments.

[0020] Figure 7A A three-dimensional (3D) radar data cube is depicted according to one or more example embodiments.

[0021] Figure 7B Two-dimensional (2D) radar images are depicted according to one or more example embodiments.

[0022] Figure 7C Sparse radar data according to one or more example embodiments is depicted.

[0023] Figure 8 This is a flowchart of another method for performing vehicle self-reflection technology according to one or more example embodiments.

[0024] Figure 9 It is a schematic diagram of a computer program according to one or more example embodiments. Detailed Implementation

[0025] In the following detailed description, reference is made to the accompanying drawings, which form a part of the description. In the drawings, similar symbols generally identify similar components unless the context otherwise requires. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein. It will be readily understood that, as generally described herein and illustrated in the drawings, aspects of this disclosure can be arranged, replaced, combined, separated, and designed in a variety of different configurations, all of which are explicitly contemplated herein.

[0026] Radar systems are used to detect objects and estimate their locations by transmitting electromagnetic signals (i.e., radar signals) and analyzing backscattered signals from objects and other surfaces in the environment. These systems can estimate the range of objects by transmitting short pulses and / or coded waveforms, such as pulse Doppler radar, which uses coherent bursts of short pulses at a specific carrier frequency. In some applications, electromagnetic energy is focused in the form of a beam onto a specific spatial sector via a parabolic reflector or an array of antenna elements associated with the radar unit.

[0027] Radar processing systems (e.g., computing devices) can process radar data to generate two-dimensional (2D) and / or three-dimensional (3D) measurements that represent aspects of the environment, such as the position, orientation, and motion of nearby objects and other surfaces occupying the vicinity of the radar system. Because radar systems can be used to measure the distance and motion of nearby objects and other surfaces, vehicle radar systems are increasingly being adopted in vehicles. These systems acquire and provide useful information about vehicle navigation, obstacle avoidance, and other vehicle operations, thereby improving overall vehicle safety. Radar can be used to detect and identify the position, orientation, and motion of nearby vehicles, bicycles, pedestrians, and animals. Radar can also reveal information about other features in the environment surrounding the vehicle, such as the location, arrangement, and position of road boundaries, road conditions (e.g., smooth or bumpy surfaces), weather conditions (e.g., wet or snowy roads), and the relative positions of traffic signs and signals. Therefore, radar provides vehicle systems with a way to continuously monitor and understand changes during navigation in a variety of environments and can supplement sensor data from other types of sensors.

[0028] In some applications, vehicle radar systems can provide information designed to assist the vehicle driver. For example, radar measurements can be used to generate alerts when a vehicle deviates from its lane, or when it gets too close to another vehicle or object, and / or in other ways that help the driver maintain safe control of the vehicle. Radar measurements can also be used to help the vehicle achieve autonomous or semi-autonomous operation. In particular, as mentioned above, control systems can use radar data to understand and safely navigate the vehicle's environment in near real-time.

[0029] In some cases, the reflective properties of various objects encountered by a vehicle during navigation can affect the accuracy of sensor data. Furthermore, some objects can possess specular reflective properties that can distort sensor data. For example, metallic signs and other objects in the vehicle environment (e.g., other vehicles) can cause self-reflections of the vehicle to appear in the sensor data, which can trigger unintended reactions from the control system (e.g., braking or turning). While the detection of ghost vehicles can affect images from cameras and LiDAR data, radar may be negatively impacted more frequently during vehicle navigation due to the specular reflective properties of a large number of objects at radar wavelengths. That is, metallic traffic signs and other objects can act as mirrors and produce strong reflections, which can cause the radar system to observe one or more self-reflections of the vehicle itself approaching the vehicle from behind the object at approximately twice the vehicle's current speed. Although the ghost vehicle detected in the radar data is merely a reflection of the vehicle itself, the vehicle's control system may fail to distinguish the ghost vehicle from other physical objects and respond by performing unintended actions (e.g., braking or turning) to avoid the ghost vehicle. Therefore, the ability to detect and avoid vehicle self-reflections is needed to increase the reliability of sensor data.

[0030] The example embodiments presented herein describe vehicle autoreflection filtering techniques that enable the detection and removal of vehicle autoreflections within radar. In some examples, the vehicle autoreflection filtering techniques described herein can be performed when processing sensor data from another type of sensor, such as a LiDAR camera (e.g., a time-of-flight camera). In some examples, vehicle autoreflection filtering techniques can involve using a model that is pre-calibrated and can be used to represent what the vehicle itself looks like in radar data (or other type of sensor data). By representing how likely ghost vehicle data is to appear in incoming radar data, the processing system can detect and filter radar data to remove (or otherwise detect and ignore) ghost vehicle detections, which can improve the accuracy of radar measurements. Furthermore, the performance of the filtering techniques can help prevent the control system from engaging in unnecessary actions during navigation, such as turning or braking, to avoid detected ghost vehicles.

[0031] To further illustrate, for a measurement surface detected at a range R and azimuth angle θ, the computing device can use a predefined model to infer what a vehicle's self-reflection (i.e., a ghost vehicle) would look like outside the measurement surface. Specifically, the computing device can use the model to infer how likely a ghost vehicle would be to appear at a location approximately twice the size of the measurement surface and aligned with the vehicle at the same azimuth angle (i.e., a ghost vehicle at a distance of 2R and azimuth angle θ). The inferred ghost vehicle can then be filtered from the measurement data representing the location at a distance of 2R and azimuth angle θ.

[0032] For example, a metallic sign located 6 meters from a vehicle might cause a vehicle self-reflection to appear at a distance of 12 meters. Both the metallic sign and the ghost vehicle representing the self-reflection can be aligned relative to the vehicle along the same azimuth angle. Thus, the model allows a computing device to predict the power associated with the detection of the ghost vehicle based on the radar cross-section (RCS) measured against the metallic sign. The RCS measurement represents the degree of radar detection of the metallic sign. When processing radar data corresponding to a surface positioned along the same azimuth angle and 12 meters from the vehicle (i.e., the ghost vehicle's location), the computing device can subtract the predicted power to prevent the vehicle self-reflection from affecting radar measurements of a surface actually located 12 meters from the vehicle in the environment.

[0033] An extended model can be developed that can be used for autoreflection filtering when processing radar echoes during vehicle navigation. This extended model can simulate the process over multiple locations measured by the vehicle's radar. For example, the model can allow filtering of potential autoreflections of the vehicle based on measurements across the entire field of view associated with the operation of one or more radar units.

[0034] To develop extended models, the computing device can perform the above process on multiple hypothetical surfaces (e.g., all densely sampled locations [R, θ]) and aggregate the expected ghost vehicle effects based on the measured RCS at each location. Therefore, instead of predicting that vehicle self-reflections might appear in radar data at a single location, the filtering technique can consider how vehicle self-reflections might appear at multiple locations measured within the radar echo, since objects or other surfaces in the environment can have different RCS measurements that can produce vehicle self-reflections in different ways. Thus, the computing device can effectively filter the input radar data to account for potential vehicle self-reflections that can appear in radar measurements, which can help prevent vehicle control systems from performing unwanted operations in response to the detection of a ghost vehicle.

[0035] In practice, the system can infer what vehicle self-reflection will look like at different stages of radar processing. For example, the system can perform self-reflection filtering on complex raw radar return data input on a field-programmable gate array (FPGA). The computing device can use the radar data to determine a radar data cube containing voxels representing portions of the environment. The radar data cube can have dimensions representing different information from radar measurements (e.g., range data, azimuth data, and Doppler data). Thus, each voxel within the radar data cube can indicate RCS measurements and other information (e.g., Doppler data) about surfaces located at a given range and azimuth relative to the vehicle. Therefore, it is possible to construct a data cube with multiple voxels to convey information about various surfaces measured via radar.

[0036] To predict potential vehicle self-reflections, the computing device can use the model described above to generate a vehicle self-reflection data cube, which can be used to filter potential power associated with ghost vehicles from the power associated with the surface represented by the radar data cube. Specifically, the vehicle self-reflection data cube can also include voxels that can be aligned with the radar data cube representing radar measurements. However, the vehicle self-reflection data cube can be generated to represent predicted RCS measurements in data indicating how a ghost vehicle will appear in different parts of the environment.

[0037] Each predicted RCS measurement represented in the vehicle reflection data cube conveys the power level associated with potential ghost vehicle detection and can depend on a given RCS measurement associated with a given surface represented by voxels in the radar data cube. The predicted RCS measurement can be estimated and assigned to a given location based on the relationship to the surface in the environment and information determined for that surface. For example, if the radar cross section determined for a surface in the environment is above a threshold power level, the computing device can determine that a marker or another type of reflective surface is likely located at that location. Reflective surfaces can cause a large amount of electromagnetic energy to be reflected back and received by the radar system. Thus, the computing device can use the model to infer the location and manner in which the surface might cause a ghost vehicle to appear in the radar. Specifically, the model can indicate that the surface could cause a ghost vehicle to appear at a location approximately twice the size of the surface and aligned at approximately the same azimuth angle, making the ghost vehicle appear to be traveling towards the vehicle from behind the surface. Furthermore, the model can also indicate that the Doppler of the ghost vehicle is approximately twice the Doppler measured for the surface.

[0038] Using this model, the computing device can generate voxels within a vehicle autoreflection data cube to represent predicted RCS measurements at multiple locations with various RCS measurements represented by a radar data cube. For example, for a surface with a given RCS measurement, a Doppler value of "D", and located 10 meters from the vehicle and at an azimuth angle θ, the computing device can use this model to infer that the vehicle autoreflection is likely to occur at a location 20 meters from the vehicle and at the same azimuth angle θ, with a Doppler value of "2D" (e.g., approximately twice the Doppler of the surface). Furthermore, the estimated RCS associated with a ghost vehicle will depend on the RCS measurement of a surface located 10 meters from the vehicle. The computing device can infer ghost vehicle power estimates at various locations based on similar techniques applied to various surfaces measured via radar and represented by voxels within a radar data cube. These inferences can be used to construct voxels for the vehicle autoreflection data cube.

[0039] To remove potential ghost vehicles from radar data, the computing device can then filter predicted RCS measurements within the vehicle autoreflection data cube from different locations represented in the radar data cube. That is, the predicted power of ghost vehicles can be directly filtered (e.g., subtracted) from the radar data cube, resulting in a modified radar data cube representing radar measurements of the environment without the power associated with vehicle autoreflection. The modified radar data cube can then be used to map the environment without the influence of potential vehicle autoreflection on measurements within the map. Furthermore, the filtering operation can involve subtraction as shown in the example or other filtering techniques.

[0040] In some examples, the system is able to perform similar operations on 2D projected radar images, although subtraction may no longer be optimal in some cases. Instead, the system can render the autoreflection as an image layer, which is then fed into the projected radar image of a deep network (e.g., a neural network) to compute an appropriate function representing "filtering" (i.e., removing reflections). Visually, this reflection image can contain multiple superimposed ghost copies of the vehicle throughout the image.

[0041] The Radar Range Equation (RRE) represents an example predictive model that can be used to infer the signal power of vehicle self-reflections (e.g., ghost vehicles located at double distance (2R), the same azimuth (θ), and double Doppler (2D)) using a prior vehicle model RCS(θ) and surface-measured RCS(2R,θ,2D). The prior vehicle model RCS(θ) can be constructed by surrounding the vehicle with a known RCS reflector (e.g., a radar corner cube often used for calibration) and measured by the vehicle's radar in an otherwise empty scene, or by a second vehicle measuring the first vehicle via a direct path. In some examples, the system can perform a similar process for multiple vehicle self-reflections, which can occur across all positive multiples of 2R. Therefore, this subtractive approach enables vehicle radar systems to detect and identify objects otherwise “masked” by vehicle self-reflections. This differs from other techniques that may involve discarding (i.e., not using) any object detection, where the system sees unwanted vehicle self-reflections. In other examples, other models can be used to develop predictive models that can be used to infer the signal power of vehicle self-reflection.

[0042] The following detailed description can be used in one or more radar elements having one or more antenna arrays. The one or more antenna arrays can employ single-input single-output, single-input multiple-output (SIMO), multiple-input single-output (MISO), multiple-input multiple-output (MIMO), and / or synthetic aperture radar (SAR) radar antenna architectures. In some embodiments, an example radar element architecture may include multiple “dual-aperture waveguide” antennas. The term “DOEWG” may refer to a short segment of a horizontal waveguide channel plus a vertical channel divided into two parts. Each of the two parts of the vertical channel may include an output port configured to radiate at least a portion of the electromagnetic waves entering the radar element. Furthermore, in some cases, multiple two-degree-of-freedom antennas may be arranged in one or more antenna arrays.

[0043] Some example radar systems can be configured to operate at electromagnetic wave frequencies in the W-band (e.g., 77 GHz). The W-band corresponds to electromagnetic waves on the millimeter scale (e.g., 1 mm or 4 mm). Radar systems can use one or more antennas capable of focusing radiated energy into a tight beam for highly accurate environmental measurements. Such antennas can be compact (typically with a rectangular form factor), efficient (i.e., very little of the 77 GHz energy is lost, resulting in heating within the antenna or reflection back into the transmitter electronics), low-cost, and easy to manufacture (i.e., radar systems with these antennas can be mass-produced).

[0044] Additionally or optionally, using different radar elements with different polarizations can prevent interference during radar system operation. For example, the radar system can be configured to interrogate (i.e., transmit and / or receive radar signals) in a direction perpendicular to the autonomous vehicle's direction of travel via SAR functionality. Thus, the radar system can be able to determine information about roadside objects that the vehicle passes. In some examples, this information can be two-dimensional (e.g., the distances of various objects from the roadside). In other examples, this information can be three-dimensional (e.g., a point cloud of various parts of the detected objects). Thus, for example, the vehicle can be able to "map" the sides of the road while driving.

[0045] Furthermore, the configuration of the radar systems in the examples can vary. For instance, some radar systems can consist of radar elements, each configured with one or more antenna arrays. An antenna array can include a group of multiple connected antennas capable of working together as a single antenna to transmit or receive signals. By combining multiple radiating elements (i.e., antennas), an antenna array can enhance the performance of a radar element compared to a radar element using a non-array antenna. In particular, higher gain and a narrower beam can be achieved when a radar element is equipped with one or more antenna arrays. Therefore, a radar element can be designed with an antenna array configured to enable the radar element to measure specific areas of the environment, such as target areas located at different ranges (distances) from the radar element.

[0046] Radar units equipped with antenna arrays can vary in their overall configuration. For example, in this example, the number of arrays, their positions, orientations, and the size of the antenna arrays on the radar unit can be varied. Furthermore, the number, position, alignment, and orientation of the radiating elements (antennas) within the array of the radar unit can also be varied. Therefore, the configuration of a radar unit typically depends on the desired performance of the radar unit. For example, the configuration of a radar unit designed to measure distances far from the radar unit (e.g., the far range of the radar unit) can differ from the configuration of a radar unit used to measure areas near the radar unit (e.g., the near field of the radar unit).

[0047] To further illustrate, in some examples, a radar unit may include the same number of transmit antenna arrays and receive antenna arrays (e.g., four transmit antenna arrays and four receive antenna arrays). In other examples, a radar unit may include a different number of transmit antenna arrays than the number of receive antenna arrays (e.g., six transmit antenna arrays and three receive antenna arrays). Furthermore, some radar units may operate in conjunction with parasitic arrays capable of controlling radar transmission. Other example radar units may include one or more drive arrays having radiating elements connected to an energy source, which, compared to parasitic arrays, are capable of having a smaller total energy loss.

[0048] Antennas on a radar element can be arranged in one or more linear antenna arrays (i.e., antennas in the array are arranged in a straight line). For example, a radar element may include multiple linear antenna arrays arranged in a specific configuration (e.g., in parallel lines on the radar element). In other examples, antennas can also be arranged in planar arrays (i.e., antennas are arranged in multiple parallel lines on a single plane). Furthermore, some radar elements can have antennas arranged in multiple planes, thus forming a three-dimensional array.

[0049] Radar units can also comprise multiple types of arrays (e.g., a linear array on one part and a planar array on another). Therefore, radar units configured with one or more antenna arrays can reduce the total number of radar units a radar system might need to measure its surroundings. For example, a vehicle radar system may include radar units with antenna arrays, which can be used to measure specific areas of the environment as needed during vehicle navigation.

[0050] Some radar units can have different functions and operating characteristics. For example, one radar unit can be configured for long-range operation, while another radar unit can be configured for short-range operation. A radar system can use combinations of different radar units to measure different areas of the environment. Therefore, it is desirable to optimize the signal processing of the short-range radar unit for radar reflections in the near field of the radar unit.

[0051] Now refer to the attached diagram, Figure 1 This is a functional block diagram of vehicle 100, which represents a vehicle capable of operating fully or partially in an autonomous mode. More specifically, vehicle 100 can operate in an autonomous mode without human interaction by receiving control commands from a computing system (e.g., a vehicle control system). As part of operating in autonomous mode, vehicle 100 can use sensors (e.g., sensor system 104) to detect and possibly identify objects in the surrounding environment for safe navigation. In some example embodiments, vehicle 100 may also include a subsystem that enables a driver (or remote operator) to control the operation of vehicle 100.

[0052] like Figure 1 As shown, vehicle 100 includes various subsystems, such as a propulsion system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, a data storage device 114, and a user interface 116. The subsystems and components of vehicle 100 can be interconnected in various ways (e.g., wired or secure wireless connections). In other examples, vehicle 100 may include more or fewer subsystems. Furthermore, the functionality of vehicle 100 described herein can be divided into additional functions or physical components, or combined into fewer functions or physical components in implementation.

[0053] The propulsion system 102 may include one or more components operable to provide powered motion to the vehicle 100, and may include an engine / motor 118, an energy source 119, a transmission 120, and wheels / tires 121, as well as other possible components. For example, the engine / motor 118 may be configured to convert the energy source 119 into mechanical energy, and may correspond to one or a combination of an internal combustion engine, one or more electric motors, a steam engine, or a Stirling engine, as well as other possible options. For example, in some embodiments, the propulsion system 102 may include multiple types of engines and / or motors, such as gasoline engines and electric motors.

[0054] Energy source 119 refers to an energy source that can provide power, in whole or in part, to one or more systems of vehicle 100 (e.g., engine / motor 118). For example, energy source 119 can correspond to gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and / or other power sources. In some embodiments, energy source 119 may include a combination of a fuel tank, battery, capacitor, and / or flywheel.

[0055] The transmission 120 can send mechanical power from the engine / motor 118 to the wheels / tires 121 and / or other possible systems of the vehicle 100. Thus, the transmission 120 may include a gearbox, clutch, differential, and drive shaft, as well as other possible components. The drive shaft may include an axle connected to one or more wheels / tires 121.

[0056] The wheels / tires 121 of vehicle 100 can have various configurations in the example embodiment. For example, among other possible configurations, vehicle 100 can exist as a unicycle, bicycle / motorcycle, tricycle, or four-wheeled car / truck. Thus, the wheels / tires 121 can be attached to vehicle 100 in various ways and can be made of different materials, such as metal and rubber.

[0057] Sensor system 104 can include various types of sensors, such as a Global Positioning System (GPS) 122, an Inertial Measurement Unit (IMU) 124, one or more radar units 126, a laser rangefinder / LIDAR unit 128, a camera 130, a steering sensor 123, and a throttle / brake sensor 125, as well as other possible sensors. In some embodiments, sensor system 104 may also include sensors configured to monitor the internal systems of vehicle 100 (e.g., O2 monitor, fuel gauge, engine oil temperature, brake status).

[0058] GPS 122 may include a transceiver operable to provide information about the position of vehicle 100 relative to the Earth. IMU 124 may be configured to use one or more accelerometers and / or gyroscopes and can sense changes in the position and orientation of vehicle 100 based on inertial acceleration. For example, when vehicle 100 is stationary or moving, inertial measurement unit 124 can detect the pitch and yaw of vehicle 100.

[0059] Radar unit 126 may represent one or more systems configured to use radio signals to sense objects (e.g., radar signals) within the local environment of vehicle 100, including the object's speed and orientation. Thus, radar unit 126 may include one or more radar units equipped with one or more antennas configured to transmit and receive radar signals as described above. In some embodiments, radar unit 126 may correspond to an mountable radar system configured to acquire measurements 100 of the vehicle's surrounding environment. For example, radar unit 126 may include one or more radar units configured to be coupled to the underside of the vehicle body.

[0060] The laser rangefinder / LIDAR 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components, and may operate in a coherent mode (e.g., using heterodyne detection) or an incoherent detection mode. The camera 130 may include one or more devices (e.g., a still camera or a video camera) configured to capture images of the environment of the vehicle 100.

[0061] Steering sensor 123 can sense the steering angle of vehicle 100, which may involve measuring the angle of the steering wheel or measuring an electrical signal representing the steering wheel angle. In some embodiments, steering sensor 123 can measure the angles of the wheels of vehicle 100, such as detecting the angle of the wheel relative to the forward axis of vehicle 100. Steering sensor 123 can also be configured to measure a combination (or subset) of the steering wheel angle, the electrical signal representing the steering wheel angle, and the wheel angles of vehicle 100.

[0062] The throttle / brake sensor 125 can detect the throttle position or braking position of the vehicle 100. For example, the throttle / brake sensor 125 can measure the angle of both the accelerator pedal (accelerator) and the brake pedal, or it can measure an electrical signal that may represent, for example, the angle of the accelerator pedal (accelerator) and / or the angle of the brake pedal. The throttle / brake sensor 125 can also measure the angle of the throttle body of the vehicle 100, which may include part of a regulating physical mechanism that supplies energy source 119 to the engine / motor 118 (e.g., a butterfly valve or carburetor). Furthermore, the throttle / brake sensor 125 can measure the pressure of one or more brake pads on the rotor of the vehicle 100, or a combination (or subset) of the angle of the accelerator pedal (accelerator) and the brake pedal, an electrical signal representing the angle of the accelerator pedal (accelerator) and the brake pedal, the angle of the throttle body, and the pressure exerted by at least one brake pad on the rotor of the vehicle 100. In other embodiments, the throttle / brake sensor 125 may be configured to measure the pressure applied to a vehicle pedal, such as the accelerator or brake pedal.

[0063] The control system 106 may include components configured to assist navigation of the vehicle 100, such as a steering unit 132, a throttle 134, a braking unit 136, a sensor fusion algorithm 138, a computer vision system 140, a navigation / path planning system 142, and an obstacle avoidance system 144. More specifically, the steering unit 132 is operable to adjust the heading of the vehicle 100, and the throttle 134 controls the operating speed of the engine / motor 118 to control the acceleration of the vehicle 100. The braking unit 136 can decelerate the vehicle 100, which may involve using friction to slow down the wheels / tires 121. In some embodiments, the braking unit 136 can convert the kinetic energy of the wheels / tires 121 into electrical current for subsequent use by one or more systems of the vehicle 100.

[0064] Sensor fusion algorithm 138 may include Kalman filters, Bayesian networks, or other algorithms capable of processing data from sensor system 104. In some implementations, sensor fusion algorithm 138 may provide evaluations based on input sensor data, such as evaluations of individual objects and / or features, evaluations of specific situations, and / or evaluations of the potential impact of a given situation.

[0065] Computer vision system 140 may include hardware and software operable to process and analyze images in an effort to determine objects, environmental objects (e.g., parking lights, road boundaries, etc.), and obstacles. Thus, computer vision system 140 may use, for example, object recognition, construct from motion (SFM), video tracking, and other algorithms used in computer vision to identify objects, map the environment, track objects, estimate object velocities, etc.

[0066] The navigation / path planning system 142 can determine the driving path of the vehicle 100, which may involve dynamically adjusting the navigation during operation. Thus, the navigation / path planning system 142 can navigate the vehicle 100 using data from sensor fusion algorithm 138, GPS 122, maps, and other sources. The obstacle avoidance system 144 can assess potential obstacles based on sensor data and enable the vehicle 100's system to avoid or otherwise traverse potential obstacles.

[0067] like Figure 1 As shown, vehicle 100 may also include peripheral devices 108, such as a wireless communication system 146, a touchscreen 148, a microphone 150, and / or a speaker 152. Peripheral devices 108 can provide users with controls or other elements to interact with user interface 116. For example, touchscreen 148 can provide information to users of vehicle 100. User interface 116 can also accept input from users via touchscreen 148. Peripheral devices 108 can also enable vehicle 100 to communicate with devices such as other vehicle equipment.

[0068] The wireless communication system 146 can securely communicate wirelessly with one or more devices, either directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as CMDA, EVDO, GSM / GPRS, or 4G cellular communication, such as WiMAX or LTE. Optionally, the wireless communication system 146 can communicate with a wireless local area network (WLAN) using WiFi or other possible connections. For example, the wireless communication system 146 can also communicate directly with devices using an infrared link, Bluetooth, or ZigBee. In the context of this disclosure, other wireless protocols, such as those used in various vehicle communication systems, are also possible. For example, the wireless communication system 146 may include one or more dedicated short-range communication (DSRC) devices, which can include public and / or private data communication between vehicles and / or roadside stations.

[0069] Vehicle 100 may include a power source 110 for supplying power to components. In some embodiments, power source 110 may include a rechargeable lithium-ion battery or a lead-acid battery. For example, power source 110 may include one or more batteries configured to provide electrical energy. Vehicle 100 may also use other types of power sources. In an example embodiment, power source 110 and energy source 119 may be integrated into a single energy source.

[0070] Vehicle 100 may also include a computer system 112 to perform operations such as those described herein. Thus, computer system 112 may include at least one processor 113 (which may include at least one microprocessor) operable to execute instructions 115 stored in a non-transitory computer-readable medium such as data storage 114. In some embodiments, computer system 112 may represent multiple computing devices that can be used to control individual components or subsystems of vehicle 100 in a distributed manner.

[0071] In some implementations, data storage 114 may contain instructions 115 (e.g., program logic) executable by processor 113 to perform various functions of vehicle 100, including those described above. Figure 1 The functions described. Data storage 114 may also contain additional instructions, including sending data to it, receiving data from it, interacting with and / or controlling one or more of the propulsion system 102, sensor system 104, control system 106 and peripheral devices 108.

[0072] In addition to instruction 115, data storage 114 can store data such as road maps, route information, and other information. This information can be used by vehicle 100 and computer system 112 while vehicle 100 is operating in autonomous, semi-autonomous, and / or manual modes.

[0073] Vehicle 100 may include a user interface 116 for providing information to or receiving input from a user of vehicle 100. The user interface 116 may control or enable the layout of content and / or interactive images that may be displayed on touchscreen 148. Furthermore, the user interface 116 may include one or more input / output devices within a set of peripheral devices 108, such as a wireless communication system 146, touchscreen 148, microphone 150, and speaker 152.

[0074] Computer system 112 can control the functions of vehicle 100 based on input received from various subsystems (e.g., propulsion system 102, sensor system 104, and control system 106) and from user interface 116. For example, computer system 112 can use input from sensor system 104 to estimate the outputs generated by propulsion system 102 and control system 106. Depending on the embodiment, computer system 112 may be operable to monitor many aspects of vehicle 100 and its subsystems. In some embodiments, computer system 112 may disable some or all functions of vehicle 100 based on signals received from sensor system 104.

[0075] The components of vehicle 100 can be configured to work in an interconnected manner with other components, either internally or externally to their respective systems. For example, in an example embodiment, camera 130 can capture multiple images that may represent information about the environmental state of vehicle 100 operating in autonomous mode. The environmental state may include parameters of the road on which the vehicle is traveling. For example, computer vision system 140 may be able to identify slope (incline) or other features based on multiple images of the road. Additionally, a combination of features identified by GPS 122 and computer vision system 140 can be used with map data stored in data storage 114 to determine specific road parameters. Furthermore, radar unit 126 can also provide information about the environment surrounding the vehicle.

[0076] In other words, the combination of various sensors (which may be referred to as input indication and output indication sensors) and computer system 112 can interact to provide indications of inputs for controlling the vehicle or indications of the environment surrounding the vehicle.

[0077] In some embodiments, computer system 112 can make determinations about various objects based on data provided by systems other than radio systems. For example, vehicle 100 may have lasers or other optical sensors configured to sense objects in the vehicle's field of view. Computer system 112 can use the outputs from various sensors to determine information about objects in the vehicle's field of view, and can determine distance and orientation information to various objects. Computer system 112 can also determine whether an object is desired or undesirable based on the outputs from various sensors. Furthermore, vehicle 100 may also include a telematics control unit (TCU) 160. TCU 160 can enable vehicle connectivity and in-vehicle passenger equipment connectivity via one or more wireless technologies.

[0078] Although Figure 1 Various components of vehicle 100 are shown, namely, wireless communication system 146, computer system 112, data storage 114, and user interface 116, which are integrated into vehicle 100. However, one or more of these components may be installed separately from or associated with vehicle 100. For example, data storage 114 may exist partially or entirely separate from vehicle 100. Therefore, vehicle 100 may be provided in the form of device elements, which may be positioned individually or together. The device elements constituting vehicle 100 may be communicatively coupled together in a wired and / or wireless manner.

[0079] Figure 2A , Figure 2B , Figure 2C , Figure 2D and Figure 2EDifferent views of the physical configuration of vehicle 100 are illustrated. Various views are included to depict example sensor locations 202, 204, 206, 208, and 210 on vehicle 100. In other examples, the sensors can be located at different positions on vehicle 100. Although vehicle 100 is... Figures 2A-2E The vehicle is depicted as a van, but vehicle 100 can have other configurations in the example, such as truck, car, semi-trailer truck, motorcycle, bus, shuttle, golf cart, off-road vehicle, robotic equipment or agricultural vehicle, and other possible examples.

[0080] As described above, vehicle 100 may include sensors coupled to various external locations, such as sensor locations 202-210. Vehicle sensors include one or more types of sensors, each configured to capture information from the surrounding environment or perform other operations (e.g., communication links, obtaining overall positioning information). For example, sensor locations 202-210 may serve as locations for any combination of one or more cameras, radar units, LiDAR units, rangefinders, radio devices (e.g., Bluetooth and / or 802.11), acoustic sensors, and other possible types of sensors.

[0081] When coupled in Figures 2A-2E When the example sensor locations 202-210 are shown, various mechanical fasteners can be used, including permanent or non-permanent fasteners. For example, bolts, screws, clips, pins, rivets, anchors, and other types of fasteners can be used. In some examples, the sensor can be coupled to the vehicle using adhesive. In further examples, the sensor can be designed and manufactured as part of a vehicle component (e.g., part of a vehicle rearview mirror).

[0082] In some embodiments, one or more sensors may be positioned at sensor locations 202-210 using a movable mount operable to adjust the orientation of the sensors. The movable mount may include a rotating platform capable of rotating the sensors to acquire information from multiple directions around the vehicle 100. For example, the sensor at sensor location 202 may use a movable bracket capable of rotating and scanning within a specific angular and / or azimuth range. Thus, the vehicle 100 may include a mechanical structure that allows one or more sensors to be mounted on top of the vehicle 100. Furthermore, other mounting locations are possible in this example. In some cases, sensors coupled to these locations may provide data that can be used by a remote operator to assist the vehicle 100.

[0083] Figure 3This is a simplified block diagram illustrating a computing device 300, showing some components that may be included in a computing device arranged to operate according to embodiments herein. The computing device 300 may be a client device (e.g., a device actively operated by a user (e.g., a remote operator), a server device (e.g., a device providing computing services to client devices), or some other type of computing platform.

[0084] In some embodiments, computing device 300 may be implemented as computer system 112, which is located on vehicle 100 and performs processing operations related to vehicle operation. For example, computing device 300 may be used to process sensor data received from sensor system 104, develop control commands, implement wireless communication with other devices, and / or perform other operations. Alternatively, computing device 300 may be located remotely from vehicle 100 and communicate via secure wireless communication. For example, computing device 300 may operate as a remotely positioned device that a remote human operator can use to communicate with one or more vehicles.

[0085] exist Figure 3 In the example embodiment shown, computing device 300 includes a processor 302, a memory 304, an input / output unit 306, and a network interface 308, all of which can be coupled via a system bus 310 or a similar mechanism. In some embodiments, computing device 300 may include other components and / or peripheral devices (e.g., removable storage, sensors, etc.).

[0086] Processor 302 can be one or more of any type of computer processing element, such as a central processing unit (CPU), a coprocessor (e.g., a math, graphics, or cryptographic coprocessor), a digital signal processor (DSP), a network processor, and / or some form of integrated circuit or controller that performs processor operations. In some cases, processor 302 can be one or more single-core processors. In other cases, processor 302 can be one or more multi-core processors with multiple independent processing units. Processor 302 may also include register memory for temporarily storing instructions being executed and associated data, and cache memory for temporarily storing recently used instructions and data.

[0087] Memory 304 can be any form of computer-usable memory, including but not limited to random access memory (RAM), read-only memory (ROM), and non-volatile memory. This can include flash memory, hard disk drives, solid-state drives, rewritable optical discs (CDs), rewritable digital video discs (DVDs), and / or magnetic tape storage, to name just a few. Computing device 300 can include fixed memory and one or more removable memory units, including but not limited to various types of secure digital storage (SD) cards. Therefore, memory 304 can represent both main memory units and long-term storage. Other types of memory may include biological memory.

[0088] Memory 304 may store program instructions and / or data on which the program instructions can operate. For example, memory 304 may store such program instructions on a non-transitory computer-readable medium such that the instructions are executable by processor 302 to perform any of the methods, processes, or operations disclosed in this specification or the accompanying drawings.

[0089] like Figure 3 As shown, memory 304 may include firmware 314A, core 314B, and / or application 314C. Firmware 314A may be some or all of the program code used to boot or otherwise start computing device 300. Core 314B may be an operating system, including modules for memory management, process scheduling and management, input / output, and communication. Core 314B may also include a device driver that allows the operating system to communicate with hardware modules of computing device 300 (e.g., memory cells, network interfaces, ports, and buses). Application 314C may be one or more user-space software programs, such as web browsers or email clients, and any software libraries used by these programs. In some examples, application 314C may include one or more neural network applications and other deep learning-based applications. Memory 304 may also store data used by these and other programs and applications.

[0090] Input / output unit 306 facilitates interaction between users and peripheral devices and computing device 300 and / or other computing systems. Input / output unit 306 may include one or more types of input devices, such as a keyboard, mouse, one or more touchscreens, sensors, biosensors, etc. Similarly, input / output unit 306 may include one or more types of output devices, such as a screen, monitor, printer, speaker, and / or one or more light-emitting diodes (LEDs). Additionally or alternatively, for example, computing device 300 may communicate with other devices using a Universal Serial Bus (USB) or High Definition Multimedia Interface (HDMI) port interface. In some examples, input / output unit 306 can be configured to receive data from other devices. For example, input / output unit 306 may receive sensor data from vehicle sensors.

[0091] like Figure 3 As shown, the input / output unit 306 includes a GUI 312, which can be configured to provide information to a remote operator or another user. The GUI 312 can be one or more display interfaces, or another type of mechanism for conveying information and receiving input. In some examples, the representation of the GUI 312 may vary depending on the vehicle conditions. For example, the computing device 300 may provide the GUI 312 in a specific format, such as a format with a single optional option for selection by a remote operator.

[0092] Network interface 308 can take the form of one or more wired interfaces, such as Ethernet (e.g., Fast Ethernet, Gigabit Ethernet, etc.). Network interface 308 can also support communication over one or more non-Ethernet media, such as coaxial cable or power lines, or over wide area media, such as Synchronous Optical Network (SONET) or Digital Subscriber Line (DSL) technologies. Network interface 308 can additionally take the form of one or more wireless interfaces, such as IEEE 802.11 (Wi-Fi). A Global Positioning System (GPS) or wide-area wireless interface may be used. However, other forms of physical layer interfaces and other types of standard or proprietary communication protocols may be used on network interface 308. Furthermore, network interface 308 may include multiple physical interfaces. For example, some embodiments of computing device 300 may include Ethernet, And a Wi-Fi interface. In some embodiments, network interface 308 enables computing device 300 to connect to one or more vehicles to allow the remote assistance technologies presented herein.

[0093] In some embodiments, one or more instances of computing device 300 may be deployed to support a cluster architecture. The exact physical location, connectivity, and configuration of these computing devices may be unknown and / or insignificant to client devices. Therefore, the computing device may be referred to as a "cloud-based" device, which can be housed in various remote data center locations. Furthermore, computing device 300 can achieve the performance of the embodiments described herein, including efficient processing of sensor data.

[0094] Figure 4 This is a system for filtering vehicle autoreflections from sensor data. In an example embodiment, system 400 includes a computing device 402, a radar system 404, a sensor system 406, and a vehicle 408. In other embodiments, system 400 may include... Figure 4 Other components not shown.

[0095] System 400 is capable of performing the vehicle autoreflection filtering techniques presented herein to enhance the performance of vehicle 408. Specifically, computing device 402 is capable of using one or more filtering techniques to detect and filter sensor data representing autoreflections of vehicle 408 caused by metallic markings, vehicles, and other surfaces in the environment. System 400 is capable of iteratively performing the filtering techniques to remove autoreflections of vehicle 408 from radar data or other types of sensor data used to enable vehicle 408 to navigate autonomously between locations.

[0096] During navigation of vehicle 408, radar system 404 and sensor system 406 can capture sensor data representing environmental measurements. In some cases, one or more objects can cause unintended self-reflections in the radar data (or another type of sensor data) involving vehicle 408. For example, the reflective properties of a metallic sign can cause a ghost vehicle to appear to approach vehicle 408 from behind the sign, which can trigger control system 416 to brake or steer to avoid the ghost vehicle. These actions of control system 416 are undesirable because the ghost vehicle is merely a reflection caused by the metallic sign, and changing the strategy can negatively impact the autonomous performance of vehicle 408. Thus, via one or more vehicle self-reflection techniques presented herein, system 400 can assist the navigation of vehicle 408 by enabling these ghost vehicles to be anticipated, detected, and removed from the sensor data acquired from radar system 404 and sensor system 406.

[0097] Computing device 402 refers to one or more processing units within system 400, capable of performing one or more operations described herein. For example, computing device 402 can be configured to perform, respectively... Figure 6 and Figure 8 Method 600 and / or method 800 are shown. The computing device 402 can be implemented as... Figure 1The computing system 112, Figure 3 The computing device 402 may be a computing device 300 or another type of processing device or group of devices. In some embodiments, the computing device 402 is located on the vehicle 408 and may be combined with one or more FPGAs associated with the radar system 404. By placing the computing device 402 on the vehicle 408, the time required for communication between components within the system 400 can be reduced. In other embodiments, the computing device 402 may be located at a location physically separate from the vehicle 408. Specifically, the computing device 402 may be remotely located and wirelessly communicate with one or more computing systems (e.g., control system 416) located on the vehicle.

[0098] Radar system 404 represents one or more radar elements capable of transmitting radar signals into the environment and receiving radar reflections from surfaces in the environment. The distance to radio reflection features in the environment can be determined based on the time delay between transmission and reception. Radar system 404 is capable of transmitting signals with frequencies that vary over time, such as signals with time-varying frequency ramps, and then associating the frequency difference between the transmitted and reflected signals with a range estimate. In some examples, radar system 404 can also estimate the relative motion of the reflecting object based on the Doppler frequency shift in the received reflected signal. Furthermore, radar system 404 can use directional antennas to transmit and / or receive signals to associate each range estimate with an azimuth angle. More generally, directional antennas can also be used to focus radiated energy onto a given field of interest. Combining the measured range and orientation information allows mapping of surrounding environmental features.

[0099] Sensor system 406 refers to other types of sensors capable of supplementing the navigation operation of vehicle 408. For example, sensor system 406 may include a camera system with one or more types of cameras (e.g., a time-of-flight camera) and / or a LIDAR system capable of measuring the environment surrounding vehicle 408. Furthermore, vehicle 408 can correspond to any type of vehicle capable of using radar system 404 in some way. For example, vehicle 408 can correspond to a passenger vehicle transporting passengers between different locations. Vehicle 408 can similarly be used to transport goods or other items.

[0100] To assist navigation of vehicle 408, computing device 402 can perform filtering techniques to remove ghost vehicle detection (i.e., self-reflection of vehicle 408) from radar data obtained via radar system 404 and / or other sensor data from sensor system 406. For example, computing device 402 enables radar system 404 to send radar signals to the environment surrounding vehicle 408 and obtain radar data representing reflections from objects and other surfaces in the environment while vehicle 408 is in motion. When performing processing techniques, computing device 402 can separate targets from clutter based on Doppler content and amplitude characteristics. After IF amplification and phase-sensitive detection, computing device 402 can perform a conversion of radar signals to digital form. Computing device 402 can communicate with other components within system 400 via wired or wireless communication using communication interface 414.

[0101] The computing device 402 is capable of determining a radar representation that conveys information about surfaces in the environment. In some embodiments, the radar representation can include RCS measurements assigned to represent surfaces located at different locations in the environment. To filter out the self-reflection of vehicle 408 from the radar representation, the computing device 402 can generate a vehicle self-reflection representation that assigns predicted vehicle self-reflection information (e.g., predicted RCS values) to different locations in the environment based on a vehicle self-reflection model 410. Specifically, the vehicle self-reflection model 410 can assign predefined self-reflection values ​​to said locations based on information about surfaces located at other locations and the relationships between these locations. The relationships between locations can depend on positions arranged relative to vehicle 408 along the same azimuth angle and can reflect the expected nature of a ghost vehicle now covering twice the surface area, as if the ghost vehicle were traveling towards vehicle 408 from behind the reflective surface. In this way, information in the generated vehicle self-reflection representation can be filtered from the radar representation, resulting in a modified radar representation that includes measurements corresponding to surfaces in the environment without data representing the self-reflection of vehicle 408.

[0102] After filtering ghost vehicle detection from radar data (or another type of sensor data), computing device 402 can use the remaining data to detect and identify objects, and typically map the environment. Computing device 402 can then provide the environment-mapped information to control system 416, which can use this information to safely navigate vehicle 408 through its surroundings. In some embodiments, computing device 402 can perform filtering techniques in parallel on radar data obtained from different radar units within radar system 404. Similarly, computing device 402 can also perform filtering techniques simultaneously on radar data and sensor data from sensor system 406.

[0103] In some embodiments, computing device 402 may use neural network 412 to perform one or more aspects of the filtering process. For example, computing device 402 may feed radar data and predicted power associated with the autoreflection of vehicle 408 within the radar data to neural network 412 to filter predicted power from the radar data. Neural network 412 can enable radar data to be used for object recognition and localization without unwanted noise from the potential autoreflection of vehicle 408. In some examples, neural network 412 is trained with a dataset containing ground truth labels for “vehicle or non-vehicle” (e.g., bounding boxes drawn around any real vehicle) provided by a human labeler. The labeler may use recorded laser and / or camera data to help determine the ground truth 3D boxes. Thus, the relative position and orientation of the radar with respect to the camera and laser are known, making it possible to determine the location of these 3D boxes in the radar image.

[0104] In some embodiments, computing device 402 can use sensor system 406 to supplement the processing of radar data from radar system 404. For example, sensor system 406 can be used to confirm whether a detected object is located in the environment of vehicle 408 when filtering techniques fail to remove radar data indicating a ghost vehicle.

[0105] Figure 5A and Figure 5B The illustration depicts a vehicle performing an autoreflective filtering technique according to one or more example embodiments. In the example embodiment, scenario 500 represents a simplified illustration depicting a vehicle 502 using sensor data to travel a path to understand its surroundings and achieve safe navigation. For example, the path may correspond to a route traversing an urban environment, a highway environment, or a rural environment, etc. In this way, the sensor data enables the vehicle 502 to detect, identify, and avoid potential obstacles, while also allowing the vehicle 502 to navigate according to traffic rules and road boundaries.

[0106] Figure 5A The illustration depicts scene 500 in a bird's-eye view, showing vehicle 502 traveling along a straight path while acquiring sensor data (e.g., radar data) to understand its surroundings. As shown, vehicle 502 may obtain radar measurements indicating the presence of a sign 504 and a ghost vehicle 506 aligned at an azimuth angle 510 relative to the line of sight 508. Due to the reflective nature of sign 504, vehicle 502's self-reflection may appear at the location of ghost vehicle 506, which could then appear in the radar data. Thus, without filtering the data corresponding to ghost vehicle 506, vehicle 502 could potentially perform unintended actions to avoid ghost vehicle 506.

[0107] Figure 5BThe side view of scenario 500 illustrates the range (distance) between vehicle 502, sign 504, and ghost vehicle 506. As described above, reflective surfaces can cause self-reflection of vehicle 502 to appear in radar data and other types of sensor data. In scenario 500, sign 504 is located at range 520 relative to vehicle 502, and can cause ghost vehicle 506 to appear in radar data at range 522 relative to vehicle 502. Range 522 (i.e., the distance between vehicle 502 and ghost vehicle 506) is approximately twice the distance between vehicle 502 and sign 504 (i.e., range 520). For example, range 520 could represent 12 meters between vehicle 502 and sign 504, while range 522 could represent 24 meters between vehicle 502 and ghost vehicle 506. Thus, a predefined model can predict the position of ghost vehicle 506 based on measurements obtained for sign 504, which further enables a computing device to filter ghost vehicle 506 from the data.

[0108] In some examples, the radar can indicate that marker 504 is measured using a Doppler value "D". Thus, a model used to predict the radar power corresponding to ghost vehicle 506 can indicate that ghost vehicle 506 has approximately twice the Doppler value "D". Therefore, the computing device can take Doppler measurements into account when anticipating and filtering ghost vehicle 506 from radar data representing one or more surfaces indicating its location.

[0109] Figure 6 This is a flowchart of an example method 600 for filtering vehicle self-reflection according to one or more embodiments. Method 600 may include one or more operations, functions, or actions as depicted by one or more of blocks 602, 604, 606, 608, 610, 612, and 614, each of which may be implemented by any system shown in the previous figures, among other possible systems.

[0110] Those skilled in the art will understand that the flowcharts described herein illustrate the functionality and operation of certain embodiments of this disclosure. In this regard, each block of the flowchart may represent a module, segment, or portion of program code, comprising one or more instructions executable by one or more processors for implementing a specific logical function or step in the process. The program code may be stored on any type of computer-readable medium, such as storage devices including disks or hard disk drives.

[0111] Furthermore, each block may represent a circuit that is wired to perform a specific logic function during the process. Alternative implementations are included within the scope of the exemplary implementations of this application, wherein functions may not be performed in the order shown or discussed, including substantially simultaneous or reverse order, depending on the functions involved, as understood by those skilled in the art.

[0112] In block 602, method 600 relates to receiving radar data. Sensor processing systems (e.g., Figure 4 The computing device 402 shown can acquire radar data from the vehicle's radar system for subsequent processing to understand the vehicle's environment. In other embodiments, different types of sensors, such as LIDAR or time-of-flight cameras, can be used. Furthermore, the radar data can originate from one or more radar units associated with the vehicle's radar system.

[0113] In block 604, method 600 relates to initiating a vehicle self-reflective filtering process. In some embodiments, the self-reflective filtering process can be a pipeline with a series of stages. Although for Figure 6 The method 600 shown illustrates the stages in a linear sequence, but other example methods can involve executing multiple stages in parallel. Furthermore, in some example embodiments, method 600 may involve automatically applying one or more filtering techniques. Specifically, radar processing may involve automatically filtering vehicle self-reflections to improve the accuracy of radar measurements. In some examples, the filtering process may involve performing a specific stage or a set of stages.

[0114] In block 606, method 600 relates to applying a first-stage filtering. Specifically, the vehicle autoreflection filtering process may initially involve determining a radar representation based on radar data that conveys information about surfaces in the environment. The first stage of the pipeline may involve using a radar data cube to convey environmental information and generating a corresponding data cube representing vehicle autoreflection information, which can be used to remove unwanted data from the radar.

[0115] Figure 7A An example radar data cube 700 is illustrated, comprising voxels arranged to represent radar measurements of the environment. Specifically, the radar data cube 700 may indicate range data 702, Doppler data 704, and azimuth data 706 of different surfaces located within the environment and detected within the radar data. For example, voxel 708 may convey measured RCS and Doppler values ​​of surfaces located within a specific range and azimuth of the environment relative to the vehicle.

[0116] The radar data cube 700 can represent information about a vehicle scene collected by radar hardware. That is, for each voxel in the radar data cube 700, the RCS can be measured and associated with that voxel. For example, a computing device can measure the RCS (range, θ, Doppler) of a voxel using a set of coordinates (range, θ, Doppler) and repeat this process for each voxel. In some cases, the RCS measurements of voxels can be determined in parallel.

[0117] The structure of the radar data cube 700 can be similarly used to represent vehicle autoreflection information. For example, a computing device can generate a similarly configured data cube (i.e., a vehicle autoreflection cube) representing the predicted power associated with vehicle autoreflection. To generate the vehicle autoreflection cube, the computing device can use a predefined model to associate the predicted RCS with each voxel based on the RCS measured for the corresponding voxel within the radar data cube 700. The predefined model involves factoring the measured RCS of surfaces in the environment to predict the RCS associated with vehicle autoreflection, as these may appear in the radar data.

[0118] As an example, for any RCS measurement, range, azimuth, and Doppler (e.g., RCS_self(RCS(range, θ, Doppler)), the computing device can use a model of what a vehicle would look like to the radar system if reflected from a reference isotropic transmitter. A metal marker located at a range of 10 meters, an azimuth of 45 degrees, and a Doppler value of D could cause a ghost vehicle to appear at 20 meters, an azimuth of 45 degrees, and a Doppler value of 2D. Thus, the vehicle self-reflection cube can indicate the estimated RCS power of the ghost vehicle at that location (e.g., 20 meters, 45 degrees azimuth, and a Doppler value of 2D). Another metal marker located at a range of 12 meters, an azimuth of 40 degrees, and a Doppler value of D could cause a ghost vehicle to appear at 24 meters, an azimuth of 40 degrees, and a Doppler value of 2D. The vehicle self-reflection cube can also indicate the estimated RCS power of the ghost vehicle at that location (e.g., 24 meters, 40 degrees azimuth, and a Doppler value of 2D).

[0119] The first stage of the filtering process can involve filtering information from within the vehicle autoreflection cube of the radar data cube 700. Specifically, it is possible to filter (e.g., subtract) the predicted RCS of each voxel within the vehicle autoreflection cube from the measured RCS of the corresponding voxel within the radar data cube 700. Thus, the modified radar data cube 700 includes RCS measurements that do not include the power associated with the vehicle autoreflection.

[0120] In block 608, method 600 relates to applying a second-stage filtering. The second-stage filtering may involve using one or more two-dimensional radar images corresponding to the radar data. For example, the radar data can be used to determine a 2D radar image comprising pixels, each pixel indicating information about a given surface.

[0121] Figure 7B A 2D radar image 710 is depicted, which includes pixels representing range data 712 and azimuth data 714 of a measurement surface within the environment. In other embodiments, the radar image 710 may convey both range data and Doppler data. Second-stage filtering may also involve generating a corresponding 2D image having pixels representing a predicted RCS measurement associated with vehicle autoreflection within the radar data (i.e., a vehicle autoreflection 2D image).

[0122] To generate a 2D image of the vehicle's autoreflection, the computing device can use a predefined model and information from within the radar image 710. Specifically, information represented by pixels (e.g., pixel 716) within the radar image 710 can be used to estimate the power of the vehicle's autoreflection. A second-stage filtering process can then involve filtering (e.g., subtracting) the estimated power associated with the vehicle's autoreflection from the RCS measurement represented by the 2D radar image 710. In some examples, the computing device can use a neural network to perform the filtering process.

[0123] In box 610, method 600 involves applying third-stage filtering. Third-stage filtering may involve radar data using a sparse data format. Figure 7C The illustration depicts sparse radar data according to one or more example embodiments. Sparse radar data 720 is shown as a graph with data points arranged according to an X-axis 722 and a Y-axis 724. Each point also has an associated radial velocity (calculated from its measured Doppler). Thus, filtering can include removing data points based on the predicted position and radial velocity of vehicle self-reflections within the sparse radar data 720. For example, if a predicted self-reflection is at a position and radial velocity sufficiently close to the measured point, that point is considered a self-reflection and is filtered out.

[0124] In box 612, method 600 relates to providing information about the mapping environment. For example, a computing device can provide control commands based on objects detected and identified within radar data. Object detection can be performed accurately without potentially phantom vehicles appearing in the radar data due to the use of different filtering techniques.

[0125] In some example embodiments, the processing system can execute multiple blocks of method 600 in parallel. For example, the processing system can utilize multiple digital signal processing cores that contribute computational resources to the execution of method 600.

[0126] Figure 8This is a flowchart of an example method 800 for performing vehicle self-reflection technology according to one or more embodiments. Method 800 may include one or more operations, functions, or actions, as depicted in one or more of blocks 802, 804, 806, 808, and 810, and each operation, function, or action may be performed by any system shown in the previous figures, among other possible systems.

[0127] In block 802, method 800 relates to receiving radar data indicating the environment at a computing device and from radar units coupled to the vehicle. One or more radar units are capable of acquiring the radar data.

[0128] In some embodiments, the computing device can cause the radar unit to transmit a radar signal having an extended linear frequency modulation (LFM) waveform. Therefore, the computing device is able to receive radar data based on the radar signal having an extended LFM waveform.

[0129] In box 804, method 800 involves determining a first radar representation based on radar data that conveys information about multiple surfaces in the environment.

[0130] The first radar representation can vary in different embodiments. In some examples, the computing device can determine a radar data cube having a first plurality of voxels, each voxel indicating information about a given surface in the environment. Each voxel in the first plurality of voxels can also indicate the extent, azimuth, and Doppler of the given surface in the environment.

[0131] In some embodiments, the computing device may use radar data to determine a two-dimensional radar image having a first plurality of pixels, each pixel indicating information about a given surface in the environment. The computing device may also use radar data in the form of a sparse dataset.

[0132] In box 806, method 800 involves generating a second radar representation using a predefined model, which assigns predicted autoreflectance values ​​to corresponding locations in the environment based on information about multiple surfaces conveyed by the first radar representation. The predefined model is capable of assigning predefined autoreflectance values ​​to a first location based on information about surfaces located at the second location and a relationship between the first and second locations. In some examples, the relationship between the first and second locations indicates that the first location is located at a first range and a specific azimuth relative to the vehicle, and the second location is located at a second range and a specific azimuth relative to the vehicle, where the first range is approximately twice the second range. In some examples, the relationship may further indicate that the first location includes a first Doppler relative to the vehicle, and the second location includes a second Doppler relative to the vehicle, where the first Doppler is approximately twice the second Doppler.

[0133] In some examples, the computing device can generate a second radar representation as a reflection data cube with a second plurality of voxels, each voxel representing a predicted autoreflectance value for a given location in the environment. Each voxel in the second plurality of voxels can also indicate the range, azimuth, and Doppler corresponding to the given location in the environment. The computing device can generate a two-dimensional reflection image with a second plurality of pixels, each pixel representing a predicted autoreflectance value for a given location in the environment.

[0134] In block 808, method 800 involves modifying the first radar representation based on predicted autoreflection values ​​in the second radar representation.

[0135] In some examples, modifying the first radar representation involves performing a filtering process to remove the reflection data cube from the radar data cube. For example, information about multiple surfaces indicated by multiple voxels includes radar cross-section (RCS) measurements for each surface. Thus, the computing device can perform a filtering process to subtract the predicted autoreflectance values ​​represented by a second set of voxels from the RCS measurements of the multiple surfaces indicated by the first set of voxels. In some examples, the computing device can also perform object detection using the first set of voxels based on the subtraction of the predicted autoreflectance values ​​represented by the second set of voxels from the RCS measurements of the multiple surfaces indicated by the first set of voxels.

[0136] In some examples, modifying the first radar representation involves performing a filtering process to remove the two-dimensional reflection image from the two-dimensional radar image. For example, information about the plurality of surfaces indicated by the first plurality of pixels may include RCS measurements of each surface. Thus, the computing device can filter (e.g., subtract) the predicted autoreflection values ​​represented by a second plurality of pixels from the RCS measurements of the plurality of surfaces indicated by the first plurality of pixels. In some examples, the computing device may also provide the two-dimensional reflection image and the two-dimensional radar image to a neural network configured to obtain the predicted autoreflection values ​​represented by the second plurality of pixels from the RCS measurements of the plurality of surfaces indicated by the first plurality of pixels.

[0137] In some examples, two-dimensional radar images include both Doppler measurements and RCS measurements. This allows for modified filtering schemes: instead of filtering (e.g., subtracting) each predicted ghost pixel, the system can first check if the measured Doppler of the first image pixel matches the predicted Doppler of the ghost pixel, and only perform filtering if the two are sufficiently close (within a threshold). Otherwise, the observed pixel data may not be autoreflective. The predicted Doppler of a ghost pixel is twice that of the first reflecting surface Doppler. Matching comparisons can also use modular distances to resolve Doppler aliasing. For example, for an object moving towards the SDC at 50 meters per second (m / s), a given radar waveform can measure the same Doppler as an object moving away from the SDC at 50 m / s; therefore, the modular distance between +49 m / s and -49 m / s would be 2 m / s instead of 98 m / s.

[0138] In box 810, method 800 relates to providing instructions to a vehicle control system based on a modified first radar representation. In some examples, the computing device may use the modified first radar representation to identify one or more objects in the environment and provide instructions to the control system representing the one or more objects.

[0139] Figure 9 This is a schematic diagram illustrating a conceptual portion view of an example computer program product, which includes a computer program for performing computer processing on a computing device, the computer program being arranged according to at least some of the embodiments presented herein. In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a non-transitory computer-readable storage medium, or computer program instructions encoded on other non-transitory media or articles of art.

[0140] In one embodiment, a signal carrying medium 902 is used to provide an example computer program product 900. The signal carrying medium 902 may include one or more programming instructions 904, which, when executed by one or more processors, can provide the above reference. Figures 1-8The described function or part of the function. In some examples, signal carrying medium 902 may include a non-transitory computer-readable medium 906, such as, but not limited to, hard disk drives, optical discs (CDs), digital video discs (DVDs), digital magnetic tapes, memory, etc. In some embodiments, signal carrying medium 902 may include a computer-recordable medium 908, such as, but not limited to, memory, read / write (R / W) CDs, R / W DVDs, etc. In some embodiments, signal carrying medium 902 may include a communication medium 910, such as, but not limited to, digital and / or analog communication media (e.g., optical fiber, waveguides, wired communication links, wireless communication links, etc.). Therefore, for example, signal carrying medium 902 can be transmitted wirelessly via communication medium 910.

[0141] One or more programming instructions 904 can be, for example, computer-executable and / or logic-implemented instructions. In some examples, such as Figure 1 The computing device of computer system 112 can be configured to provide various operations, functions, or actions in response to programming instructions 904 conveyed to computer system 112 by one or more of computer-readable medium 906, computer-recordable medium 908, and / or communication medium 910. Other devices can perform the operations, functions, or actions described herein.

[0142] Non-transitory computer-readable media can also be distributed across multiple data storage elements, which can be geographically separated from each other. The computing device that executes some or all of the stored instructions can be a vehicle, such as... Figures 1-2E The vehicle 100 shown. Alternatively, the computing device that executes some or all of the stored instructions can be another computing device, such as a server.

[0143] The detailed description above, with reference to the accompanying drawings, illustrates various features and functions of the disclosed systems, apparatus, and methods. While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be readily apparent. The various aspects and embodiments disclosed herein are for illustrative purposes and not for limitation; the true scope is indicated by the following claims.

[0144] It should be understood that the arrangements described herein are merely illustrative. Therefore, those skilled in the art will understand that other arrangements and other elements (e.g., machines, devices, interfaces, functions, sequences, and functional groupings, etc.) can be used interchangeably, and some elements may be omitted entirely depending on the desired outcome. Furthermore, many of the elements described are functional entities that can be implemented as discrete or distributed components or combined with other components in any suitable combination and location.

Claims

1. A method for filtering vehicle self-reflections in radar, comprising: At the computing device and from the radar unit coupled to the vehicle, radar data indicating the environment is received; Based on radar data, a first radar representation is determined that conveys information about multiple surfaces in the environment; The computing device generates a second radar representation using a predefined model. This second radar representation assigns predicted autoreflection values ​​to corresponding locations in the environment based on information about the plurality of surfaces conveyed by the first radar representation, indicating how likely the vehicle's autoreflection is to appear in radar data captured by radar units coupled to the vehicle. The predefined model enables the assignment of predicted autoreflection values ​​to a first location based on information about surfaces located at a second location, indicating how likely the vehicle's autoreflection is to appear in radar data at that first location. The first location is located at a first range and a specific azimuth angle relative to the vehicle, and the second location is located at a second range and the specific azimuth angle relative to the vehicle, wherein the first range is twice the size of the second range. Modify the first radar representation based on the predicted self-reflection value in the second radar representation; and The first radar is modified to provide instructions to the vehicle's control system.

2. The method according to claim 1, further comprising: The computing device causes the radar unit to transmit a radar signal with an extended linear frequency modulated waveform; as well as The radar data received to indicate the environment includes: Radar data is received based on radar signals with extended linear frequency modulation waveforms.

3. The method according to claim 1, wherein, The first radar representation that conveys information about multiple surfaces in the environment includes: A radar data cube with a first plurality of voxels is determined, each voxel indicating information about a given surface in the environment.

4. The method according to claim 3, wherein, Generating a second radar representation that assigns the predicted autoreflection values ​​to the corresponding locations in the environment includes: A reflection data cube with a second set of voxels is generated using a predefined model. Each voxel represents a predicted autoreflectance value at a given location in the environment. The relationship between the first and second locations further indicates that the first location includes a first Doppler relative to the vehicle, and the second location includes a second Doppler relative to the vehicle, wherein the first Doppler is twice the second Doppler.

5. The method according to claim 4, wherein, Each of the first plurality of voxels also indicates the extent, azimuth, and Doppler of a given surface in the environment, and Each of the second plurality of voxels also indicates the range, azimuth, and Doppler corresponding to a given location in the environment.

6. The method according to claim 5, wherein, Modifying the first radar representation based on the predicted self-reflection value in the second radar representation includes: Perform a filtering process to remove reflection data cubes from radar data cubes.

7. The method according to claim 6, wherein, Information about the multiple surfaces indicated by the first plurality of voxels includes radar cross section (RCS) measurements for each surface; and The filtering process for removing reflection data cubes from radar data cubes includes: Subtract the predicted autoreflectance value represented by the second set of voxels from the RCS measurements of the multiple surfaces indicated by the first set of voxels.

8. The method of claim 7, further comprising: Object detection is performed using the first plurality of voxels by subtracting the predicted autoreflectance values ​​represented by the second plurality of voxels from the RCS measurements of the multiple surfaces indicated by the first plurality of voxels.

9. The method according to claim 1, wherein, The first radar representation that conveys information about multiple surfaces in the environment includes: A two-dimensional radar image with a first plurality of pixels is determined, each pixel indicating information about a given surface in the environment.

10. The method according to claim 9, wherein, Generating a second radar representation that assigns the predicted autoreflection values ​​to the corresponding locations in the environment includes: Generate a two-dimensional reflection image with a second set of pixels, each pixel representing a predicted autoreflectance value for a given location in the environment.

11. The method according to claim 10, wherein, Each of the first plurality of pixels also indicates the extent and azimuth of a given surface in the environment, and Each of the second plurality of pixels also indicates the range and azimuth angle corresponding to a given location in the environment.

12. The method according to claim 11, wherein, Modifying the first radar representation based on the predicted self-reflection value in the second radar representation includes: Perform a filtering process to remove two-dimensional reflection images from two-dimensional radar images.

13. The method according to claim 12, wherein, Information about the multiple surfaces indicated by the first plurality of pixels includes radar cross section (RCS) measurements for each surface; and The filtering process for removing two-dimensional reflection images from two-dimensional radar images includes: Subtract the predicted autoreflection value represented by the second set of pixels from the RCS measurements of the multiple surfaces indicated by the first set of pixels.

14. The method according to claim 11, wherein, Subtracting the predicted autoreflectance value represented by the second set of pixels from the RCS measurements of the multiple surfaces indicated by the first set of pixels includes: The two-dimensional reflection image and the two-dimensional radar image are provided to a neural network, which is configured to filter predicted autoreflection values ​​represented by a second plurality of pixels from RCS measurements of a plurality of surfaces indicated by a first plurality of pixels.

15. The method of claim 1, further comprising: Based on the modification of the first radar representation, one or more objects in the environment are identified using the first radar representation; and The provision of instructions to the vehicle's control system based on the modification of the first radar representation includes: Provide instructions to the control system representing one or more objects.

16. A system for filtering vehicle self-reflections in radar, comprising: Radar unit coupled to the vehicle; The computing device is configured as follows: Receive radar data indicating the environment from the radar unit; Based on radar data, a first radar representation is determined that conveys information about multiple surfaces in the environment; A second radar representation is generated using a predefined model. This second radar representation assigns predicted autoreflection values ​​to corresponding locations in the environment based on information about multiple surfaces transmitted by the first radar representation, indicating how likely the vehicle's autoreflection is to appear in radar data captured by radar units coupled to the vehicle. The predefined model enables the assignment of predicted autoreflection values ​​to a first location based on information about surfaces located at the second location, indicating how likely the vehicle's autoreflection is to appear in radar data at the first location. The first location is located at a first range and a specific azimuth angle relative to the vehicle, and the second location is located at a second range and the specific azimuth angle relative to the vehicle, wherein the first range is twice the size of the second range. Modify the first radar representation based on the predicted self-reflection value in the second radar representation; and The first radar is modified to provide instructions to the vehicle's control system.

17. The system according to claim 16, wherein, The relationship between the first position and the second position indicates that the first position is located at a first range and a specific azimuth angle relative to the vehicle, and the second position is located at a second range and the specific azimuth angle relative to the vehicle, wherein the first range is twice the size of the second range.

18. The system according to claim 17, wherein, Information regarding the plurality of surfaces in the environment includes radar cross section (RCS) measurements; and The computing device is further configured as follows: Subtract the predicted self-reflection value from the corresponding RCS measurement in the first radar representation.

19. A non-transitory computer-readable medium storing instructions executable by one or more processors to cause a computing system to perform functions, said functions including: Receives radar data indicating the environment from the radar unit coupled to the vehicle; Based on radar data, a first radar representation is determined that conveys information about multiple surfaces in the environment; A second radar representation is generated using a predefined model. This second radar representation assigns predicted autoreflection values ​​to corresponding locations in the environment based on information about multiple surfaces conveyed by the first radar representation, indicating how likely the vehicle's autoreflection is to appear in radar data captured by radar units coupled to the vehicle. The predefined model enables the assignment of predicted autoreflection values ​​to a first location based on information about surfaces located at the second location, indicating how likely the vehicle's autoreflection is to appear in radar data at the first location. The first location is located at a first range and a specific azimuth angle relative to the vehicle, and the second location is located at a second range and the specific azimuth angle relative to the vehicle, wherein the first range is twice the size of the second range. Modify the first radar representation based on the predicted self-reflection value in the second radar representation; and The first radar is modified to provide instructions to the vehicle's control system.

Images