Method and system for detecting adverse road conditions using radar

Abstract

The example embodiment relates to a technique for detecting adverse road conditions using radar. A computing device may generate a first radar representation representing a field of view of a radar unit coupled to a vehicle during clear weather conditions and store the first radar representation in a memory. The computing device may receive radar data from the radar unit during navigation of the vehicle on a road and determine a second radar representation based on the radar data. The computing device may also perform a comparison between the first radar representation and the second radar representation and determine a road condition of the road based on the comparison. The road condition may represent precipitation located on the road and control instructions may be provided to the vehicle based on the road condition of the road.

Description

Technical Field

[0001] The present disclosure relates to methods and systems for detecting adverse road conditions using radar. Background Art

[0002] Radio detection and ranging systems (“radar systems”) are used to estimate distances to environmental features by transmitting radio signals and detecting the returned reflected signals. The distance to a radio-reflective feature in the environment can then be determined based on the time delay between transmission and reception. Radar systems can transmit signals with a frequency that varies over time, such as a signal with a time-varying frequency ramp, and then correlate the frequency difference between the transmitted signal and the reflected signal with a range estimate. Some radar systems can also estimate the relative motion of a reflecting object based on the Doppler shift in the received reflected signal.

[0003] Directional antennas can be used for transmission and / or reception of signals to associate each distance estimate with a bearing. More generally, directional antennas can also be used to focus the radiated energy on a given field of view of interest. Combining the measured distance and direction information can allow mapping of the surrounding environmental features. Summary of the Invention

[0004] Example embodiments describe techniques for using radar to detect adverse road conditions in real time. Radar measurements captured by a vehicle radar system during navigation can be compared with radar data obtained from the same radar unit under clear conditions to detect when adverse road conditions, such as rain or snow that may affect the navigation of the vehicle, are present. When the radar measurements detect road spray kicked up by the vehicle's wheels, the control system can adjust the vehicle operation to safely navigate the detected adverse road conditions. The adverse road conditions can be communicated to other vehicles to increase safe navigation across the vehicle fleet.

[0005] In one aspect, an example method is provided. The method involves generating a first radar representation that represents the field of view of a radar unit during clear weather conditions. The radar unit is coupled to the outer surface of a vehicle. The method also involves storing the first radar representation in a memory and receiving, at a computing device, radar data from the radar unit while the vehicle is navigating on a road. The method also involves determining, by the computing device, a second radar representation based on the radar data and performing, by the computing device, a comparison between the first radar representation and the second radar representation. The method also involves determining, based on the comparison, the road condition of the road. The road condition represents the precipitation present on the road. The method also involves providing control instructions to the vehicle based on the road condition of the road.

[0006] In another aspect, an example system is provided. The system includes a radar unit coupled to a vehicle and a computing device. The computing device is configured to generate a first radar representation representing the field of view of the radar unit during clear weather conditions and is configured to store the first radar representation in a memory. The computing device is further configured to receive radar data from the radar unit during navigation of the vehicle on a road and is configured to determine a second radar representation based on the radar data. The computing device is further configured to perform a comparison between the first radar representation and the second radar representation and is configured to determine the road condition of the road based on the comparison. The road condition represents the precipitation on the road. The computing device is further configured to provide control instructions to the vehicle based on the road condition of the road.

[0007] In another example, an example non-transitory computer-readable medium is configured to store instructions that, when executed by a computing system including one or more processors, cause the computing system to perform operations. The operations involve generating a first radar representation representing the field of view of the radar unit during clear weather conditions. The radar unit is coupled to an outer surface of the vehicle. The operations further involve storing the first radar representation in a memory and receiving radar data from the radar unit during navigation of the vehicle on a road. The operations further involve determining a second radar representation based on the radar data and performing a comparison between the first radar representation and the second radar representation. The operations further involve determining the road condition of the road based on the comparison. The road condition represents the precipitation on the road. The operations further involve providing control instructions to the vehicle based on the road condition of the road.

[0008] The foregoing summary is illustrative only and is not intended to be limiting in any way. In addition to the above-described illustrative aspects, embodiments, and features, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a functional block diagram of a vehicle showing one or more example embodiments.

[0010] Figure 2A shows a side view of a vehicle according to one or more example embodiments.

[0011] Figure 2B shows a top view of a vehicle according to one or more example embodiments.

[0012] Figure 2C shows a front view of a vehicle according to one or more example embodiments.

[0013] Figure 2D shows a rear view of a vehicle according to one or more example embodiments.

[0014] Figure 2EShows additional views of a vehicle in accordance with one or more example embodiments.

[0015] Figure 3 Is a simplified block diagram of a computing system in accordance with one or more example embodiments.

[0016] Figure 4 Is a system for wireless communication between a vehicle and a computing device in accordance with one or more example embodiments.

[0017] Figure 5A Shows a wheel that generates road spray during forward movement in accordance with one or more example embodiments.

[0018] Figure 5B Shows a side view of a vehicle that uses a rear-facing radar unit to detect road spray in accordance with one or more example embodiments.

[0019] Figure 5C Shows a top view of a vehicle that uses a rear-facing radar unit to detect road spray in accordance with one or more example embodiments.

[0020] Figure 5D Shows a top view of a vehicle that uses multiple radar units to detect road spray in accordance with one or more example embodiments.

[0021] Figure 6 Shows a vehicle that uses forward-facing radar to detect road spray caused by another vehicle in accordance with one or more example embodiments.

[0022] Figure 7 Is a flowchart of a method for using radar to detect adverse road conditions in real time in accordance with one or more example embodiments.

[0023] Figure 8 Is a schematic diagram of a computer program in accordance with one or more example embodiments. Detailed Description

[0024] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, like symbols typically identify like components unless the context dictates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are explicitly contemplated herein.

[0025] A radar system is used to detect objects and estimate their location by transmitting electromagnetic signals (radar signals) and analyzing the backscattered signals from objects and other surfaces in the environment. The system can estimate the range of an object by transmitting short pulses and / or coded waveforms (such as pulsed Doppler radar, which involves coherent bursts of short pulses at a certain carrier frequency). In some applications, the electromagnetic energy is concentrated in the form of a beam into a specific spatial sector via a parabolic reflector or an array of antenna elements associated with the radar unit.

[0026] A radar processing system (e.g., a computing device) can process radar data to generate two-dimensional (2D) and / or three-dimensional (3D) measurements representing aspects of the environment, such as the location, orientation, and movement of nearby objects and other surfaces occupying the environment near the radar system. Since radar systems can be used to measure the distance and motion of nearby objects and other surfaces, vehicles are increasingly incorporating vehicle radar systems, which can obtain and provide useful information for vehicle navigation, obstacle avoidance, and other vehicle operations that can enhance overall vehicle safety. Radar can be used to detect and identify the location, orientation, and movement of nearby vehicles, bicycles, pedestrians, and animals. In addition, radar can also reveal information about other features in the vehicle's surrounding environment, such as the location, arrangement, and positioning of road boundaries, road conditions (e.g., smooth or bumpy surfaces), weather conditions (e.g., wet or snow-covered roads), and the relative positioning of traffic signs and signals. Thus, radar provides a means for vehicle systems to continuously monitor and understand the changes during navigation in various environments and can supplement sensor data from other types of sensors.

[0027] In certain applications, a vehicle radar system can provide information intended to assist the driver of the vehicle. For example, radar measurements can be used to generate alerts when the vehicle drifts out of its lane, when the vehicle is traveling too close to another vehicle or object, and / or can be used in other ways to help the driver safely control the vehicle. Radar measurements can also be used to help enable autonomous or semi-autonomous operation of the vehicle. In particular, the control system can use radar data to understand and safely navigate the vehicle's environment in near real-time, as described above.

[0028] During navigation, adverse road conditions caused by rain, snow, puddles, etc. can affect the performance of the vehicle. Precipitation on the road can be kicked up by the wheels to become road spray and may cause sensor degradation, such as aperture contamination. Road spray can degrade sensor performance, and the slow accumulation of road spray is not easily noticeable and difficult to detect. Therefore, there is a need for a system that can detect adverse road conditions during vehicle navigation.

[0029] The example embodiments presented herein relate to techniques for using radar to detect adverse road conditions in real time. A vehicle radar system can be used to sense backscattered energy from rain and snow that enters the field of view of one or more radar units positioned on the vehicle. For example, a rear-mounted radar unit placed on the vehicle can detect rain and snow kicked up by the natural rotation of the vehicle's own tires. In some examples, the Doppler resolvability of road spray from nearby ground clutter can be used to improve the detectability of road spray. Thus, observation of the near-range response power can be used to detect road spray and puddles located on the road. A real-time detector can be constructed by measuring the cumulative energy of spatial and / or velocity filters and comparing the energy to clear weather conditions.

[0030] Understanding road conditions can enable an autonomous driving system to operate autonomously in adverse weather conditions. In particular, knowledge of adverse road conditions can be used to inform the vehicle's control system that it may suffer reduced traction or that the sensors may be vulnerable to road spray from other vehicles ahead of the vehicle. If the control system detects any of these conditions, the control system can adjust vehicle operation to increase the response time and increase the vehicle following distance. In such conditions, the control system can also apply the brakes earlier. Additionally, the location of puddles or other road precipitation can be used for route planning to identify areas that the autonomous vehicle should avoid driving through.

[0031] Some examples may involve using automotive radar to directly measure road spray. For example, a computing device can detect the presence and amount of road spray based on near-range, power integration, range-Doppler maps, and / or spatial cells of expected road spray compensated for background (e.g., no road spray conditions). Integration of spatial filters in range-angle space can be used to obtain a high signal-to-noise ratio (SNR) for an imaging radar.

[0032] Furthermore, applications can stem from real-time knowledge of road spray. Road spray can cause sensor degradation (e.g., aperture contamination) and can cause the vehicle to have false positive object detections. Thus, potential mitigations performed in response to adverse road conditions can include sensor cleaning and calibration, adjusting sensor detection thresholds, and increasing the following distance relative to other vehicles to maintain a safety buffer for a given road condition.

[0033] Other mitigation techniques may also be performed. For example, vehicles can share adverse road conditions in real time, which can enable route adjustment to avoid specific roads (e.g., flooded roads) suffering from poor conditions. In some embodiments, the map used by the vehicle for route planning and navigation can be updated based on the detected adverse road conditions. Thus, the vehicle can utilize the map to avoid potential conditions with adverse conditions during route planning. In a further example, weather reports and trends can be utilized to predict which roads may have adverse conditions at a given time. For example, a vehicle can proactively avoid roads that typically flood during rainfall based on weather reports (e.g., hourly weather forecasts) and / or when detecting rainfall or another type of precipitation.

[0034] In some examples, for forward-facing radar, road spray from the leading vehicle may cause an increase in kinematically consistent false detections, which may trigger false tracking in the radar or other sensor pipelines (e.g., lidar). In such examples, the sensing power from the rear radar road spray can be used to provide a consistent measurement of the amount of road spray present on the road being traveled. The consistent measurement of the amount of road spray can be used to adjust the constant false alarm rate (CFAR) detection threshold to reduce false alarm tracking. Thus, the vehicle system can continue to safely navigate in a manner that takes into account the road spray on the road without potential false tracking that may interfere with autonomous operation.

[0035] For further illustration, example techniques can involve a computing device that generates a background radar representation representing the field of view of a radar unit during clear weather conditions. For example, the background radar representation can represent clear weather conditions of a rear-facing radar unit having a field of view that measures the environment behind the vehicle. Thus, the background radar representation can indicate the measurements that the radar unit may obtain during clear weather conditions. For example, clear weather conditions can represent a dry condition where precipitation (e.g., snow, rain) is not falling and there is no thin film of water on the ground in the form of puddles. In clear weather conditions, the road may be clean and free of precipitation that may be lifted into the field of view of the radar unit on the vehicle. As a result, the background radar representation will not include energy from detected road spray and can be used as a baseline (e.g., energy level) against which subsequent radar returns can be compared to detect adverse road conditions.

[0036] The generation of a background radar representation can vary in example embodiments. For example, a computing device can generate a background radar representation based on a series of radar measurements from a radar unit. These radar measurements can be obtained at different times, in various environments, and during navigation when the vehicle is traveling on different roads to obtain various radar measurements that can be used to generate a background radar representation that represents the average of the radar measurements. The various measurements can represent different environmental conditions where the road is dry but other factors may vary. Thus, the background radar representation can represent the average energy level detected by the radar unit across its field of view when traveling on a road without adverse conditions. The computing device can filter measurements having energy levels that may indicate the vehicle is traveling on an adverse road or a nearby object has been detected in order to generate the background radar representation as a representation of the clear condition of the field of view of the radar unit. In other examples, the computing device can receive the background representation from a remote computing system, such as a fleet management system that can calibrate the background representation and send the background representation to multiple vehicles.

[0037] Thus, adverse road condition detection techniques may involve using the generated background representation to detect such conditions in real time as they occur during vehicle navigation. In particular, the background representation can be stored and accessed from memory at any time the vehicle is navigating and performing the adverse road condition detection technique. For example, the system can periodically check the road condition (e.g., every two minutes or every 10 minutes of navigation) or can be configured to continuously monitor for adverse road conditions.

[0038] In some examples, to identify adverse road conditions in real time, the computing device can compare the radar returns to the background radar representation. The comparison can yield information indicating when the radar unit's detection has been splashed by one or more wheels, which may indicate the vehicle is traveling on a wet road. The vehicle can then adjust its operation to accommodate the wet road, which may involve increasing the following space relative to other vehicles and making other adjustments (e.g., applying brakes earlier based on the wet road).

[0039] In some examples, the degree of adjustment can depend on the amount of road splash detected on the road. For example, when the technique indicates a higher level of precipitation on the road, the vehicle can perform techniques designed for a high level of precipitation, and these techniques can be different from those performed for a low level of precipitation on the road. Similarly, detecting rain or snow may further contribute to additional adjustments to the vehicle control system.

[0040] A computing device can use radar data in the form of two-dimensional (2D) radar images to detect adverse road conditions. In particular, the background radar representation and newly acquired radar measurements can take the form of 2D radar images representing radar measurements (e.g., range and Doppler information). Thus, the radar images generated when a vehicle is navigating along a path on a road can be compared with the background radar representation to identify when road spray is being kicked up by the vehicle's wheels due to adverse road conditions.

[0041] For example, the energy from the background radar representation can be compared with the energy from a newly determined radar image generated during vehicle navigation. When these energy levels are different, the computing device can use the difference to detect and quantify the road spray kicked up by the vehicle's wheels. For example, when the difference is greater due to the radar image having more energy, this may indicate that more precipitation is on the road. The computing device can also factor in other parameters such as the vehicle's current speed which can affect the amount of water kicked up by the tires. In other embodiments, the radar representation can represent the energy level in another way, such as a numerical quantity representing the total energy associated with measurements across the field of view of the radar unit.

[0042] In some examples, a vehicle system can maintain an average energy level from measurements of a radar unit during clear road conditions and generate a signal when the detected energy from the radar increases above a threshold energy level. The increase in energy can indicate the presence of an object or road spray. The vehicle system can then distinguish between the object and the road spray, which can involve using stored past energy returns from the road spray or using additional measurements from the radar unit and / or another sensor (e.g., a camera).

[0043] In addition, a vehicle system can detect adverse road conditions based on road spray generated by other nearby vehicles. For example, the vehicle system can perform similar techniques to detect the energy of road spray measurements generated from another vehicle traveling in front of the vehicle. In addition, the vehicle system can also share information about road conditions with other nearby vehicles. For example, the vehicle system can send a map indicating the presence of adverse road conditions to other vehicles located within a threshold geographical distance relative to a road with adverse road conditions. In some examples, the adverse conditions may cause the vehicle system to perform radome cleaning techniques. For example, the vehicle may apply heat to the radome to eliminate sleet or snow that may affect potential radar measurements. In other examples, the radome can use wipers or blowers to help remove the precipitation located on the radome.

[0044] The following detailed description may be used in conjunction with one or more radar units having one or more antenna arrays. The one or more antenna arrays may take the form of a single-input single-output (SISO), single-input multiple-output (SIMO), multiple-input single-output (MISO), multiple-input multiple-output (MIMO), and / or synthetic aperture radar (SAR) radar antenna architecture. In some embodiments, an example radar unit architecture may include a plurality of “dual-ended open waveguide” (DOEWG) antennas. The term “DOEWG” may refer to a short section of a horizontal waveguide channel plus a vertical channel split into two parts. Each of the two parts of the vertical channel may include an output port configured to radiate at least a portion of the electromagnetic waves into the radar unit. Additionally, in some instances, the plurality of DOEWG antennas may be arranged into one or more antenna arrays.

[0045] Some example radar systems may be configured to operate at an electromagnetic wave frequency in the W-band (e.g., 77 gigahertz (GHz)). The W-band may correspond to electromagnetic waves on the millimeter scale (e.g., 1 millimeter or 4 millimeters). Such antennas may be compact (typically having a rectangular form factor), efficient (i.e., little 77 GHz energy is lost to heat in the antenna or reflected back to the transmitter electronics), low-cost, and easy to manufacture (i.e., radar systems with these antennas can be mass-produced).

[0046] An antenna array may involve a set of multiple connected antennas that can work together as a single antenna to transmit or receive signals. By combining multiple radiating elements (i.e., antennas), the antenna array can enhance the performance of the radar unit compared to a radar unit using a non-array antenna. In particular, when the radar unit is equipped with one or more antenna arrays, higher gain and a narrower beam can be achieved. In some examples, the radar unit may include the same number of transmit antenna arrays and receive antenna arrays (e.g., four transmit antenna arrays and four receive antenna arrays). In other examples, the radar unit may include a plurality of transmit antenna arrays with a different number than the receive antenna arrays (e.g., six transmit antenna arrays and three receive antenna arrays). Additionally, some radar units may operate with parasitic arrays that can control radar transmissions. Other example radar units may include one or more drive arrays having radiating elements connected to an energy source, which may have less total energy loss compared to parasitic arrays.

[0047] The antennas on the radar unit can be arranged in one or more linear antenna arrays (i.e., the antennas within the array are aligned in a straight line). For example, the radar unit can include multiple linear antenna arrays arranged in a specific configuration (e.g., in parallel lines on the radar unit). In other examples, the antennas can also be arranged in a planar array (i.e., the antennas are arranged in multiple parallel lines on a single plane). Additionally, some radar units can have antennas arranged in multiple planes, resulting in a three-dimensional array. The radar unit can also include multiple types of arrays (e.g., a linear array on one part and a planar array on another part).

[0048] Furthermore, radar units can have different functions and operational characteristics. For example, one radar unit can be configured for long-range operation, while another radar unit can be configured for short-range operation. The radar system can use a combination of different radar units to measure different areas of the environment. Therefore, it may be desirable to optimize the signal processing of the short-range radar unit for radar reflections in the near field of the radar unit.

[0049] Now referring to the drawings, Figure 1 is a functional block diagram showing a vehicle 100, which represents a vehicle capable of operating in an autonomous mode, either fully or partially. More specifically, the vehicle 100 can operate in an autonomous mode without human interaction by receiving control instructions from a computing system (such as a vehicle control system). As part of operating in an autonomous mode, the vehicle 100 can use sensors (e.g., the sensor system 104) to detect and possibly identify objects in the surrounding environment to enable safe navigation. In some example embodiments, the vehicle 100 can also include a subsystem that enables a driver (or a remote operator) to control the operation of the vehicle 100.

[0050] As Figure 1 shown, the vehicle 100 includes various subsystems, such as a propulsion system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, a power supply 110, a computer system 112, a data storage 114, and a user interface 116. The subsystems and components of the vehicle 100 can be interconnected in various ways (e.g., wired or secure wireless connections). In other examples, the vehicle 100 can include more or fewer subsystems. Additionally, within an implementation, the functions of the vehicle 100 described herein can be divided into additional functions or physical components, or combined into fewer functions or physical components.

[0051] The propulsion system 102 may include one or more components operable to provide powered movement for the vehicle 100, and may include an engine / motor 118, an energy source 119, a transmission 120, and wheels / tires 121, as well as other possible components. For example, the engine / motor 118 may be configured to convert the energy source 119 into mechanical energy, and may correspond to an internal combustion engine, one or more electric motors, a steam engine, or a Stirling engine, among one or a combination thereof, as well as other possible options. For example, in some embodiments, the propulsion system 102 may include multiple types of engines and / or motors, such as a gasoline engine and an electric motor.

[0052] The energy source 119 represents a source of energy that may power, in whole or in part, one or more systems of the vehicle 100 (e.g., the engine / motor 118). For example, the energy source 119 may correspond to gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and / or other sources of electricity. In some embodiments, the energy source 119 may include a combination of a fuel tank, a battery, a capacitor, and / or a flywheel.

[0053] The transmission 120 may transfer mechanical power from the engine / motor 118 to the wheels / tires 121 and / or other possible systems of the vehicle 100. Accordingly, the transmission 120 may include a gearbox, a clutch, a differential, and a drive shaft, as well as other possible components. The drive shaft may include axles connected to one or more wheels / tires 121.

[0054] Within an example embodiment, the wheels / tires 121 of the vehicle 100 may have various configurations. For example, the vehicle 100 may exist in the form of a unicycle, a bicycle / motorcycle, a tricycle, or a sedan / truck four-wheel form, as well as other possible configurations. Accordingly, the wheels / tires 121 may be connected to the vehicle 100 in various ways and may exist in different materials (such as metal and rubber).

[0055] The sensor system 104 may include various types of sensors, such as a Global Positioning System (GPS) 122, an Inertial Measurement Unit (IMU) 124, one or more radar units 126, a laser rangefinder / LIDAR unit 128, a camera 130, a steering sensor 123, and a throttle / brake sensor 125, as well as other possible sensors. In some embodiments, the sensor system 104 may further include sensors configured to monitor the internal systems of the vehicle 100 (e.g., an O2 monitor, a fuel gauge, an engine oil temperature, the condition of the brakes).

[0056] The GPS 122 may include a transceiver operable to provide information regarding the positioning of the vehicle 100 relative to the Earth. The IMU 124 may have a configuration using one or more accelerometers and / or gyroscopes and may sense changes in the positioning and orientation of the vehicle 100 based on inertial acceleration. For example, when the vehicle 100 is stationary or in motion, the IMU 124 may detect the pitch and yaw of the vehicle 100.

[0057] The radar unit 126 may represent one or more systems configured to sense objects (e.g., radar signals) within the local environment of the vehicle 100 using radio signals, including the speed and heading of the objects. Thus, the radar unit 126 may include one or more radar units equipped with one or more antennas configured to transmit and receive radar signals as described above. In some embodiments, the radar unit 126 may correspond to a mountable radar system configured to obtain measurements of the surrounding environment of the vehicle 100. For example, the radar unit 126 may include one or more radar units configured to be coupled to the bottom of the vehicle.

[0058] The laser rangefinder / LIDAR 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components, and may operate in a coherent mode (e.g., using heterodyne detection) or in a non-coherent detection mode. The camera 130 may include one or more devices (e.g., a still camera or a video camera) configured to capture images of the environment of the vehicle 100.

[0059] The steering sensor 123 may sense the steering angle of the vehicle 100, which may involve measuring the angle of the steering wheel or measuring an electrical signal representative of the angle of the steering wheel. In some embodiments, the steering sensor 123 may measure the angle of the wheels of the vehicle 100, such as detecting the angle of the wheels relative to the front axle of the vehicle 100. The steering sensor 123 may also be configured to measure a combination (or subset) of the angle of the steering wheel, the electrical signal representative of the angle of the steering wheel, and the angle of the wheels of the vehicle 100.

[0060] The throttle / brake sensor 125 can detect the position of the throttle position or brake position of the vehicle 100. For example, the throttle / brake sensor 125 can measure the angles of both the accelerator pedal (throttle) and the brake pedal, or can measure an electrical signal that can represent, for example, the angle of the accelerator pedal (throttle) and / or the angle of the brake pedal. The throttle / brake sensor 125 can also measure the angle of the throttle body of the vehicle 100, which can include a portion of the physical mechanism that provides modulation of the energy source 119 to the engine / motor 118 (e.g., a butterfly valve or a carburetor). In addition, the throttle / brake sensor 125 can measure the pressure of one or more brake pads on the rotor of the vehicle 100, or a combination (or subset) of the angles of the accelerator pedal (throttle) and the brake pedal, the electrical signal representing the angles of the accelerator pedal (throttle) and the brake pedal, the angle of the throttle body, and the pressure exerted by at least one brake pad on the rotor of the vehicle 100. In other embodiments, the throttle / brake sensor 125 can be configured to measure the pressure applied to the pedals of the vehicle (such as the throttle or brake pedal).

[0061] The control system 106 can include components configured to assist in navigating the vehicle 100, such as a steering unit 132, a throttle 134, a brake unit 136, a sensor fusion algorithm 138, a computer vision system 140, a navigation / path system 142, and an obstacle avoidance system 144. More specifically, the steering unit 132 is operable to adjust the heading of the vehicle 100, and the throttle 134 can control the operating speed of the engine / motor 118 to control the acceleration of the vehicle 100. The brake unit 136 can decelerate the vehicle 100, which may involve using friction to slow down the wheels / tires 121. In some embodiments, the brake unit 136 can convert the kinetic energy of the wheels / tires 121 into an electric current for subsequent use by one or more systems of the vehicle 100.

[0062] The sensor fusion algorithm 138 can include a Kalman filter, a Bayesian network, or other algorithms that can process data from the sensor system 104. In some embodiments, the sensor fusion algorithm 138 can provide an assessment based on the incoming sensor data, such as an evaluation of individual objects and / or features, an evaluation of a specific situation, and / or an evaluation of potential impacts within a given situation.

[0063] The computer vision system 140 can include hardware and software operable to process and analyze images in an effort to identify objects, environmental objects (e.g., stop lights, road boundaries, etc.), and obstacles. Thus, the computer vision system 140 can use object recognition, structure from motion (SFM), video tracking, and other algorithms used in computer vision, for example, to identify objects, map the environment, track objects, estimate the speed of objects, and so on.

[0064] The navigation / path system 142 can determine the driving path of the vehicle 100, which can involve dynamically adjusting the navigation during operation. Thus, the navigation / path system 142 can use data from the sensor fusion algorithm 138, GPS 122, maps, and other sources to navigate the vehicle 100. The obstacle avoidance system 144 can evaluate potential obstacles based on sensor data and cause the systems of the vehicle 100 to avoid or negotiate potential obstacles.

[0065] As Figure 1 shown, the vehicle 100 can also include peripheral devices 108, such as a wireless communication system 146, a touch screen 148, a microphone 150, and / or a speaker 152. The peripheral devices 108 can provide controls or other elements for a user to interact with the user interface 116. For example, the touch screen 148 can provide information to the user of the vehicle 100. The user interface 116 can also receive input from the user via the touch screen 148. The peripheral devices 108 can also enable the vehicle 100 to communicate with devices such as other vehicle devices.

[0066] The wireless communication system 146 can communicate securely and wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as WiMAX or LTE. Alternatively, the wireless communication system 146 can communicate with a wireless local area network (WLAN) using WiFi or other possible connections. For example, the wireless communication system 146 can also communicate directly with devices using an infrared link, Bluetooth, or ZigBee. Within the context of the present disclosure, other wireless protocols, such as various vehicle communication systems, are also possible. For example, the wireless communication system 146 can include one or more dedicated short range communication (DSRC) devices, which can include public and / or private data communication between vehicles and / or roadside stations.

[0067] The vehicle 100 also includes a power source 110 for powering the components. In some embodiments, the power source 110 can include a rechargeable lithium ion or lead acid battery. For example, the power source 110 can include one or more batteries configured to provide power. The vehicle 100 can also use other types of power sources. In an exemplary embodiment, the power source 110 and the energy source 119 can be integrated into a single energy source.

[0068] Vehicle 100 may also include a computer system 112 to perform operations such as those described herein. Thus, computer system 112 may include at least one processor 113 (which may include at least one microprocessor) operable to execute instructions 115 stored in a non-transitory computer-readable medium such as data store 114. In some embodiments, computer system 112 may represent multiple computing devices that may be used to control individual components or subsystems of vehicle 100 in a distributed manner.

[0069] In some embodiments, data store 114 may contain instructions 115 (e.g., program logic) executable by processor 113 to perform various functions of vehicle 100, including those described above in connection with Figure 1 those). Data store 114 may also contain additional instructions, including instructions for transmitting data to, receiving data from, interacting with, and / or controlling one or more of propulsion system 102, sensor system 104, control system 106, and peripheral devices 108.

[0070] In addition to instructions 115, data store 114 may store data such as road maps, path information, and other information. Such information may be used by vehicle 100 and computer system 112 during operation of vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0071] Vehicle 100 may include a user interface 116 for providing information to or receiving input from a user of vehicle 100. User interface 116 may control the content and / or layout of interactive images that may be displayed on touch screen 148, or enable control of the content and / or layout of interactive images that may be displayed on touch screen 148. Additionally, user interface 116 may include one or more input / output devices within a group of peripheral devices 108, such as wireless communication system 146, touch screen 148, microphone 150, and speaker 152.

[0072] Computer system 112 may control the functions of vehicle 100 based on inputs received from various subsystems (e.g., propulsion system 102, sensor system 104, and control system 106) as well as from user interface 116. For example, computer system 112 may utilize inputs from sensor system 104 to estimate the outputs generated by propulsion system 102 and control system 106. Depending on the embodiment, computer system 112 may be operable to monitor many aspects of vehicle 100 and its subsystems. In some embodiments, computer system 112 may disable some or all functions of vehicle 100 based on signals received from sensor system 104.

[0073] The components of vehicle 100 can be configured to operate in a manner that interconnects with other components within or outside their respective systems. For example, in an exemplary embodiment, camera 130 can capture multiple images, which can represent information about the state of the environment of vehicle 100 operating in autonomous mode. The state of the environment can include parameters of the road on which the vehicle operates. For example, computer vision system 140 is capable of identifying a ramp (gradient) or other features based on multiple images of the road. Additionally, the combination of features identified by GPS 122 and computer vision system 140 can be used together with map data stored in data store 114 to determine specific road parameters. Additionally, radar unit 126 can also provide information about the environment surrounding the vehicle.

[0074] In other words, the combination of various sensors (which can be referred to as input indication and output indication sensors) and computer system 112 can interact to provide an indication of the input provided to control the vehicle or an indication of the environment surrounding the vehicle.

[0075] In some embodiments, computer system 112 can make determinations about various objects based on data provided by systems other than the radio system. For example, vehicle 100 can have a laser or other optical sensor configured to sense objects in the field of view of the vehicle. Computer system 112 can use the outputs from various sensors to determine information about the objects in the field of view of the vehicle, and can determine distance and direction information to various objects. Computer system 112 can also determine whether an object is desired or not based on the outputs from various sensors. Additionally, vehicle 100 can also include a telematics control unit (TCU) 160. TCU 160 can enable vehicle connectivity and internal passenger device connectivity through one or more wireless technologies.

[0076] Although Figure 1 the various components of vehicle 100 (i.e., wireless communication system 146, computer system 112, data store 114, and user interface 116) are shown as being integrated into vehicle 100, one or more of these components can be installed or associated separately from vehicle 100. For example, data store 114 can exist partially or entirely separately from vehicle 100. Thus, vehicle 100 can be provided in the form of device elements that can be separately or jointly located. The device elements constituting vehicle 100 can be communicatively coupled together in a wired and / or wireless manner.

[0077] Figure 2A , Figure 2B , Figure 2C , Figure 2D and Figure 2EShows different views of the physical configuration of vehicle 100. Various views are included to depict example sensor locations 202, 204, 206, 208, 210 on vehicle 100. In other examples, the sensors may have different locations on vehicle 100. Although vehicle 100 is depicted as a van in Figures 2A - 2E it may have other configurations within the example, such as a truck, car, semi - truck, motorcycle, bus, shuttle, golf cart, off - road vehicle, robotic device, or agricultural vehicle, and other possible examples.

[0078] As discussed above, vehicle 100 may include sensors coupled at various external locations, such as sensor locations 202 - 210. The vehicle sensors include one or more types of sensors, each configured to capture information from the surrounding environment or perform other operations (e.g., communication link, obtain global positioning information). For example, sensor locations 202 - 210 may serve as locations for any combination of one or more cameras, radar units, LIDAR units, rangefinders, radio devices (e.g., Bluetooth and / or 802.11), and acoustic sensors, and other possible types of sensors.

[0079] When coupled at the Figures 2A - 2E example sensor locations 202 - 210 shown, various mechanical fasteners may be used, including permanent or non - permanent fasteners. For example, bolts, screws, clips, latches, rivets, anchors, and other types of fasteners may be used. In some examples, adhesives may be used to couple the sensors to the vehicle. In further examples, the sensors may be designed and built as part of a vehicle component (e.g., part of a vehicle mirror).

[0080] In some embodiments, one or more sensors may be positioned at sensor locations 202 - 210 using a movable base operable to adjust the orientation of one or more sensors. The movable base may include a rotating platform that can rotate the sensor to obtain information from multiple directions around vehicle 100. For example, a sensor located at sensor location 202 may use a movable base that enables rotation and scanning within a specific range of angles and / or azimuths. Thus, vehicle 100 may include a mechanical structure that enables one or more sensors to be mounted on the top of the vehicle 100's roof. Additionally, in the example, other mounting locations are possible. In some cases, sensors coupled at these locations may provide data that can be used by a remote operator to assist with vehicle 100.

[0081] Figure 3FIG. 0 is a simplified block diagram illustrating a computing device 300, showing some of the components that may be included in a computing device arranged to operate in accordance with embodiments herein. The computing device 300 may be a client device (e.g., a device actively operated by a user (e.g., a remote operator)), a server device (e.g., a device that provides computing services to client devices), or some other type of computing platform. In some embodiments, the computing device 300 may be implemented as a computer system 112 that may be located on a vehicle 100 and perform processing operations related to vehicle operation. For example, the computing device 300 may be used to process sensor data received from a sensor system 104, develop control instructions, enable wireless communication with other devices, and / or perform other operations. Alternatively, the computing device 300 may be located remote from the vehicle 100 and communicate via secure wireless communication. For example, the computing device 300 may operate as a remotely located device that a remote human operator may use to communicate with one or more vehicles.

[0082] In Figure 3 In the example embodiment shown, the computing device 300 includes a processor 302, a memory 304, an input / output unit 306, and a network interface 308, all of which may be coupled via a system bus 310 or a similar mechanism. In some embodiments, the computing device 300 may include other components and / or peripherals (e.g., removable storage, sensors, etc.).

[0083] The processor 302 may be one or more of any type of computer processing element (such as a central processing unit (CPU), a coprocessor (e.g., a math, graphics, or encryption coprocessor), a digital signal processor (DSP), a network processor, and / or in the form of an integrated circuit or controller that performs processor operations). In some cases, the processor 302 may be one or more single-core processors. In other cases, the processor 302 may be one or more multi-core processors having multiple independent processing units. The processor 302 may also include register memory for temporarily storing instructions and associated data being executed, and cache memory for temporarily storing recently used instructions and data.

[0084] The memory 304 can be any form of computer-usable memory, including but not limited to random access memory (RAM), read-only memory (ROM), and non-volatile memory. This can include flash memory, hard disk drives, solid state drives, rewritable compact discs (CDs), rewritable digital video discs (DVDs), and / or tape storage, to name a few. The computing device 300 can include fixed memory as well as one or more removable memory units, the latter including but not limited to various types of secure digital (SD) cards. Thus, the memory 304 can represent both the main memory unit and long-term storage. Other types of memory can include biological memory.

[0085] The memory 304 can store program instructions and / or the data on which the program instructions can operate. By way of example, the memory 304 can store these program instructions on a non-transitory computer-readable medium such that the processor 302 can execute these instructions to perform any of the methods, processes, or operations disclosed in this specification or the drawings.

[0086] As Figure 3 shown, the memory 304 can include firmware 314A, a kernel 314B, and / or applications 314C. The firmware 314A can be program code for booting or otherwise starting some or all of the computing device 300. The kernel 314B can be an operating system, including modules for memory management, scheduling and management of processes, input / output, and communication. The kernel 314B can also include device drivers that allow the operating system to communicate with the hardware modules of the computing device 300 (e.g., memory units, networking interfaces, ports, and buses). The applications 314C can be one or more user-space software programs, such as a web browser or an email client, and any software libraries used by these programs. In some examples, the applications 314C can include one or more neural network applications and other deep learning-based applications. The memory 304 can also store the data used by these and other programs and applications.

[0087] The input / output unit 306 may facilitate the interaction of a user and peripheral devices with the computing device 300 and / or other computing systems. The input / output unit 306 may include one or more types of input devices, such as a keyboard, a mouse, one or more touchscreens, sensors, biometric sensors, and the like. Similarly, the input / output unit 306 may include one or more types of output devices, such as a screen, a monitor, a printer, speakers, and / or one or more light-emitting diodes (LEDs). Additionally or alternatively, for example, the computing device 300 may communicate with other devices using a Universal Serial Bus (USB) or a High-Definition Multimedia Interface (HDMI) port interface. In some examples, the input / output unit 306 may be configured to receive data from other devices. For example, the input / output unit 306 may receive sensor data from vehicle sensors.

[0088] As Figure 3 shown, the input / output unit 306 includes a GUI 312, which may be configured to provide information to a remote operator or another user. The GUI 312 may be one or more display interfaces that can be displayed, or another type of mechanism for communicating information and receiving input. In some examples, the representation of the GUI 312 may vary depending on the vehicle situation. For example, the computing device 300 may provide the GUI 312 in a specific format (such as a format having a single selectable option for the remote operator to select from).

[0089] The network interface 308 may take the form of one or more wired interfaces, such as Ethernet (e.g., Fast Ethernet, Gigabit Ethernet, etc.). The network interface 308 may also support communication over one or more non-Ethernet media (such as coaxial cable or power line) or over wide-area media (such as Synchronous Optical Network (SONET) or Digital Subscriber Line (DSL) technology). The network interface 308 may additionally take the form of one or more wireless interfaces, such as IEEE 802.11 (Wifi), Global Positioning System (GPS), or wide-area wireless interfaces. However, other forms of physical layer interfaces and other types of standard or proprietary communication protocols may be used on the network interface 308. Additionally, the network interface 308 may include multiple physical interfaces. For example, some embodiments of the computing device 300 may include an Ethernet, and a Wifi interface. In some embodiments, the network interface 308 may enable the computing device 300 to connect to one or more vehicles to allow for the remote assistance techniques presented herein.

[0090] In some embodiments, one or more instances of computing device 300 may be deployed to support a clustered architecture. The exact physical location, connectivity, and configuration of these computing devices may be unknown and / or unimportant to client devices. Thus, the computing devices may be referred to as "cloud-based" devices, which may be housed at various remote data center locations. Additionally, computing device 300 may enable the execution of the embodiments described herein, including adverse road condition detection techniques using radar or another type of sensor data.

[0091] Figure 4 is a system for wireless communication between a computing device and a vehicle according to one or more example embodiments. In particular, system 400 is shown to have a vehicle 402, a remote computing device 404, and a server 406 that communicate wirelessly via network 408. System 400 may include other components not shown in other embodiments, such as firewalls and multiple networks, etc.

[0092] Vehicle 402 may be configured to autonomously (or semi-autonomously) transport passengers or objects (e.g., cargo) between locations and may take the form of any one or more of the vehicles discussed above, including passenger vehicles, freight vehicles (e.g., trucks), agricultural and manufacturing vehicles, and dual-use vehicles. When operating in autonomous mode, vehicle 402 may navigate to pick up and drop off passengers (or cargo) between desired destinations by relying on sensor measurements that detect and understand the surrounding environment. In some embodiments, vehicle 402 may operate as part of a fleet that may be managed by a central system (e.g., remote computing device 404 and / or other computing devices).

[0093] Remote computing device 404 may represent any type of one or more devices configured to perform operations including but not limited to those described herein. In an example, the positioning of remote computing device 404 relative to vehicle 402 may vary. For example, remote computing device 404 may have a remote positioning from vehicle 402, such as operating within a physical building. In some implementations, the operations described herein performed by remote computing device 404 may additionally or alternatively be performed by vehicle 402 (i.e., by any system or subsystem of vehicle 200).

[0094] Furthermore, the operations described herein may be performed by any component communicating via network 408. For example, remote computing device 404 may use information from vehicle 402 and / or other external sources (e.g., server 406) to determine a route and / or operations to be performed by vehicle 402. In some embodiments, remote computing device 404 may generate a GUI to display one or more selectable options for a remote operator to view.

[0095] Server 406 can be configured to wirelessly communicate with remote computing device 404 and vehicle 402 (or possibly directly with remote computing device 404 and / or vehicle 402) via network 408. Accordingly, server 406 can represent any computing device configured to receive, store, determine, and / or send information related to vehicle 402 and its remote assistance. Server 406 can be configured to perform any (one or more) operations described herein as being performed by remote computing device 404 and / or vehicle 402 or portions of such (one or more) operations. Some implementations of the wireless communication related to remote assistance can utilize server 406, while other implementations may not use it.

[0096] Network 408 represents an infrastructure that can enable wireless communication between computing devices such as vehicle 402, remote computing device 404, and server 406. For example, network 408 can correspond to a wireless communication network such as the Internet or a cellular wireless communication network. The various systems described above can perform various operations. These operations and related features will now be described.

[0097] In some embodiments, vehicle 402 can communicate with remote computing device 404 and / or server 406 via network 408 to receive and / or provide information related to the adverse road condition detection techniques described herein. For example, vehicle 402 can communicate adverse road conditions detected during navigation to server 406 for storage and subsequent access by other vehicles. Vehicle 402 can also directly communicate information related to adverse road conditions with other vehicles. For example, vehicle 402 can be part of a convoy of other vehicles and share information related to road conditions with other vehicles within the convoy.

[0098] Figure 5A A wheel that generates road spray during forward movement is shown according to one or more example embodiments. Scene 500 is a simplified representation showing wheel 502 traveling forward on road 508 at a speed 510 with a rotational speed 512. As wheel 502 rotates, wheel 502 can cause precipitation (shown as water layer 506) to spray off road 508 in the form of road spray 504. Accordingly, the elements shown in scene 500 can be different from Figure 5A the configuration shown in

[0099] Wheel 502 represents any type of tire or wheel that a vehicle can use for navigation. For example, wheel 502 can be positioned on a car, van, truck, bus, motorcycle, and / or another type of vehicle. When wheel 502 travels forward, the rotation of wheel 502 may cause water from water layer 506 to be lifted and form road spray 504. The size, distance from wheel 502, and shape of road spray 504 can vary based on various factors such as the amount of weight on wheel 502, rotational speed 512, and speed 510. Additionally, the structure and depth of water layer 506 may also affect road spray 504. In some examples, water layer 506 can be snow or slush that also affects road spray 504.

[0100] Figure 5B A side view of a vehicle using a rear-facing radar unit to detect road spray in accordance with one or more example embodiments is shown. In an example embodiment, scene 520 shows vehicle 522 navigating a forward path on road 524 at speed 526, which causes a portion of water layer 534 positioned on road 524 to spray behind vehicle 522 in the form of road spray 532. As a result, vehicle 522 can use radar unit 528 to perform example techniques to detect adverse road conditions on road 524. In other example embodiments, Figure 5B the components shown in may be different. For example, in another example implementation, radar unit 528 may have a different positioning on vehicle 522.

[0101] Radar unit 528 is shown in a rear-facing configuration and coupled to the rear external portion relative to the rear bumper on vehicle 522. At this positioning, radar unit 528 operates with a field of view 530 that extends behind vehicle 522. Field of view 530 represents the elevation, range, and azimuth coverage of radar unit 528. Thus, radar unit 528 can transmit radar beams in the opposite direction of vehicle 522 to increase selectivity and gain in the direction of road spray 532.

[0102] Accordingly, road spray 532 can be physically lifted into the line of sight of radar unit 528, enabling radar unit 528 to sense road spray 532. For example, road spray 532 can have a trail that extends approximately 12 meters behind vehicle 522. The trail can remain contained within the lane of vehicle 522 and can have different dimensions from vehicles entering the area relative to vehicle 522, thereby reducing false positives.

[0103] The radar unit 528 can sense radio frequency (RF) backscattered from rain, snow, and road spray in the atmosphere. The effect from the road spray 532 can produce an elevated response power in a range-Doppler bin, for example, where a droplet resides in a range-Doppler map. Additionally, the radar unit 528 can detect the elevated response power spatially located where the road spray 532 is highly concentrated. In some examples, the computing device can use a power detector to detect the road spray 532.

[0104] Similar to Figure 5A the road spray 504 shown in, the road spray 532 can vary based on various factors during the navigation of the vehicle 522. Example factors can include one or more vehicle-based parameters such as the weight of the vehicle 522, the size and configuration of the tires on the vehicle 522, and the speed 526 at which the vehicle 522 is traveling. Environment-based parameters can also affect the road spray 504, such as the condition of the road 524 (e.g., puddles), and the depth and consistency of the water layer 534 (e.g., puddles or sleet), and so on. Additionally, since the depth and accumulation of the water layer 534 can vary, the road spray 504 can fluctuate along the forward navigation path of the vehicle 522. For example, some portions of the road 524 can be formed in a way that allows water to accumulate, which can produce more road spray 504 when the vehicle 522 drives over these portions.

[0105] Figure 5C A top view of a vehicle using a rear-facing radar unit to detect road spray is shown in accordance with one or more example embodiments. In an example embodiment, the scene 540 shows a top view of the vehicle 522 using the radar unit 528 to detect both the road spray 532A and the road spray 532B. At the center location of the vehicle 522, the radar unit 528 operates using the field of view 530. When the vehicle 522 is navigating a road with adverse conditions (e.g., during rain or snowfall), the rear wheels may lift water on the road and form the road spray 532A and the road spray 532B, respectively. The computing device can use the radar measurements from the radar unit 528 to detect the road spray 532A and the road spray 532B and determine that the vehicle 522 is currently navigating a road with adverse conditions. The computing device can perform one or more of the techniques described herein.

[0106] Figure 5DA top view of a vehicle using multiple radar units to detect road spray is shown in accordance with one or more example embodiments. In an example embodiment, scene 550 shows a vehicle 522 equipped with multiple rear-facing radar units 528A and radar unit 528B. Radar unit 528A operates using a field of view 530A, and radar unit 528B operates using a field of view 530B. A computing device can use the detection of road spray 532A by radar unit 528A and the detection of road spray 532B by radar unit 528B to identify an adverse condition. In some examples, one of radar unit 528A or radar unit 528B can detect road spray, which can indicate that only a portion of the road has precipitation accumulated thereon, such as near a curb or another road boundary. As a result, the computing device can cause vehicle 522 to modify its positioning within the lane to move away from the precipitation and / or adjust lanes to navigate in a different lane with more suitable road conditions for navigation (e.g., a center lane positioned farther from a drainage system on the side of the road).

[0107] Figure 6 A vehicle using forward-facing radar to detect road spray is shown in accordance with one or more example embodiments. In an example embodiment, scene 600 shows vehicle 602 using a forward-facing radar unit 604 to detect road spray 608 lifted by vehicle 606 during forward navigation.

[0108] As vehicle 602 travels along a forward path, a control system can use sensor data to detect and understand the surrounding environment. The vehicle sensor system can include a radar system, which can include one or more radar units (e.g., radar unit 604) positioned relative to the front of vehicle 602 to measure the forward environment. Radar unit 604 represents any type of radar unit within the vehicle radar system. Radar unit 604 is positioned near the front of vehicle 602 and is oriented to have a field of view 610 for capturing measurements of the forward environment of vehicle 602. Radar unit 604 can detect objects such as other vehicles, signs, lane markings, and road boundaries.

[0109] In some instances, radar unit 604 may capture measurements of road spray originating from the wheels of other vehicles. For illustration, scenario 600 shows that the rear wheels of vehicle 606 cause a thin layer of water on the road to be lifted and sprayed as road spray 608 behind vehicle 606. If vehicle 602 travels close enough to vehicle 606, a high level of road spray 608 may enter the field of view 610 of radar unit 604. Accordingly, the vehicle system may perform the operations disclosed herein to detect and quantify the level of road spray present on the path and adjust operations in response. Adjusting operations may involve changing lanes, adjusting speed and following distance, modifying the route, and / or other potential mitigation techniques. In some embodiments, vehicle 606 and vehicle 602 may share information related to the detected adverse condition. For example, vehicle 606 may operate as a lead in a platoon that includes vehicle 602 and may share the detected adverse road condition.

[0110] Figure 7 is a flowchart of a method for using radar to detect adverse road conditions in real time according to one or more example embodiments. Method 700 may include one or more operations, functions, or actions depicted in one or more of blocks 702, 704, 706, 708, 710, 712, and 714, each of which may be performed by any system shown in the previous figures and other possible systems.

[0111] Those skilled in the art will understand that the flowcharts depicted herein illustrate the functionality and operations of certain embodiments of the present disclosure. In this regard, each block of the flowchart may represent a module, segment, or portion of program code that includes one or more instructions executable by one or more processors to implement a particular logical function or step in the process. The program code may be stored on any type of computer-readable medium, such as a storage device including, for example, a magnetic disk or a hard drive.

[0112] Furthermore, each block may represent circuitry wired to perform a particular logical function in the process. Alternative embodiments are within the scope of the example embodiments of the present application, and in alternative embodiments, the functions may be performed in a different order than shown or discussed, including substantially concurrently or in reverse order, depending on the functions involved, as will be understood by those skilled in the art.

[0113] At block 702, method 700 involves generating a first radar representation representing the field of view of the radar unit during clear weather conditions. For example, a computing device may generate a first radar image representing the range and Doppler information of the field of view of the radar unit during clear weather conditions.

[0114] The radar unit can be part of a vehicle radar system. For example, the radar unit can be a rear-facing radar unit coupled to the vehicle such that the field of view covers a portion of the rear environment of the vehicle.

[0115] In some examples, the computing device generating the first radar representation can involve receiving a plurality of radar data sets from the radar unit during navigation of the vehicle in a plurality of environments and determining an energy level for each radar data set of the plurality of radar data sets. The computing device can also determine an average energy level based on the plurality of energy levels corresponding to the plurality of radar data sets and generate the first radar representation based on the average energy level.

[0116] In some instances, the computing device can remove one or more radar data sets having a given energy level that exceeds a threshold energy level based on determining the energy level of each radar data set. The threshold energy level can be based on a specific energy level associated with the detection of an object. Such detections can result in high radar returns that exceed the threshold energy level and can make it difficult to detect road spray. Accordingly, these data sets can be removed and additional sets can be used to detect road spray. In some examples, the computing device can then determine the average energy level based on removing the one or more radar data sets. The average energy level can then be determined across a set of radar data measurements that do not have high-power returns caused by objects detected near the vehicle.

[0117] In block 704, method 700 involves storing the first radar representation in a memory. The computing device can store the first radar representation in a local memory that is onboard and located on the vehicle. In other examples, the computing device can store the first radar representation remotely. Storing the first radar representation can involve different types of storage techniques and can also involve multiple copies of the stored first radar representation.

[0118] In block 706, method 700 involves receiving radar data from the radar unit during navigation of the vehicle on a road. As the vehicle navigates a path on various types of roads and paths, the vehicle radar system can accumulate radar measurements. In some cases, multiple radar units can capture measurements of the vehicle's surrounding environment.

[0119] In block 708, method 700 involves determining a second radar representation based on the radar data. For example, the computing device can determine a second radar image representing range and Doppler information of the field of view of the radar unit based on the radar data. Some examples may involve using a 2D range-azimuth image obtained via the radar data. Additionally, some embodiments involve processing radar returns in the form of a radar data cube. In particular, the power levels of the voxel representations within the radar data cube can be used to detect levels that may indicate the presence of road spray.

[0120] At block 710, method 700 involves performing a comparison between a first radar representation and a second radar representation. For example, a computing device may perform a comparison between a first energy from the first radar representation and a second energy from the second radar representation. The computing device may determine, based on the comparison, that the second energy from the second radar representation is at least a threshold energy level greater than the first energy from the first radar representation. In some examples, the computing device may filter the first energy from the first radar representation and the second energy from the second radar representation.

[0121] At block 712, method 700 involves determining a road condition of a road based on the comparison. The road condition may indicate a precipitation amount located on the road. In some examples, the computing device may determine the road condition of the road based on determining that the second energy from the second radar representation is at least a threshold energy level greater than the first energy from the first radar representation. In some examples, the computing device may factor in other parameters when determining the road condition. For example, the computing device may factor in a map representing road conditions from other vehicles, weather forecasts, and / or other sensor measurements.

[0122] At block 714, method 700 involves providing control instructions to a vehicle. For example, the computing device may determine that the precipitation amount exceeds a threshold precipitation level and, in response to determining that the precipitation amount exceeds the threshold precipitation level, adjust the path of the vehicle. This may involve adjusting the driving lane of the vehicle or shifting the vehicle towards the side of the lane and other potential operations.

[0123] In some examples, the computing device may adjust a threshold associated with one or more sensors based on detecting road spray. For example, when the sensing power from a rear radar road spray detector exceeds a nominal value, the detection threshold for a certain number of short-range bins may be adjusted to reduce false alarm tracking generated by road spray from closely approaching vehicles.

[0124] In some examples, method 700 may further involve modifying a map based on the road condition of the road. Based on modifying the map, the computing device may provide the modified map to at least a remote computing device. For example, the remote computing device may be configured to provide the modified map to one or more additional vehicles navigating in a path involving the use of the road.

[0125] In some examples, a computing device may receive first radar data from a first radar unit and second radar data from a second radar unit during vehicle navigation. The first radar unit and the second radar unit may be rear-facing radar units configured to measure respective portions of the environment located behind the vehicle. Thus, the computing device may determine a second radar representation based on the first radar data and a third radar representation based on the second radar data. As a result, the computing device may also perform a first comparison between the first radar representation and the second radar representation, and a second comparison between the first radar representation and the third radar representation. The computing device may determine the road condition of the road based on both the first comparison and the second comparison. For example, the computing device may determine that a precipitation layer is located on a portion of the road positioned close to the road boundary (e.g., along a drainage path positioned at the curb). The computing device may cause the vehicle to shift the position on the road away from the precipitation layer that is located on the portion of the road positioned close to the road boundary.

[0126] In some embodiments, other types of sensors may be used. For example, the computing device may use LIDAR to detect road spray. The positioning and operation of the radar units may improve the accuracy of detecting road spray. Additionally, the computing device may also update a map communicating an adverse road condition and share the map communicating the adverse road condition with other vehicles positioned at a threshold distance from the vehicle and / or at a road having an adverse road condition (e.g., within 2 miles of the vehicle and / or a road having an adverse condition).

[0127] In some examples, one or more power ranges may be used to represent the power returns obtained when there is an adverse road condition. For example, when the current radar return of a radar unit shows a power detection that falls within the power range associated with an adverse road condition, the vehicle system may determine that the current road condition is adverse and modify the control of the vehicle to accommodate these conditions. Additionally, when using power ranges, the power returns caused by reflective objects may exceed the power range associated with an adverse road condition.

[0128] In some examples, trends may be determined over time by accumulating adverse road conditions from multiple vehicles for a general location (e.g., a city or town) or a specific stretch of road (e.g., a highway). These trends showing which roads experience more adverse conditions may help indicate which roads may be vulnerable to adverse conditions for various reasons. In certain examples, the trends may help repair roads that tend to be more susceptible to water accumulation than other roads located nearby.

[0129] Figure 8FIG. 0 is a schematic diagram showing a conceptual partial view of an example computer program product arranged in accordance with at least some embodiments presented herein, the example computer program product including a computer program for performing a computer process on a computing device. In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a non-transitory computer-readable storage medium or encoded on other non-transitory media or articles in a machine-readable format.

[0130] In one embodiment, an example computer program product 800 is provided using a signal bearing medium 802, which may include one or more programming instructions 804 that, when executed by one or more processors, may provide the functions or portions of the functions described above with respect to Figures 1 - 7 those described.

[0131] In some examples, the signal bearing medium 802 may encompass a non-transitory computer-readable medium 806, such as but not limited to a hard disk drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, and the like. In some implementations, the signal bearing medium 802 may include a computer recordable medium 808, such as but not limited to a memory, a read / write (R / W) CD, an R / W DVD, and the like. In some implementations, the signal bearing medium 802 may encompass a communication medium 810, such as but not limited to digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Thus, for example, the signal bearing medium 802 may be conveyed by a wireless form of the communication medium 810.

[0132] The one or more programming instructions 804 may be, for example, computer-executable and / or logic-implemented instructions. In some examples, a computing device such as Figure 1 computer system 112 may be configured to provide various operations, functions, or actions in response to the programming instructions 804 communicated to the computer system 112 by one or more of the computer-readable medium 806, the computer recordable medium 808, and / or the communication medium 810. Other devices may perform the operations, functions, or actions described herein.

[0133] The non-transitory computer-readable medium may also be distributed among a plurality of data storage elements, which may be located remotely from each other. The computing device that executes some or all of the stored instructions may be a vehicle, such as Figures 1 - 2E vehicle 100 shown in

[0134] The various features and functions of the disclosed systems, devices, and methods have been described in detail above with reference to the accompanying drawings. Although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent. The various aspects and embodiments disclosed herein are for illustrative purposes and are not intended to be limiting, and the true scope is indicated by the appended claims.

[0135] It should be understood that the arrangements described herein are for illustrative purposes only. Accordingly, those skilled in the art will appreciate that alternative arrangements and other elements (e.g., machines, devices, interfaces, functions, orders, and groupings of functions, etc.) may be used instead, and some elements may be entirely omitted depending on the desired results. In addition, many of the elements described are functional entities that may be implemented as discrete or distributed components or combined with other components in any suitable combination and location.

Claims

1. A method for detecting road conditions, comprising: at a computing device coupled to a vehicle, generating a first radar representation representing the field of view of a radar unit based on first radar data obtained by the radar unit during navigation of the vehicle on at least a first road under clear weather conditions, wherein the radar unit is coupled to an outer surface of the vehicle, faces rearward, and is positioned such that the radar unit can detect precipitation lifted by one or more wheels of the vehicle; storing the first radar representation in a memory; at the computing device, during subsequent navigation of the vehicle on a second road, receiving second radar data from the radar unit; determining, by the computing device, a second radar representation based on the second radar data; performing, by the computing device, a comparison between the first radar representation and the second radar representation, wherein the computing device obtains the first radar representation from the memory to perform the comparison; based on the comparison, determining whether the precipitation is lifted by one or more wheels of the vehicle and thereby determining the road condition of the second road, wherein the road condition represents the amount of precipitation located on the second road; and by the computing device, based on the road condition of the second road, adjusting the constant false alarm rate (CFAR) detection threshold used when processing radar data received from a forward-facing radar unit coupled to the vehicle, and the speed and following distance maintained by the vehicle when another vehicle is traveling in front of the vehicle on the second road.

2. The method according to claim 1, wherein Generating the first radar representation includes: receiving, during navigation of the vehicle in multiple environments, multiple radar data sets from the radar unit; determining the energy level of each radar data set from the multiple radar data sets; determining an average energy level based on the multiple energy levels corresponding to the multiple radar data sets; and generating the first radar representation based on the average energy level.

3. The method according to claim 2, further comprising: based on determining the energy level of each radar data set, removing one or more radar data sets having a given energy level exceeding a threshold energy level, wherein the threshold energy level is based on a specific energy level associated with the detection of an object; and wherein determining the average energy level includes: determining the average energy level based on removing the one or more radar data sets.

4. The method according to claim 1, wherein, Generating the first radar representation includes: generating a first radar image representing the range and Doppler information of the field of view of the radar unit during clear weather conditions.

5. The method according to claim 4, wherein, Determining the second radar representation based on the second radar data includes: determining, based on the second radar data, a second radar image representing the range and Doppler information of the field of view of the radar unit.

6. The method according to claim 5, wherein Performing the comparison between the first radar representation and the second radar representation includes: performing a comparison between a first energy from the first radar representation and a second energy from the second radar representation; and based on the comparison, determining that the second energy from the second radar representation is at least a threshold energy level greater than the first energy from the first radar representation.

7. The method according to claim 6, wherein, Performing the comparison between the first energy from the first radar representation and the second energy from the second radar representation includes: filtering the first energy from the first radar representation and the second energy from the second radar representation; and wherein determining the road condition of the second road includes: Determine the road condition of a second road based on determining that second energy from a second radar representation is at least a threshold energy level greater than first energy from a first radar representation.

8. The method according to claim 1, wherein, A field of view of the radar unit covers a portion of a rear environment of the vehicle.

9. The method according to claim 1, wherein Storing the first radar representation in a memory includes: Storing the first radar representation in a local memory located on the vehicle.

10. The method according to claim 1, wherein, Providing control instructions to the vehicle includes: Determining that a precipitation amount exceeds a threshold precipitation level; and Adjusting a path of the vehicle based on determining that the precipitation amount exceeds the threshold precipitation level.

11. The method according to claim 10, wherein, Adjusting the path of the vehicle includes: Adjusting a driving lane of the vehicle.

12. The method according to claim 1, further comprising: Modifying a map based on the road condition of the second road; And Providing the modified map to at least a remote computing device, wherein the remote computing device is configured to provide the modified map to one or more additional vehicles navigating on a path including the second road.

13. The method according to claim 1, wherein, Receiving second radar data from the radar unit during a subsequent navigation of the vehicle on the second road includes: During a subsequent navigation of the vehicle, receiving first additional radar data from a first radar unit and second additional radar data from a second radar unit, wherein the first radar unit and the second radar unit are rear-facing radar units configured to measure respective portions of an environment located behind the vehicle; Determining a second radar representation based on the first additional radar data and a third radar representation determined based on the second additional radar data; and Wherein performing a comparison between the first radar representation and the second radar representation includes: Performing a first comparison between the first radar representation and the second radar representation; and Performing a second comparison between the first radar representation and the third radar representation.

14. The method according to claim 13, wherein, Determining the road condition of the second road includes: Determining the road condition of the second road based on both the first comparison and the second comparison.

15. The method according to claim 14, wherein, Determining the road condition of the second road based on both the first comparison and the second comparison includes: Determining that a precipitation layer is located on a portion of the second road that is positioned close to a road boundary; and Wherein providing control instructions to the vehicle includes: Causing the vehicle to shift a position on the second road away from the precipitation layer located on the portion of the second road that is positioned close to the road boundary.

16. The method according to claim 1, further comprising: Adjusting a power level of the radar unit based on the road condition of the second road.

17. A system for detecting a road condition, comprising: A radar unit coupled to a vehicle, wherein the radar unit faces rearward and is positioned such that the radar unit can detect precipitation lifted by one or more wheels of the vehicle; and A computing device configured to: Generate a first radar representation representing a field of view of the radar unit based on first radar data obtained by the radar unit during navigation of the vehicle on at least a first road under clear weather conditions; Store the first radar representation in a memory; Receive second radar data from the radar unit during a subsequent navigation of the vehicle on a second road; Determine a second radar representation based on the second radar data; A comparison is performed between a first radar representation and a second radar representation, wherein the computing device obtains the first radar representation from a memory to perform the comparison; Based on the comparison, determine whether precipitation is being kicked up by one or more wheels of the vehicle, and thereby determine the road condition of a second road, wherein the road condition represents the amount of precipitation located on the second road; and Based on the road condition of the second road, adjust the constant false alarm rate (CFAR) detection threshold used when processing radar data received from a forward-facing radar unit coupled to the vehicle, and the speed and following distance maintained by the vehicle when another vehicle is traveling in front of the vehicle on the second road.

18. The system according to claim 17, wherein The computing device is further configured to: Also determine the road condition of the second road based on the speed of the vehicle.

19. The system according to claim 17, wherein, The computing device is further configured to: Update a map to include an indication representing the road condition of the second road; and Transmit the map to one or more vehicles having a given location within a threshold distance of the second road.

20. A non-transitory computer-readable medium configured to store instructions that, when executed by a computing system including one or more processors, cause the computing system to perform operations including the following: Generate a first radar representation representing the field of view of a radar unit based on first radar data obtained by the radar unit during navigation of a vehicle on at least a first road under clear weather conditions, wherein, The radar unit is coupled to an outer surface of the vehicle, faces rearward, and is positioned such that the radar unit can detect precipitation kicked up by one or more wheels of the vehicle; Store a first radar representation in a memory; During subsequent navigation of the vehicle on the second road, receive second radar data from the radar unit; Determine a second radar representation based on the second radar data; Perform a comparison between the first radar representation and the second radar representation, wherein the first radar representation is obtained to perform the comparison; Based on the comparison, determine whether precipitation is being kicked up by one or more wheels of the vehicle, and thereby determine the road condition of the second road, wherein the road condition represents the amount of precipitation located on the second road; and Based on the road condition of the second road, adjust the constant false alarm rate (CFAR) detection threshold used when processing radar data received from a forward-facing radar unit coupled to the vehicle, and the speed and following distance maintained by the vehicle when another vehicle is traveling in front of the vehicle on the second road.

Images