Estimation of Friction of Vehicle Driving Surface

Abstract

Systems and methods are provided for generating data indicative of friction associated with a driving surface and for using the friction data in association with one or more vehicles. In one example, a computing system may detect a stop associated with a vehicle and initiate a steering action of the vehicle during the stop. The steering action is associated with movement of at least one tire of the vehicle relative to a driving surface. The computing system may obtain operational data associated with the steering action during the stop of the vehicle. The computing system may determine friction associated with the driving surface based at least in part on the operational data associated with the steering action. The computing system may generate data indicative of the friction associated with the driving surface.

Description

[0001] Priority claim

[0002] This application is based upon and claims the benefit of U.S. Application No. 16 / 131,727, filed on September 14, 2018, and U.S. Application No. 16 / 131,743, filed on September 14, 2018, both of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to improving the ability of computing systems to determine information about the external environment and make predictions, such as enabling a vehicle to determine information about the surrounding environment and provide data indicative of the information. Background Art

[0004] Vehicles, including non-autonomous vehicles and autonomous vehicles, increasingly include one or more computing systems. An autonomous vehicle is a vehicle that is capable of sensing its environment and navigating without human input. In particular, an autonomous vehicle may observe its surroundings using various sensors and attempt to understand the environment by performing various processing techniques on the data collected by the sensors. With knowledge of its surroundings, an autonomous vehicle may identify an appropriate path of motion for navigating through such surroundings. Similarly, a non-autonomous vehicle may include sensors for observing the external environment. Summary of the invention

[0005] Aspects and advantages of embodiments of the invention will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0006] One example aspect of the present invention relates to a computing system. The computing system includes one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations include: detecting an event including at least one of acceleration, deceleration, or stopping associated with an autonomous vehicle; in response to detecting the event, obtaining operational data associated with the autonomous vehicle during the event; determining data indicating friction associated with a surface on which the autonomous vehicle is traveling during the event based at least in part on the operational data; and controlling the autonomous vehicle based at least in part on the data indicating the friction associated with the surface.

[0007] Another example aspect of the invention relates to an autonomous vehicle comprising one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the autonomous vehicle to perform operations. The operations include: detecting an event including at least one of acceleration, deceleration, or stopping associated with the autonomous vehicle; in response to detecting the event, obtaining operational data associated with the autonomous vehicle during the event; determining data indicative of friction associated with a surface on which the vehicle is traveling during the event based at least in part on the operational data; and generating at least one constraint for the autonomous vehicle based at least in part on the data indicative of the friction associated with the surface.

[0008] Yet another example aspect of the present invention relates to a computer-implemented method comprising: detecting an event including at least one of acceleration, deceleration, or stopping associated with an autonomous vehicle; in response to detecting the event, obtaining operational data associated with the autonomous vehicle during the event; determining data indicating friction associated with a surface on which the autonomous vehicle is traveling during the event based at least in part on the operational data; and controlling the autonomous vehicle based at least in part on the data indicating the friction associated with the surface.

[0009] Yet another example aspect of the present invention relates to a computing system. The computing system includes one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations include detecting a stop associated with a vehicle and initiating a steering action of the vehicle during the stop. The steering action is associated with movement of at least one tire of the vehicle relative to a driving surface. The operations include: obtaining operational data associated with the steering action during the stop of the vehicle; determining friction associated with the driving surface based at least in part on the operational data associated with the steering action; and generating data indicative of the friction associated with the driving surface.

[0010] Yet another example aspect of the present invention relates to a vehicle comprising one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause an autonomous vehicle to perform operations. The operations include detecting a stop associated with the vehicle and initiating a steering action of the vehicle during the stop. The steering action is associated with movement of at least one tire of the vehicle relative to a driving surface. The operations include: obtaining operational data associated with the steering action during the stop of the vehicle; determining friction associated with the driving surface based at least in part on the operational data associated with the steering action; and generating data indicative of the friction associated with the driving surface.

[0011] Yet another example aspect of the invention relates to a computer-implemented method comprising detecting a stop associated with a vehicle and initiating a steering action of the vehicle during the stop. The steering action is associated with movement of at least one tire of the vehicle relative to a driving surface. The operations include: obtaining operational data associated with the steering action during the stop of the vehicle; determining friction associated with the driving surface based at least in part on the operational data associated with the steering action; and generating data indicative of the friction associated with the driving surface.

[0012] Other example aspects of the invention relate to systems, methods, vehicles, apparatus, tangible, non-transitory computer-readable media, and memory devices for determining friction associated with a driving surface on which a vehicle is traveling.

[0013] These and other features, aspects, and advantages of various embodiments will become more understood with reference to the following description and accompanying technical solutions.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present invention and together with the description, are used to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] A detailed discussion of embodiments for those of ordinary skill in the art is set forth in the specification with reference to the accompanying drawings, in which:

[0015] Figure 1 is a block diagram of an example autonomous vehicle according to an example embodiment of the present invention;

[0016] Figure 2 A flowchart depicting an example process for determining a friction estimate and controlling a vehicle using the friction estimate according to an example embodiment of the invention;

[0017] Figure 3 Depicting an example environment through which a vehicle according to an example embodiment of the present invention may travel;

[0018] Figure 4 Depicting an example environment through which a vehicle according to an example embodiment of the present invention may travel;

[0019] Figure 5 A block diagram depicting an example vehicle computing system and steering system according to an example embodiment of the present invention;

[0020] Figure 6 depicts a graph illustrating the relationship between steering angle and lateral displacement of tire bristles according to an example embodiment of the present invention;

[0021] Figure 7 depicts a graph illustrating the relationship between steering angle and lateral displacement of tire bristles including saturation conditions according to an example embodiment of the present invention;

[0022] Figure 8 depicts a graph illustrating the relationship between steering angle and alignment torque according to an example embodiment of the present invention;

[0023] Fig. 9 depicts a graph illustrating an example of an input steering torque request and a resulting rack force associated with a steering system of an autonomous vehicle according to an example embodiment of the present invention;

[0024] Fig.10 depicts a graph illustrating the relationship between steering angle and rack force, and a technique for determining a maximum rack force and a maximum steering angle, according to an example embodiment of the present invention;

[0025] Fig.11 A flowchart depicting an example process for determining a friction force estimate for a vehicle using one or more steering-based wheel motions according to an example embodiment of the present invention;

[0026] Fig.12 A block diagram depicting an example vehicle computing system and braking / propulsion system according to an example embodiment of the present invention;

[0027] Fig.13 A flowchart depicting an example process for determining a friction force estimate for a vehicle using one or more braking-based wheel actions according to an example embodiment of the present invention;

[0028] Fig.14 A flowchart depicting an example process for determining a friction force estimate for a vehicle using one or more propulsion-based wheel motions according to an example embodiment of the present invention; and

[0029] Fig.15 Depicted is a block diagram of an example computing system according to an example embodiment of the invention. DETAILED DESCRIPTION

[0030] Reference will now be made in detail to the embodiments, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the embodiments and not by way of limitation of the invention. Indeed, it will be appreciated by those skilled in the art that various modifications and variations may be made to the embodiments without departing from the scope or spirit of the invention. For example, a feature illustrated or described as part of one embodiment may be used together with another embodiment to produce yet another embodiment. Therefore, it is desired that aspects of the invention encompass such modifications and variations.

[0031] In general, the present invention relates to systems and methods for enabling a vehicle, such as an autonomous or non-autonomous vehicle, to automatically estimate the friction associated with a driving surface on which the vehicle is traveling, and using the friction estimate as part of the vehicle operation to provide an indication of the friction estimate associated with the vehicle and / or to transmit the friction estimate to other vehicles or computing systems. For example, the systems and methods of the present invention enable a vehicle to utilize operational data associated with the vehicle during a particular event in order to make an estimate of the friction associated with the driving surface. More specifically, in some embodiments, a vehicle computing system associated with the vehicle may obtain operational data associated with the vehicle during an event (e.g., when the vehicle decreases speed as it approaches a stop sign, stops at a stop sign, increases speed as the vehicle leaves an intersection, or increases angular acceleration in response to a change in steering angle). The vehicle computing system may use the operational data to determine data indicating the friction associated with the driving surface. The vehicle computing system may provide an indication of the friction force estimate within the vehicle, such as to a driver of the vehicle, and / or transmit the friction force estimate to other vehicles or computing systems. According to some aspects, the vehicle may be an autonomous vehicle, and the vehicle computing system may control the autonomous vehicle based at least in part on data indicating friction force.

[0032] According to an example embodiment of the present invention, data indicating friction associated with a driving surface may be generated based at least in part on operational data associated with a steering action of a vehicle, and friction data associated with vehicle operation may be used. In one example, a computing system associated with a vehicle may detect a stop associated with the vehicle and initiate a steering action of the vehicle during the stop. The steering action is associated with movement of at least one tire of the vehicle relative to a driving surface. During the stop of the vehicle, operational data associated with the steering action may be obtained. Friction associated with the driving surface may be determined based at least in part on the operational data associated with the steering action. The computing system may generate data indicating the friction associated with the driving surface.

[0033] According to example embodiments of the present invention, a vehicle may generate a friction force estimate while minimizing and / or eliminating interference with vehicle passengers. By way of example, a vehicle may detect an event during which a friction force estimate may be determined without causing significant changes that are perceptible to passengers of normal vehicle operation. These events coincide with geographic areas and / or environments in which vehicles typically reduce speed, stop, increase speed, etc. For example, a vehicle may detect that one or more events including acceleration, deceleration, or stopping of the vehicle are likely to occur at an upcoming stop light, stop sign, yield zone, etc., and determine the friction force associated with the driving surface during these detected events. Events may include acceleration events, deceleration events, and / or stop events. Such events may be associated with an increase in speed, a decrease in speed, or a stop in movement performed by the vehicle. The increase or decrease in speed may be in the forward direction of travel of the vehicle, in a direction transverse to the direction of travel, or in an angular direction that may be caused, for example, by a steering movement. Therefore, an event may include at least one of an increase in speed, a decrease in speed, or a stop of the vehicle. In some examples, the event may include a single event including a speed increase, a speed decrease, or a stop. In other examples, the event may include multiple events, such as multiple stops, speed increases, or speed decreases. In yet another example, the event may include a combination of events including two or more different event types, such as a stop and a subsequent speed increase, or a speed decrease and a subsequent stop. The event may include a speed increase or a speed decrease in the direction of travel of the vehicle, in a direction transverse to the direction of travel of the vehicle, or in an angular direction. By way of example, it may be detected that the vehicle as part of the event stops or is traveling at a low speed (e.g., when the vehicle approaches a stop sign). In response, the vehicle control system may cause the steering system of the vehicle to be manipulated. Then, the vehicle control system may determine the resulting friction between one or more tires of the vehicle and the driving surface. In another example, the braking system and / or propulsion system of the vehicle may be manipulated during the event, and the resulting friction between one or more tires of the vehicle and the road surface may be determined.

[0034] More specifically, in some embodiments, a vehicle computing system associated with a vehicle may initiate wheel-based actions during events including vehicle speed increase, vehicle speed reduction, and / or stop, and utilize the resulting operational data to determine friction. In some instances, wheel-based actions may be initiated to minimize vehicle actions that are perceptible to vehicle passengers. Typically, wheel-based actions are associated with the movement of at least one tire of the vehicle relative to the travel surface. By way of example, wheel-based actions may be associated with braking, acceleration, or steering, etc. Resulting operational data based on the action of the steering wheel may include one or more of the input torque applied during steering, the steering force caused by the input torque, and / or the wheel rotation caused by the input torque. Resulting operational data based on the action of the brake wheel may include one or more of the input braking force, the wheel torque caused by the input braking force, and / or the wheel rotation caused by the braking force. Resulting operational data based on the action of the speed increase or based on the action of the propulsion wheel may include one or more of the input propulsion force, the wheel torque associated with the propulsion force, and / or the wheel rotation associated with the propulsion force.

[0035] More specifically, in some embodiments, the vehicle is an autonomous vehicle configured to automatically estimate the friction associated with the driving surface and use the friction estimate as part of the autonomous operation. The autonomous vehicle may be a land-based autonomous vehicle (e.g., a car, truck, bus, etc.) or other types of vehicles. The autonomous vehicle may include a computing system that helps control the autonomous vehicle. In some embodiments, the autonomous vehicle computing system may include a perception system, a prediction system, a motion planning system, and a friction estimation system that cooperate to perceive the surrounding environment of the autonomous vehicle and determine one or more motion plans for controlling the motion of the autonomous vehicle accordingly. The autonomous vehicle computing system may include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the autonomous vehicle computing system to perform various operations described herein.

[0036] In some embodiments, the operational data is associated with one or more sensors of the vehicle. Specifically, in some embodiments, the perception system and / or the friction force estimation system may receive sensor data from one or more sensors coupled to the vehicle or otherwise included in the vehicle. As an example, the one or more sensors may include a light detection and ranging (LIDAR) system, a radio detection and ranging (RADAR) system, one or more cameras (such as visible spectrum cameras, infrared cameras, etc.), and / or other sensors. Sensor data from such sensors may include information describing the position of an object in the surrounding environment of the vehicle. Sensor data may be used to generate detection, classification, and / or prediction by other systems such as one or more machine learning models or vehicle computing systems of autonomous vehicles. The vehicle may also or alternatively include one or more other types of sensors configured to obtain other types of sensor data. For example, the vehicle may include an inertial measurement unit (IMU) sensor, a pressure sensor, a power sensor, a contact sheet sensor, a suspension travel sensor, a laser sensor, a radar sensor, a sound sensor, and / or any other suitable sensor. Thus, sensor data acquired by a vehicle may also or instead include information describing the position, movement (e.g., velocity, speed, acceleration, etc.), and / or forces experienced by the vehicle and / or its individual components (e.g., tires, axles, drivetrain, brakes, steering system, etc.). Additionally, the sensor data may be used to determine data indicative of the friction of a driving surface, as further described herein.

[0037] Operational data may be generated during events associated with speed increase, speed decrease, and / or stop, and may include sensor data and / or data derived from sensor data. Sensor data associated with one or more wheel-based actions initiated in response to detected events may be generated. By way of example, the sensor data may include at least one of input torque, wheel rotation, or steering force detected using one or more sensors associated with steering wheel-based actions. As another example, the sensor data may include at least one of braking force, propulsion force, wheel rotation, or wheel torque detected using one or more sensors associated with braking wheel-based actions and / or propulsion wheel-based actions.

[0038] According to an example embodiment, an action based on a steering wheel may be performed during a stop, and the associated input torque, resulting wheel rotation, and / or resulting wheel torque is determined. The input torque, resulting wheel rotation, and / or resulting wheel torque may be determined at least in part based on the output of one or more sensors during an action based on a steering wheel. The vehicle computing system may determine friction using at least one of the input torque, wheel rotation, and wheel torque. More specifically, the vehicle computing system may first detect an event, such as the stop of the current vehicle or the stop of a vehicle planned in the future. When the vehicle stops, the vehicle computing system may start applying an input torque at the steering system to cause a rotational force between one or more tires of the vehicle and the driving surface. The input torque may be applied when the vehicle stops in order to avoid being perceived by the vehicle passenger. In other examples, the input torque may be applied when the vehicle is moving, but in a manner that reduces the perception of the torque by the passenger. For example, a quick steering rotation in one direction and recovery in the opposite direction may be applied using a short move so that the passenger may not notice the steering movement. In some examples, the input torque may be associated with a change (e.g., a sudden change) in vortices associated with steering (e.g., a second derivative of the steering direction). In some examples, the input torque may increase during application, and the resulting wheel rotation and steering force (e.g., rack force of the steering system) may be determined directly or indirectly using one or more sensors. Friction may be determined by the vehicle control system based on the input torque, wheel rotation, and / or steering force. In some examples, the input torque is an alignment torque representing torque about an imaginable vertical axis through the wheel hub. In some examples, the input torque includes one or more of a driver input to the steering system and a motor drive input.

[0039] More specifically, in some examples, the maximum steering force and / or the maximum wheel rotation may be used to determine the friction between one or more tires and the driving surface. For example, the vehicle control system may analyze the change in the resulting steering force relative to the change in the resulting wheel rotation in response to an increase in input torque. Based on this analysis, at least one of the maximum steering force or the maximum wheel rotation may be determined. The vehicle computing system may then calculate the friction based on the maximum steering force and / or the maximum wheel rotation.

[0040] As another example, a brake wheel-based action may be performed during an acceleration event that includes a speed reduction, and an associated input braking force, resulting wheel rotation, and / or resulting wheel torque is determined based at least in part on output of one or more sensors during the brake wheel-based action. The vehicle computing system may determine the friction force using at least one of the braking force, wheel rotation, and wheel torque.

[0041] As part of the action based on braking wheels, one or more input braking forces may be applied to one or more wheel assemblies. In some instances, a single braking force is applied to one wheel assembly. In other instances, a single braking force is applied to multiple wheel assemblies. In yet other instances, multiple braking forces may be applied. More specifically, in some instances, the vehicle computing system may utilize differential braking forces between the wheel assemblies of the vehicle during deceleration in order to determine friction while minimizing interference to passengers. For example, the vehicle computing system may detect acceleration events that include a current or future reduction in vehicle speed. As part of decelerating the vehicle, the vehicle computing system may initiate the application of different braking forces to at least two or more wheel assemblies. For example, the vehicle computing system may apply a larger braking force to one wheel in order to cause the corresponding tire to slip or slide relative to the driving surface. The larger braking force may increase over time until the slip or slide of the tire is detected. Smaller braking forces may be applied to other wheels so that no slip or slide is experienced. In this way, passengers may not perceive the slip or slide associated with the larger braking force. The resulting wheel rotation or wheel torque associated with the wheel assembly receiving a greater braking force may be determined directly or indirectly using one or more sensors. Friction may be determined by the vehicle control system based on the greater braking force, wheel rotation, and / or wheel torque.

[0042] As yet another specific example, a propulsion wheel-based action may be performed during an event, and an associated input propulsion force, resulting wheel rotation, and / or resulting wheel torque is determined based at least in part on output of one or more sensors during the propulsion wheel-based action. The vehicle computing system may determine the friction force using at least one of the propulsion force, wheel rotation, or wheel torque.

[0043] As part of the action based on the propulsion wheel, one or more input propulsion forces may be applied to one or more wheel assemblies. In some instances, a single propulsion force is applied to one wheel assembly. In other instances, a single propulsion force is applied to multiple wheel assemblies. In yet other instances, multiple propulsion forces may be applied. More specifically, in some instances, the vehicle computing system may utilize differential propulsion forces between the wheel assemblies of the vehicle during acceleration in order to determine friction while minimizing interference with passengers. For example, the vehicle computing system may detect events that include current or future increases in the speed of the autonomous vehicle. As part of accelerating the autonomous vehicle, the vehicle computing system may initiate the application of different propulsion forces to at least two or more wheel assemblies. For example, the vehicle computing system may apply a larger propulsion force to one wheel in order to cause the corresponding tire to slip or slide relative to the driving surface. The larger propulsion force may increase over time until the slip or slide of the tire is detected. A smaller propulsion force may be applied to other wheels so that no slip or slide is experienced. In this way, passengers may not perceive the slip or slide associated with the larger propulsion force. The resulting wheel rotation or wheel torque associated with the wheel assembly receiving the greater propulsive force may be determined directly or indirectly using one or more sensors. Friction may be determined by the vehicle control system based on the greater propulsive force, wheel rotation, and / or wheel torque.

[0044] According to some example embodiments, future events for a vehicle may be detected based on at least one of map data, motion planning data, or route planning data associated with the vehicle. By way of example, a friction estimation system may determine that future events have been planned for which friction estimates may be determined without significantly disrupting vehicle passengers. By way of example, a vehicle computing system may determine that a vehicle plans to stop at an intersection within an urban environment based at least in part on map data, a planned vehicle route, and / or motion planning data for the vehicle (e.g., associated with an autonomous vehicle). Thus, the vehicle computing system may determine that friction estimates may be calculated during one or more time periods including reducing the speed of the vehicle, stopping the vehicle, and / or increasing the speed of the vehicle with respect to the intersection.

[0045] According to other example embodiments, an ongoing vehicle event may be detected based at least in part on sensor data. By way of example, a friction force estimation system may determine that a speed reduction (e.g., a car braking when approaching an intersection or braking to avoid an object) is occurring based on sensor data associated with a braking system indicating that brake pressure is currently being applied.

[0046] In some embodiments, the friction force estimate may include a measurement or value indicating the friction force associated with the driving surface. The friction force estimate measurement or value may be determined using sensor data of the vehicle. In other embodiments, the friction force estimate does not include an actual measurement or value indicating the friction force associated with the driving surface. In such embodiments, the friction force estimate may include other data indicating the friction force associated with the driving surface. For example, one or more threshold friction forces may be used in some embodiments. In such cases, the vehicle computing system may determine whether the friction force associated with the driving surface meets one or more thresholds without determining the actual measurement or value associated with the driving surface. The friction force estimate may be an indication that the friction force meets or does not meet the threshold friction force.

[0047] In some embodiments, a confidence measure associated with a friction estimate may be utilized by a vehicle computing system. As part of vehicle operation, the confidence measure may be utilized by a vehicle computing system. By way of example, when a friction estimate is first generated, the vehicle computing system may assign a relatively high confidence measure to the friction estimate. The confidence value may decrease over time to represent the longer the likelihood that the driving surface has changed since the friction estimate was first determined. In some examples, additional sensor data may be used to determine the confidence of the friction estimate. For example, image data generated by one or more image sensors, such as cameras, may be analyzed to determine the similarity between the current driving surface and the driving surface for which the last friction estimate was performed. If the current driving surface is more similar to the previous driving surface, then a higher confidence value may be assigned to the friction estimate, while if it is less similar to the current driving surface, then a lower confidence value may be assigned.

[0048] The vehicle computing system may utilize friction estimates associated with non-autonomous and autonomous vehicles in various ways. For example, a non-autonomous vehicle may provide an indication of the friction estimate to the driver of the vehicle. Additionally and / or alternatively, the non-autonomous vehicle may transmit the friction estimate to other vehicles and / or other computing systems. Furthermore, the non-autonomous vehicle may use the friction estimate as part of controlling the vehicle, for example, to control an advanced driver assistance system.

[0049] The vehicle computing system can use friction estimation to control the autonomous operation of the autonomous vehicle in various ways. For example, in some instances, the vehicle control system may generate a motion plan based at least in part on data indicating friction. In other instances, the vehicle control system may generate a route or map plan based at least in part on data indicating friction. In yet other instances, the vehicle control system may generate constraints associated with controlling the autonomous vehicle based on data indicating friction. Constraints may include, for example, motion planning constraints, mapping constraints, or general operating constraints. In some instances, controlling an autonomous vehicle based on data indicating the friction of a driving surface may include implementing motion planning and / or route planning based on friction estimation. Implementing motion planning may include translating the motion plan by a vehicle controller and / or interface and sending one or more signals to a control system (e.g., acceleration, braking, steering, etc.).

[0050] As an example, as part of generating a motion plan for an autonomous vehicle, a friction estimate may be used. For example, if it is determined that the friction of the current driving surface does not meet one or more thresholds, then the vehicle control system may utilize lower or more stringent motion planning constraints. These lowered motion planning constraints may provide a greater distance between the autonomous vehicle and surrounding vehicles, a lower top speed, a lower acceleration rate, a lower speed during turns, a longer allotted distance for stopping, and / or any other appropriate constraints.

[0051] As another example, friction estimates may be used as part of a mapping, such as when generating a route plan for a non-autonomous or autonomous vehicle. For example, if it is determined that the friction of the current driving surface does not meet one or more thresholds, the vehicle control system may utilize lower or more stringent route planning constraints. These reduced route planning constraints may provide for travel on roads with lower classification (e.g., spacing), roads with lower traffic volume, roads that have been recently cleared of snow, etc., and with any other suitable route planning constraints.

[0052] As yet another example, as part of general autonomous vehicle operation, friction estimates can be used, for example, when generating an operation plan for an autonomous vehicle. For example, if it is determined that the friction of the current driving surface does not meet one or more thresholds, then the vehicle control system can utilize reduced or more stringent operating constraints. These reduced operating constraints can provide that if the friction of the driving surface does not meet one or more thresholds, then the autonomous vehicle will stop and not perform autonomous operations.

[0053] According to one example aspect of the invention, there is provided a computing system comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations include detecting a stop associated with a vehicle, initiating a steering action of the vehicle during the stop. The steering action is associated with movement of at least one tire of the vehicle relative to a driving surface. The operations include: obtaining operational data associated with the steering action during the stop of the vehicle; determining a friction force associated with the driving surface based at least in part on the operational data associated with the steering action; and generating data indicative of the friction force associated with the driving surface.

[0054] According to one example aspect of the invention, there is provided a computing system comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations include detecting an event that includes at least one of acceleration, deceleration, or stopping associated with an autonomous vehicle. The operations include, in response to detecting the event, obtaining operational data associated with the autonomous vehicle during the event. The operations further include determining, based at least in part on the operational data, data indicating friction associated with a surface on which the autonomous vehicle is traveling during the event. The operations include controlling the autonomous vehicle based at least in part on the data indicating the friction associated with the surface.

[0055] According to one example aspect of the invention, there is provided an autonomous vehicle comprising one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations include detecting an event associated with an autonomous vehicle. The operations further include obtaining operational data associated with the autonomous vehicle during the event. The operations include determining, based at least in part on the operational data, data indicating friction associated with a surface on which the autonomous vehicle is traveling during the acceleration event. The operations further include generating at least one constraint for the autonomous vehicle based at least in part on the data indicating the friction associated with the surface.

[0056] According to one example aspect of the invention, a computer-implemented method is provided that includes detecting (e.g., by a vehicle computing system, etc.) an acceleration event associated with an autonomous vehicle. The method includes obtaining operational data associated with the autonomous vehicle during the acceleration event. The method includes determining, based at least in part on the operational data, data indicative of friction forces associated with a surface on which the autonomous vehicle is traveling during the acceleration event. The method includes determining, based at least in part on the data indicative of the friction forces associated with the surface, at least one wheel-based action of the autonomous vehicle.

[0057] The systems and methods of the present invention provide several technical effects and benefits, particularly in the field of vehicles and computing technology. For example, a vehicle according to an embodiment of the disclosed technology may generate data indicating the friction associated with a driving surface in order to improve vehicle operation and / or provide an indication of the friction to the driver. By way of example, an autonomous vehicle may utilize friction data to generate an improved motion plan for the autonomous vehicle, to generate an improved route or map plan for the autonomous vehicle, and / or to improve the general operation of the autonomous vehicle. More specifically, friction data may be used to control the movement of the autonomous vehicle and / or the route on which the autonomous vehicle travels. By utilizing friction estimates, an autonomous vehicle may be able to operate more safely by tailoring the movement and control of the autonomous vehicle based on the determined friction of the surface on which the vehicle is currently traveling. In addition, an autonomous vehicle may detect specific events during which friction estimates may be determined without significantly interfering with the passengers of the autonomous vehicle.

[0058] The systems and methods described herein can provide specific technical benefits to a vehicle computing system of a vehicle. Specifically, the vehicle computing system can initiate friction determination in response to a detected event. The vehicle computing system can initiate friction estimation using a technique that can minimize interference to vehicle passengers (e.g., by taking measures that will generally be consistent with the speed reduction, stop, speed increase, etc. of the vehicle at a certain point in time). More specifically, the vehicle computing system can initiate one or more wheel-based actions so that sensor data associated with the wheel-based actions can be obtained. Then, the vehicle computing system can use operational data to determine the friction associated with the driving surface. Then, the vehicle computing system can provide an indication of friction such as a warning to the driver, transmit data indicating friction to one or more other computing devices, and / or use friction estimation as part of autonomous vehicle operation. By way of example, as part of motion planning, route planning, or other operations associated with an autonomous vehicle, a vehicle computing system can utilize friction estimation.

[0059] Although many examples are described herein with respect to autonomous vehicles, the disclosed technology is not limited to autonomous vehicles. Any vehicle may determine a friction estimate associated with a driving surface and use the friction estimate as part of vehicle operation according to embodiments of the present invention. For example, a non-autonomous vehicle may detect an event, such as a stop, using sensor data, map data, route planning data, or other data. For example, a non-autonomous vehicle may use sensor data to identify a stop sign and detect future acceleration events, including stopping at a location close to a stop sign. As another example, a non-autonomous vehicle may use map data or route planning data to identify a stop sign or intersection at which the vehicle will stop. The vehicle may initiate a wheel-based action during a stop and determine the friction associated with the driving surface based on operational data associated with the wheel-based action.

[0060] As part of the operation of a vehicle, a non-autonomous vehicle may use friction estimates in various ways. For example, a non-autonomous vehicle may use a user interface to provide an indication of the friction estimate (e.g., visual, audible, or other) to inform the driver of the friction. The vehicle may provide a signal including data indicating the friction estimate to one or more remote computing devices, such as a server or a vehicle computing system of another vehicle. As another example, as part of an advanced driver assistance system, a non-autonomous vehicle may use data indicating friction. For example, as part of the operation of a vehicle including control of a braking system, a propulsion system, and / or a steering system, a non-autonomous vehicle may use friction estimates. More specifically, a vehicle may provide and / or adjust braking force, propulsion force, and / or steering force based on the friction associated with the driving surface. As a specific example, a vehicle may reduce or otherwise modify the amount of braking force applied to a wheel assembly based on a low friction driving surface in response to a user braking input. For example, a braking system may produce a smooth and gradually increasing braking force, rather than a sudden increase in braking force, as indicated by a user input. As another example, the vehicle may reduce or otherwise modify propulsion in response to a user propulsion input. For example, the propulsion system may generate a smooth and gradual increase in propulsion rather than a sudden increase in propulsion.

[0061] According to an example embodiment, a computer system configured to perform an operation associated with a vehicle is provided. The operation may include detecting an event associated with the vehicle. In some examples, the event may be a current or future stop of the vehicle. The operation may include starting a wheel-based action based on the detection of the event. The wheel-based action may be performed during the event, and operational data associated with the vehicle during the event may be obtained. For example, the vehicle may initiate a steering movement, and obtain data indicating rack force, steering angle, or other operational data associated with the steering movement when the vehicle stops. The vehicle may determine data indicating the friction of the driving surface at least in part based on the operational data. The vehicle may perform one or more subsequent vehicle actions at least in part based on the operational data. The vehicle action may include, for example, generating a motion plan for an autonomous vehicle or controlling an advanced driver assistance system for a non-autonomous vehicle.

[0062] Referring now to the drawings, example embodiments of the present invention will be discussed in further detail. Figure 1 A block diagram of an example autonomous vehicle 10 is depicted according to an example embodiment of the present invention. Autonomous vehicle 10 is capable of sensing its environment and navigating without human input. Autonomous vehicle 10 may be a land-based autonomous vehicle (e.g., a car, truck, bus, etc.), an air-based autonomous vehicle (e.g., an airplane, drone, helicopter, or other aircraft), or other types of vehicles (e.g., watercraft, rail-based vehicles, etc.).

[0063] The autonomous vehicle 10 includes one or more sensors 101, a vehicle computing system 102, and one or more vehicle controls 107. The vehicle computing system 102 may help control the autonomous vehicle 10. In particular, the vehicle computing system 102 may receive sensor data from the one or more sensors 101, attempt to understand the surrounding environment by performing various processing techniques on the data collected by the sensors 101, and generate an appropriate motion path through the surrounding environment. The vehicle computing system 102 may control the one or more vehicle controls 107 to operate the autonomous vehicle 10 according to the motion path.

[0064] The vehicle computing system 102 includes a computing device 110, which includes one or more processors 112 and a memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be one processor or multiple processors that are operably connected. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 that are executed by the processor 112 to cause the vehicle computing system 102 to perform operations.

[0065] like Figure 1 As illustrated in FIG. 1 , the vehicle computing system 102 may include a perception system 103 , a prediction system 104 , and a motion planning system 105 , which cooperate to perceive the surrounding environment of the autonomous vehicle 10 and determine a motion plan for controlling the motion of the autonomous vehicle 10 accordingly.

[0066] In particular, in some implementations, the perception system 103 may receive sensor data from one or more sensors 101 coupled to or otherwise included within the autonomous vehicle 10. As examples, the one or more sensors 101 may include a light detection and ranging (LIDAR) system, a radio detection and ranging (RADAR) system, one or more cameras (e.g., visible spectrum cameras, infrared cameras, etc.), and / or other sensors. The sensor data may include information describing the location of objects within the surrounding environment of the autonomous vehicle 10.

[0067] As one example, for a LIDAR system, the sensor data may include the positions of several points corresponding to an object that reflected a ranging laser (e.g., relative to the LIDAR system in three-dimensional space). For example, a LIDAR system may measure distance by measuring the time of flight (TOF) that a short laser pulse takes to travel from the sensor to the object and back, calculating the distance from the known speed of light.

[0068] As another example, for a RADAR system, the sensor data may include the positions of several points corresponding to objects that reflected ranging radio waves (e.g., relative to the RADAR system in three-dimensional space). For example, radio waves (e.g., pulsed or continuous) transmitted by the RADAR system may reflect from an object and travel back to a receiver of the RADAR system, giving information about the position and velocity of the object. Thus, the RADAR system may provide useful information about the current velocity of an object.

[0069] As yet another example, various processing techniques (e.g., range imaging techniques such as, for example, structure from motion, structured light, stereo triangulation, and / or other techniques) may be performed for one or more cameras to identify locations of a number of points corresponding to objects depicted in imagery captured by the one or more cameras (e.g., in three-dimensional space relative to the one or more cameras). Other sensor systems may also identify locations of points corresponding to objects.

[0070] As another example, the one or more sensors 101 may include a positioning system. The positioning system may determine the current location of the autonomous vehicle 10. The positioning system may be any device or circuit system for analyzing the location of the autonomous vehicle 10. For example, the positioning system may determine the location by using inertial sensors, one or more of satellite positioning systems based on IP addresses, by using triangulation and / or proximity to network access points or other network components (e.g., cellular towers, WiFi access points, etc.), and / or other suitable techniques. The location of the autonomous vehicle 10 may be used by various systems of the vehicle computing system 102.

[0071] Thus, one or more sensors 101 may be used to collect sensor data that includes information describing the location of points corresponding to objects within the surrounding environment of the autonomous vehicle 10 (e.g., relative to the autonomous vehicle 10 in three-dimensional space).

[0072] One or more sensors 101 may additionally include sensors associated with measurement parameters, characteristics, environmental data, or other aspects of the autonomous vehicle and associated hardware. For example, an inertial measurement unit (IMU) sensor, pressure sensor, power sensor, contact sheet sensor, suspension travel sensor, laser sensor, radar sensor, sound sensor, and / or any other suitable sensor may be used. The sensor data acquired by the autonomous vehicle may include information describing the position, movement (such as rate, speed, acceleration, etc.), and / or force experienced by the autonomous vehicle and / or its individual components (such as tires, axles, transmission systems, brakes, steering systems, etc.). The autonomous vehicle may include sensors that measure input steering forces such as input torque, resulting alignment torques or rack forces of input steering forces, such as steering displacements of wheel rotations or steering angles. The autonomous vehicle may include sensors that measure input braking forces applied by the motor of the braking system and / or wheel torques associated with the input braking forces. The autonomous vehicle may include sensors that measure input propulsion forces to axles, drives, or wheel assemblies, and wheel torques associated with the input propulsion forces. An autonomous vehicle may include sensors such as encoders that measure, for example, the degree of rotation of a wheel.

[0073] Sensor data is an example of operational data associated with an autonomous vehicle. The operational data may additionally or alternatively include data derived from the sensor data, such as a force, distance, or angle determined based on the sensor data. The sensor data may be used to determine data indicative of the friction of a driving surface, as further described herein.

[0074] In addition to sensor data, perception system 103 may retrieve or otherwise obtain map data 126 that provides detailed information about the surroundings of autonomous vehicle 10. Map data 126 may provide information about: the identification and location of different travel paths (e.g., roadways), road segments, buildings, or other items or objects (e.g., lamp posts, crosswalks, curbs, etc.); the location and direction of traffic lanes (e.g., the location and direction of parking lanes, turn lanes, bicycle lanes, or other lanes within a particular roadway or other travel path); traffic control data (e.g., the location and indication of signs, traffic lights, or other traffic control devices); and / or any other map data that provides information that helps vehicle computing system 102 understand and perceive its surroundings and its relationship to them.

[0075] The perception system 103 may identify one or more objects in proximity to the autonomous vehicle 10 based on sensor data received from one or more sensors 101 and / or map data 126. In particular, in some implementations, the perception system 103 may determine state data for each object describing the current state of such object as described. As an example, the state data for each object may describe estimates of the object's: current position (also referred to as location); current speed (also referred to as velocity); current acceleration; current heading; current orientation; size / occupancy area (e.g., as represented by a bounding shape such as a bounding polygon or polyhedron); classification (e.g., vehicle vs. pedestrian vs. bicycle vs. other); yaw rate; and / or other state information.

[0076] In some embodiments, the perception system 103 may determine the state data of each object over several iterations. In particular, the perception system 103 may update the state data of each object at each iteration. Thus, the perception system 103 may detect and track objects (e.g., vehicles) approaching the autonomous vehicle 10 over time.

[0077] Prediction system 104 may receive state data from perception system 103 and predict one or more future locations of each object based on such state data. For example, prediction system 104 may predict where each object will be located within the next 5 seconds, 10 seconds, 20 seconds, etc. As one example, an object may be predicted to follow its current trajectory based on its current velocity. As another example, other, more complex prediction techniques or modeling may be used.

[0078] The motion planning system 105 may determine one or more motion plans for the autonomous vehicle 10 based at least in part on the predicted one or more future positions of the object and / or the state data of the object provided by the perception system 103. In other words, given information about the current position of the object and / or the predicted future positions of nearby objects, the motion planning system 105 may determine a motion plan for the autonomous vehicle 10 that optimally navigates the autonomous vehicle 10 relative to the object at its current and / or future positions.

[0079] As one example, in some implementations, motion planning system 105 may evaluate one or more cost functions for each of one or more candidate motion plans for autonomous vehicle 10. For example, a cost function may describe the cost of pursuing a particular candidate motion plan (e.g., over time) and / or describe the reward of pursuing a particular candidate motion plan. For example, the reward may be a sign that is the opposite of the cost.

[0080] The motion planning system 105 may provide the optimal motion plan to a vehicle controller 106 which controls one or more vehicle controls 107 (e.g., actuators or other devices that control airflow, steering, braking, etc.) to execute the optimal motion plan. The vehicle controller may generate one or more vehicle control signals for the autonomous vehicle based at least in part on the output of the motion planning system.

[0081] Each of the perception system 103, the prediction system 104, the motion planning system 105, and the vehicle controller 106 may include computer logic for providing the desired functionality. In some embodiments, each of the perception system 103, the prediction system 104, the motion planning system 105, and the vehicle controller 106 may be implemented in hardware, firmware, and / or software that controls a general purpose processor. For example, in some embodiments, each of the perception system 103, the prediction system 104, the motion planning system 105, and the vehicle controller 106 includes a program file stored on a storage device, loaded into a memory, and executed by one or more processors. In other embodiments, each of the perception system 103, the prediction system 104, the motion planning system 105, and the vehicle controller 106 includes one or more computer executable instruction sets stored in a tangible computer readable storage medium such as RAM, a hard disk, or an optical or magnetic medium.

[0082] In various implementations, one or more of the perception system 103, the prediction system 104, and / or the motion planning system 105 may include or otherwise utilize one or more machine learning models, such as, for example, a convolutional neural network.

[0083] The vehicle computing system 102 includes a friction estimation system 150 according to an example embodiment of the disclosed technology. The friction estimation system 150 is configured to automatically estimate the friction associated with the driving surface on which the autonomous vehicle is traveling. The friction estimation system 150 can generate friction data including the friction estimation and provide the generated friction data to systems such as the perception system 103, the prediction system 104, and / or the motion planning system 105 for autonomous vehicle operation. The friction estimation system 150 can receive sensor data and map data from the sensor 101 and the positioning system 111, respectively. For example, the friction estimation system 150 can receive map data and / or motion planning data in order to determine an acceleration event within which the friction estimation will be performed in some examples. During the acceleration event, the friction estimation system 150 can generate friction data 152 including the friction estimation based on the sensor data from the sensor 101. The friction force estimation system 150 can detect an acceleration event associated with the autonomous vehicle and obtain operational data, such as sensor data, associated with the autonomous vehicle during the acceleration event. The friction force estimation system 150 can determine data indicating friction forces associated with the surface on which the autonomous vehicle is traveling during the acceleration event based on the operational data. The friction force estimation system 150 can provide the data indicating friction forces to one or more additional systems to control the autonomous vehicle based on the data indicating friction forces associated with the surface.

[0084] although Figure 1 An autonomous vehicle is depicted, but a non-autonomous vehicle according to the disclosed technology may include similar components. For example, a non-autonomous vehicle may include the described components, but may not necessarily include perception system 103, prediction system 104, and / or motion planning system 105. However, in some embodiments, a non-autonomous vehicle may include one or more of perception system 103, prediction system 104, and / or motion planning system 105.

[0085] Figure 2 is a flow chart depicting an example process 200 for generating data indicative of friction of a driving surface and controlling a vehicle based at least in part on the data indicative of friction. One or more portions of process 200 (and other processes described herein) may be performed by, for example, Figure 1 The computing device 110 within the vehicle control system 102, or Fig.15 In addition, one or more parts of the processes described herein may be implemented as algorithms on hardware components of the devices described herein (e.g., as in Figure 1 and 15In an example embodiment, process 200 may be performed by friction estimation system 150 of vehicle computing system 102, for example, to generate data indicating the friction of a driving surface on which the vehicle is traveling.

[0086] At 202, a request for a friction estimate associated with an autonomous vehicle is received. In some embodiments, the request may be received by the vehicle computing system 102, or more specifically, the request may be received by the friction estimation system 150 of the vehicle computing system 102. One or more components of the autonomous driving system, such as the perception system 103, the prediction system 104, the motion planning system 105, and / or the vehicle controller 106, may issue the request for the friction estimate according to embodiments of the disclosed technology.

[0087] At 204, an event is detected. In an example embodiment, the friction estimation system 150 may detect an event in response to one or more signals from various components of the vehicle computing system. For example, the friction estimation system 150 may detect an event in response to sensor data from one or more sensors 101. The friction estimation system 150 may detect a stop, speed reduction, and / or speed increase by the vehicle as an event. For example, the friction estimation system 150 may use imagery or other sensor data to identify a stop sign, a yield sign, or other sign or object in the environment outside the vehicle that indicates a future event. In another example, the friction estimation system 150 may detect an event in response to motion planning data or route planning data from a motion planning system. For example, the friction estimation system 150 may detect an event in response to motion planning data indicating a planned stop or speed reduction, or a planned speed increase by the autonomous vehicle. As yet another example, the friction estimation system 150 may detect an event in response to motion planning data or other sensor data indicating the rotation of the steering wheels of the autonomous vehicle.

[0088] At 206, a wheel-based action request is issued to the vehicle control system of the vehicle. In an example embodiment, the friction estimation system 150 may issue a wheel-based action request to the vehicle controller 106. The vehicle controller 106 may in turn issue one or more requests and / or commands to initiate wheel-based actions to the appropriate vehicle control. For example, the friction estimation system 150 may issue a request for a braking action, a propulsion action, and / or a steering action. More specifically, the wheel-based action may be an action based on a steering wheel performed during a stop, or when the vehicle is moving in a manner that is not perceptible to the user. As another example, the friction estimation system 150 may issue a request for a propulsion wheel-based action during a detected acceleration event that includes an increase in speed. As yet another example, the friction estimation system 150 may issue a request for a braking wheel-based action during a detected acceleration event that includes a decrease in speed.

[0089] At 208, operational data associated with wheel-based actions during the detected acceleration event is obtained. The operational data may be obtained by the friction force estimation system 150 from one or more sensors 101 and / or other components of the vehicle computing system (e.g., the perception system 103, the prediction system 104, and / or the motion planning system 105). Examples of operational data may include data indicating one or more of input steering force, wheel rotation, wheel torque, rack force, alignment torque, etc. associated with steering wheel-based actions. Other examples may include data indicating input braking force, wheel rotation, wheel torque, etc. associated with braking wheel-based actions. Other examples of operational data may include data indicating input propulsion force, wheel rotation, wheel torque, etc. associated with propulsion wheel-based actions.

[0090] At 210, friction is determined based on the operational data obtained at 208. In an example embodiment, the friction may be a friction estimate. By way of example, the friction associated with a steering wheel-based action may be determined based on an input steering force applied to a steering system, and a wheel rotation and / or torque (e.g., alignment torque, rack force, etc.) caused by the input steering force. More specifically, the maximum steering force and / or maximum wheel rotation may be used to determine the friction associated with the interaction of one or more tires with a driving surface. The friction associated with a braking wheel-based action may be determined based on an input braking force applied to a braking system, a resulting wheel rotation associated with the input braking force, and / or a resulting wheel torque. The friction associated with a propulsion wheel-based action may be determined based on an input propulsion force applied to a propulsion system, a resulting wheel rotation associated with the input propulsion force, and / or a resulting wheel torque.

[0091] At 212, data indicative of the friction forces determined at 210 may be provided. By way of example, the friction force estimation system may provide an indication of one or more friction force estimates to a driver via a user interface of the vehicle. As another example, the friction force estimation system 150 may provide data indicative of one or more friction force estimates to a remote computing system, such as a remote server or another vehicle. In some examples, the friction force estimation system 150 may provide the one or more friction force estimates to the perception system 103, the prediction system 104, and / or the motion planning system 105. The friction force estimation system 150 may provide data indicative of the friction forces associated with the driving surface.

[0092] At 214, the vehicle is controlled based at least in part on data indicating the friction of the driving surface. By way of example, the motion planning system 105 may generate one or more motion plans based on the estimated friction of the driving surface. As another example, the vehicle control system may generate a route or map plan based at least in part on the friction estimate. As yet another example, the vehicle control system may generate constraints associated with controlling the vehicle based at least in part on data indicating the friction. The constraints may include motion planning constraints, map constraints, and / or general operating constraints. In another example, the vehicle control system may control an ADAS or other system of a non-autonomous vehicle based at least in part on the friction of the driving surface. It should be noted that the operation at 214 is optional. For example, data indicating the estimated friction may be provided at 212 without controlling the vehicle based on the friction.

[0093] In some examples, a vehicle control system may generate a motion plan based at least in part on data indicative of friction. Controlling an autonomous vehicle based on data indicative of friction of a driving surface may include implementing a friction-based motion plan and / or route planning. Implementing a motion plan may include translating the motion plan by a vehicle controller and / or interface and sending one or more signals to a control system (e.g., acceleration, braking, steering, etc.).

[0094] As part of controlling an autonomous vehicle, various thresholds may be utilized together with friction data. The vehicle control system may utilize lower or more stringent motion planning constraints in response to lower friction estimates. These reduced motion planning constraints may provide a larger distance between the autonomous vehicle and surrounding vehicles, a lower top speed, a lower acceleration rate, a lower speed during a turn, a longer allocated distance for stopping, and / or any other appropriate constraints. As part of a map, friction data may be used, for example, when generating a route plan for an autonomous vehicle. The vehicle control system may utilize lower or more stringent route planning constraints in response to a driving surface with friction that does not meet one or more thresholds. These reduced route planning constraints may provide travel on roads with lower classifications (e.g., spacing), roads with lower traffic flow, roads that have recently been cleared of snow, etc., and on roads with any other appropriate route planning constraints. As part of general autonomous vehicle operations, friction data may be used, for example, when generating an operational plan for an autonomous vehicle. For example, a vehicle control system may utilize reduced or more stringent operational constraints in response to a driving surface with friction that does not meet one or more thresholds. These reduced operating constraints may provide that if the friction of the driving surface does not meet one or more thresholds, the autonomous vehicle will stop and not perform autonomous operations.

[0095] As a specific example, an autonomous vehicle may utilize three or more operating modes depending on the friction estimate. If the friction of the driving surface meets a first threshold (e.g., a friction coefficient greater than or equal to 0.3), normal driving operation may be utilized. If the friction meets a second threshold (a friction coefficient greater than or equal to 0.2) but fails to meet the first threshold, increased operational, map, and / or general operational constraints may be used. If the friction fails to meet the second threshold, the vehicle may be restricted or prohibited from autonomous operation. Other numbers and examples of constraints and / or thresholds may be used according to example embodiments.

[0096] Figure 3 An example outdoor environment 300 is depicted illustrating an implementation of friction estimation associated with an event according to an example embodiment of the invention. The outdoor environment 300 may include one or more travel paths (e.g., intersecting roadways). The scene illustrated in the outdoor environment 300 may include one or more static and / or dynamic objects (e.g., pedestrians, vehicles, cyclists, sidewalks, lamp posts, signs, etc.). The environment 300 may be an environment in which a vehicle is traveling and / or will travel. Figure 3 The environment 300 shown in FIG. 1 is presented by way of example only and is not intended to be limiting.

[0097] Vehicle computing system 102 may obtain sensor data associated with environment 300. For example, vehicle computing system 102 may obtain sensor data including two-dimensional and / or three-dimensional data associated with environment 300. For example, the three-dimensional data may include a plurality of points. The plurality of points may be included in a three-dimensional point cloud associated with environment 300.

[0098] The vehicle may use sensor data and / or motion planning data planning related to the environment 300 to determine the friction associated with the driving surface based on the action of the wheels. For example, the perception system 103 may detect a traffic signal 302 and / or one or more pedestrians 304 from the sensor data. The motion planning system 105 may generate one or more motion plans based on the detected traffic signal 302 and / or pedestrians 304. The one or more motion plans may include reducing the speed of the autonomous vehicle to comply with a traffic signal indicating that the vehicle must stop, or reducing the speed of the autonomous vehicle to allow the pedestrian 304 to cross the road. As another example, the one or more motion plans may include increasing the speed of the autonomous vehicle in response to a traffic signal indicating that the vehicle may proceed from or pass through an intersection.

[0099] The friction estimation system 150 may detect an event such as a reduction in the speed of the autonomous vehicle in response to a traffic signal 302 and / or a pedestrian 304. In response, the friction estimation system 150 may issue one or more wheel-based motion requests to determine the friction during the detected acceleration event. For example, the friction estimation system 150 may issue a steering wheel-based motion request to determine the friction when the vehicle 10 stops in response to the traffic signal 302 and / or the pedestrian 304. As another example, the friction estimation system may issue a braking wheel-based motion request to determine the friction of the driving surface when the vehicle reduces speed in response to the traffic signal and the plurality of pedestrians 304. Similarly, the friction estimation system 150 may issue a propulsion wheel-based motion request to determine the friction of the driving surface when the vehicle increases speed in response to a traffic signal indicating that the vehicle may pass or in response to clearing one or more pedestrians 304 from the road.

[0100] Figure 4Another example outdoor environment 350 is depicted and data indicating friction of a driving surface is generated based on detected acceleration events. The outdoor environment 350 may include one or more travel lanes (e.g., intersecting roadways). The scene illustrated in the outdoor environment 350 may include one or more static and / or dynamic objects (e.g., pedestrians, vehicles, cyclists, sidewalks, lamp posts, signs, etc.), such as traffic signals 352. The friction estimation system 150 may initiate wheel-based motions to determine friction estimates for the travel lanes at intervals planned based on the environment 350. More specifically, the wheel-based motions may be initiated in a manner to minimize or eliminate perception of the wheel-based motions by passengers of the autonomous vehicle.

[0101] More specifically, in Figure 4 In an example of , the vehicle computing system 102 may generate one or more motion plans that cause the autonomous vehicle to roughly traverse a path, as shown in Figure 354. The vehicle computing system 102 may determine that the autonomous vehicle will stop at one or more of the traffic signals 352. Based on the detection of acceleration events associated with stopping and / or continuing from stopping associated with one or more of the traffic signals, the friction estimation system 150 may plan wheel-based actions to minimize any interference with passengers of the autonomous vehicle. For example, the vehicle computing system may initiate a steering wheel-based action in response to the autonomous vehicle stopping in response to one or more of the traffic signals. As another example, the vehicle computing system may initiate a braking wheel-based action in response to the autonomous vehicle reducing speed when approaching the traffic signal. As yet another example, the vehicle computing system may initiate a propulsion wheel-based action in response to the autonomous vehicle increasing speed when it continues from its stop position at the traffic signal. Still further, the vehicle computing system may initiate wheel-based actions in response to a detected acceleration event associated with the crosswalk 356 .

[0102] Figure 5 is a block diagram depicting additional details of a friction estimation system 150 and the generation of friction estimates according to an example embodiment of the disclosed technology. Figure 5In the example of , the friction estimation system 150 includes a friction detection controller 402 and a friction estimator 404. The friction detection controller 402 communicates with an autonomous driving system, which is generally depicted as an autonomous driving system 410. In another example (not depicted), the friction detection controller 402 may communicate with a vehicle control system of a non-autonomous vehicle rather than with the autonomous driving system 410 of an autonomous vehicle. The autonomous driving system 410 may include a perception system 103, a prediction system 104, and / or a motion planning system 105. In addition, the friction detection controller 402 communicates with a vehicle motion state component 430 and a steering actuator 420. It should be understood that the components and external components of the friction estimation system 150 are provided by way of example only. For example, the friction detection controller 402 may communicate with any number of vehicle controls and / or vehicle controller 106.

[0103] Examples of signals transmitted between various components of an autonomous vehicle as part of the friction estimation process are shown in Figure 5 4. One or more components of the autonomous driving system 410 (or a vehicle control system for a non-autonomous vehicle) may issue a detection request to the friction detection controller 402. A detection request is one example of a request for a friction estimate, however, it should be appreciated that other types of requests may be utilized. In an example embodiment, the autonomous driving system may issue a request for a friction estimate or measurement when it is uncertain that the friction associated with the driving surface is high and knowledge of the friction is required in order to provide safe and convenient travel.

[0104] The friction force detection controller 402 can reply to the vehicle control system of the autonomous driving system 410 or the non-autonomous vehicle whether there is an indication of the detection availability that can be used to measure or otherwise estimate the friction force associated with the driving surface. In some examples, when the friction force detection controller 402 determines that there is a future or ongoing acceleration event, there is detection availability. In some examples, as part of determining whether there is detection availability, the vehicle motion state component 430 can provide an indication of whether the vehicle is moving or stationary. In some examples, detection availability can only exist when the vehicle is stationary. However, in other examples, detection availability can exist at other times, such as when the vehicle reduces speed and / or increases speed. If the direction of motion received from the vehicle motion state component 430 is suitable for performing friction estimation, the friction force detection controller 402 can reply to the autonomous driving system to indicate the detection availability. However, if the direction of motion is not suitable for performing the friction estimation process, the friction force detection controller 402 can indicate the detection availability to the autonomous driving system 410.

[0105] In response to determining that the friction estimation process can be executed, the friction detection controller 402 issues a steering torque request to the steering actuator 420. The steering actuator 420 is an example of the vehicle control member 107. The steering torque request is an example of a request for a wheel-based action. In some examples, the steering torque request may specify an input torque to be applied to the steering system. More specifically, the steering torque request may specify an increase in the slope of the steering torque or an increase in the steering torque from a low value to a high value. In some examples, a steering torque request requesting an increase in torque may be issued to the steering actuator until the peak friction is measured or otherwise found. At that point, the steering torque request may cause the steering torque to slope down until it reaches zero. In some examples, if the steering actuator has a problem actuating in response to the steering torque request, the detection availability may be false.

[0106] In response to the steering torque applied by the steering actuator 420, the steering actuator 420 provides an indication of the resulting steering displacement, such as a wheel steering angle. For example, data indicative of the resulting wheel steering angle may be provided back to the friction detection controller 402 and the friction estimator 404. Additionally, rack force may be measured by one or more sensors associated with the steering actuator and an indication of rack force provided to the friction detection controller 402 and the friction estimator 404. In some examples, the rack force is an alignment torque associated with the steering system.

[0107] The friction force estimator 404 can obtain rack force and steering angle data and determine the peak friction force associated with the action based on the steering wheel. Generally, the higher-level friction force will be determined in response to the higher-level peak rack force and the larger steering angle applied to reach the peak rack force level. Similarly, the lower-level friction force will be determined in response to the lower-level peak rack force and the smaller steering angle applied to reach the peak rack force level. More details describing the determination of friction force based on rack force and steering angle are provided below.

[0108] Once peak friction is found, the friction estimator 404 provides a signal to the friction detection controller 402 that the steering torque request may be ended. The friction estimator 404 provides data to the autonomous driving system 410 indicative of friction measured using wheel based motions.

[0109] A detailed explanation of one example for determining a friction force or friction force estimate associated with a driving surface using a steering wheel based motion Figure 6502 is provided in FIG. 503. In general, the forces generated in association with the tire contact patch contacting the driving surface are different, depending on the friction of the driving surface. The vehicle steering system may be associated with an alignment torque equal to the torque about a notional vertical axis through the wheel hub. It may be observed that, generally, when the alignment torque is smaller, the friction of the driving surface is also smaller. Generally, the expected alignment torque varies for different steering angles and friction levels.

[0110] consider Figure 6 The graph shown in shows the steering angle δ as a function of the lateral displacement x of the bristles of the tire in the tire patch area, where the length of the tire patch area is equal to 2a. If the kingpin axis is assumed to pass through the center of the wheel hub, then the equation for the alignment torque can be defined. It can be first assumed that there is complete adhesion (friction is infinitely high and none of the bristles of the tire slips). When the tire turns at the steering angle δ, the tire responds as a torsion spring. Equation 1 states the lateral deflection v(x), and equation (2) states the lateral force distribution τ y (x).

[0111] v(x) = x tanδ Equation 1

[0112] τ y (x) = c py x tanδ Equation 2

[0113] In the equation, δ is the steering angle, x is the lateral displacement of the bristles, and c py is the transverse bristle stiffness (N / M) / m.

[0114] With Equations 1 and 2 established, the alignment torque M can be defined as stated in Equation 3 z .

[0115]

[0116] Equation 3 provides the dependence of the steering angle δ, the tire stiffness c py , and the alignment torque M of the tire contact surface (length equal to 2a) z expression.

[0117] Figure 7 Another graph depicts the steering angle δ as a function of the lateral displacement x without assuming infinitely high friction. At the end of the tire contact patch, for example, the bristles may no longer be able to withstand the high lateral forces and may start to slip. Figure 7 In , the lateral displacement is divided into an adhesion region where no slip can occur and a slip region where slip can occur. A uniform normal distribution across the tire contact patch can be assumed. Next, the lateral displacement is constrained as shown in Equation 4.

[0118]

[0119] It can be expected that the tire will slip when the conditions in Equation 5 or 6 are met.

[0120]

[0121]

[0122] The conditions that induce slip are stated in Equation 7.

[0123]

[0124] Full slip can occur when the conditions in Equation 8 are met.

[0125] |x s |=0→δ=90° Equation 8

[0126] If the adhesion and sliding area are integrated, the alignment torque can be defined as stated in Equation 9.

[0127]

[0128] Therefore, the alignment torque can be expressed as stated in Equations 10, 11 and 12.

[0129]

[0130] Otherwise, Equation 11

[0131]

[0132] Figure 8 is a graph that plots an expression for alignment torque as a function of steering angle. The graph depicts three lines 552, 554, and 556 that represent driving surfaces using three different friction coefficients. Line 552 represents a driving surface with a maximum friction coefficient of mue=1. Line 554 represents a driving surface with a mid-range friction coefficient of mue=0.5. Line 556 represents a driving surface with a low-range friction coefficient of mue=0.2.

[0133] Line 556 illustrates that a running surface with a low coefficient of friction has a low peak alignment torque. Additionally, line 556 illustrates that for a running surface with a low coefficient of friction, the peak alignment torque will be reached at a relatively low steering angle. Line 554 illustrates that for a running surface with a mid-range coefficient of friction, the peak alignment torque is greater in the mid-range of steering angles to reach the peak alignment torque. Finally, line 552 illustrates that for a running surface with a high coefficient of friction, the peak alignment torque is in the high range and requires a larger steering angle to reach the peak alignment torque. Together, the lines illustrate that the alignment torque and steering angle characteristics near the origin are similar and slightly independent of friction.

[0134] It should be noted that the above expressions in equations 10, 11 and 12 are valid for a steering axis passing through the center of the tire contact patch. In general, the center rotation may have a longitudinal displacement (such as castor track) and a lateral displacement (scrape radius). Equation 13 states the definition of the alignment torque Ta incorporating these effects as a scaling factor γ.

[0135] T a =γM z Equation 13

[0136] In Equation 13, M z is the alignment torque stated in Equations 10, 11, and 12 without the caster path and scrape radius.

[0137] In an example embodiment, the sensor 101 includes a rack force sensor and / or a pinion angle sensor. In some examples, the rack force sensor may be a virtual sensor. The alignment torque at the input pinion of the steering system may be equal to the sum of the motor input to the steering system and the driver input. The rack force sensor may provide an indication of the rack force F r The rack force can be defined as stated in Equation 14.

[0138] F r =bcosδ*T a,1 +bcosδ*T a,2 =γbcosδ(M z,1 +M z,2 ) Equation 14

[0139] For a vehicle with equal steering and normal forces on both front wheels, the rack force can be defined as stated in Equation 15.

[0140] F r =2γbcosδM z,1 Equation 15

[0141] If the tire becomes saturated, the rack force can be defined as stated in Equation 16.

[0142]

[0143] Keeping in mind the above expressions, the friction estimation system 150 can determine the friction associated with the driving surface using the friction detection controller 402 and / or the friction estimator 404. The friction detection controller 402 can collect samples of the alignment torque and steering angle data. In some examples, a threshold number of samples can be collected and used to estimate the friction. In addition, threshold excitations such as threshold steering angles and / or input torques can be used to estimate the friction.

[0144] Fig. 9 A graph illustrating a technique for measuring friction associated with a driving surface based on the action of a steering wheel according to an example embodiment of the disclosed technology is depicted. In response to a probe request and detecting an acceleration event, the friction probe controller 402 may ramp the steering torque request 630 from a starting torque equal to zero at A to a maximum level at B, as shown in graph 530. As shown in graph 540, the rack force will result from the input steering torque request. Fig. 9 The rack force in is shown as a function of steering angle. As the steering angle increases in response to the input steering torque request, the rack force increases from point A to point B. Once enough samples are collected and a threshold excitation has been observed, the steering torque request can be ramped down as shown at point B to point C. As the steering torque ramps down, the rack force decreases from point B to point C. By way of example, in some examples, the time from point A to point C can be approximately 1 second. In other examples, the time can be greater or less than 1 second.

[0145] Fig.10 is a graph depicting a technique for calculating friction associated with a surface based on a measured rack force associated with an input steering torque. In one example, the friction estimator 404 can determine the friction by finding the peak value of the rack force resulting from an increasing input steering torque request. In an example embodiment, the peak value of the rack force defined in Equation 15 can be used.

[0146] Various techniques can be used to find the peak of the rack force. In one example, the peak can be found by defining a threshold for the inclination of the curve defined by equation 17.

[0147] dF r / dδ Equation 17

[0148] Equation 17 defines the change of the rack force with respect to the change of the steering angle. More specifically, the derivative of the derivative of the rack force with respect to the steering angle is defined as the quotient of the two derivatives (respectively).

[0149] Refer to Equation 17 and Fig.10 , when the steering torque is requested, it can be Fig.10 The obtained rack force and steering data (e.g., steering angle) are sampled as shown in the circles in FIG. When the change in rack force relative to the change in steering angle meets a predetermined threshold (e.g., greater than or equal to a threshold), the peak steering angle δ may be used to generate a signal. peak and peak rack force δ peak Find the peak friction force. More specifically, the friction force can be calculated as stated in Equation 18.

[0150]

[0151] In Equation 18, γab can be a design parameter, and F z This may be the normal load determined from the suspension sensors or from a vehicle state estimator.

[0152] Fig.11 6 is a flow chart depicting an example process 600 for determining friction of a driving surface associated with steering wheel-based action during a detected acceleration event according to an example embodiment of the disclosed technology. In some examples, the process 600 may be performed by the friction estimation system 150 of the vehicle computing system 102. The process 600 may be performed in association with an autonomous vehicle and / or a non-autonomous vehicle.

[0153] At 602, a request is issued to apply an input torque at a steering system of an autonomous or non-autonomous vehicle. For example, the friction detection controller 402 of the friction estimation system 150 may issue a steering torque request to the steering actuator 420. The steering torque request may include an increase in steering torque. In some examples, the steering torque request may include a request to ramp up the steering torque applied to the steering system. In response to the input torque request, the steering actuator may increase the input torque applied to the steering system at 604. In some examples, the input torque is an alignment torque.

[0154] At 606, steering displacement data, such as wheel rotation data, associated with the input torque applied at the steering system is obtained. In an example embodiment, the wheel rotation data may include data indicating a steering angle resulting from the input torque application. In an example embodiment, the wheel rotation data may be obtained by the friction detection controller 402 and / or the friction estimator 404.

[0155] At 608, steering force data associated with the input torque applied at the steering system is obtained. In an example embodiment, the steering force data may include an indication of the input torque, alignment torque, and / or rack force associated with the input torque, among other things.

[0156] At 610 , a maximum wheel rotation associated with the input torque application is determined. In an example embodiment, the maximum wheel rotation is a maximum steering angle.

[0157] At 612 , a maximum steering force associated with the input torque application is determined. In an example embodiment, the maximum steering force is a maximum rack force associated with the input torque.

[0158] In some examples, the maximum wheel rotation at 610 and the maximum steering force at 612 may be determined based on the change in the maximum wheel rotation and the maximum steering force relative to each other. For example, the change in rack force relative to the change in wheel rotation may be analyzed for increasing input torque in order to determine when the relative change meets a threshold. More specifically, as previously described, the quotient derived from the derivative of the rack force divided by the derivative of the steering angle may be observed as the input torque increases until the quotient meets or exceeds a threshold. When the threshold is met, the corresponding steering angle and rack force may be determined as the maximum wheel rotation and the maximum steering force, respectively.

[0159] At 614, friction of the driving surface is determined based on the maximum steering force and / or the maximum wheel rotation. In an example embodiment, data indicative of friction of the driving surface is generated at 614. In some examples, friction of the driving surface may be determined based at least in part on the peak rack force and the peak steering angle.

[0160] Fig.12 is a block diagram depicting additional details of the friction estimation system 150 and the use thereof to generate friction data based on the actions of braking wheels and / or propelling wheels. Fig.12 In the example of FIG. 4 , the friction force detection controller 402 may determine the friction force estimation process by providing a braking system and / or a propulsion system (collectively depicted as Fig.12 The braking system / propulsion system 440 in the vehicle control 107 issues a braking request and / or a propulsion request is executed in response. The braking system / propulsion system 440 is an instance of the vehicle control 107, and the braking and / or propulsion request is an instance of a request for a wheel-based action. In some instances, the braking and / or propulsion request may specify an input braking force to be applied to the braking system. More specifically, the braking force request may specify a ramp-up of the input braking force or an increase in the braking force from a low value to a high value. In another example, the braking and / or propulsion request may specify an input propulsion force to be applied to the propulsion system, such as an input amount provided by an accelerator. More specifically, the propulsion request may specify a ramp-up of the input propulsion force or an increase in the propulsion force from a low value to a high value.

[0161] In some examples, a braking and / or propulsion force may be issued to the braking / propulsion system requesting increased force until a peak friction force is measured or otherwise found. At that point, the braking / propulsion request may ramp down the input force until it reaches zero or another steady-state level. In some examples, detect availability may be false if the braking / propulsion system has problems actuating in response to the input force request.

[0162] In response to the input braking and / or propulsion force applied by the braking system / propulsion system 440, the braking system / propulsion system provides an indication of the resulting wheel rotation. In some examples, the wheel rotation is, for example, a wheel rotation relative to the surface of the road in response to tire slip relative to the surface. For example, data indicating the resulting wheel rotation can be provided back to the friction force detection controller 402 and the friction force estimator 404. In addition, a braking force such as an input braking force or a wheel torque measured by one or more sensors associated with the braking system / propulsion system can be provided to the friction force detection controller 402 and the friction force estimator 404. In some examples, the braking force data may include an indication of the input braking force, the resulting wheel torque, etc. In other examples, the braking force data may include an indication of the rotational torque associated with the wheel and / or tire. As a specific example, the input braking force provided by the motor can be measured and provided as braking force data.

[0163] The friction estimator 404 can obtain the braking force and wheel rotation data and determine the peak friction force associated with the action based on braking and / or propelling the wheel. Generally, the higher-level friction force will be determined in response to the higher-level peak wheel torque and the smaller wheel rotation. Similarly, the lower-level friction force will be determined in response to the lower-level peak wheel torque and the larger wheel rotation.

[0164] Once peak friction is found, the friction estimator 404 provides a signal to the friction detection controller 402 that the braking and / or propulsion request may end. The friction estimator 404 provides data indicative of friction measured using wheel-based motion to an autonomous driving system 410 of an autonomous vehicle or a vehicle control system of a non-autonomous vehicle.

[0165] Fig.13 is a flow chart depicting an example process 700 for determining friction associated with braking wheel-based actions during a detected event in accordance with an example embodiment of the disclosed technology. In some examples, process 700 may be performed by friction estimation system 150 of vehicle computing system 102. Fig.13 Depicts a specific example in which a differential braking force is applied. However, it should be noted that a differential braking force is not required. For example, one or more input braking forces may be applied to one or more wheel assemblies as part of an action based on braking a wheel. In some examples, a single braking force is applied to one wheel assembly. In other examples, a single braking force is applied to multiple wheel assemblies. In yet other examples, multiple braking forces may be applied.

[0166] At 702, one or more requests for braking wheel-based actions are issued to the vehicle control system. In an example embodiment, the request for wheel-based actions may be issued in response to a request for friction determination and detection of acceleration events. More specifically, in some embodiments, the friction estimation system 150 may issue one or more requests to the vehicle controller 106 to apply differential braking forces at two or more wheel assemblies of the autonomous vehicle. The vehicle controller 106 may issue one or more requests and / or commands to initiate wheel-based actions to an appropriate vehicle control system, such as a braking system. For example, the friction detection controller 402 may issue a request for differential braking actions during a stop, or when the vehicle is moving in a manner that is not perceptible to the user.

[0167] At 704, a differential braking force is applied to at least two wheel assemblies of the vehicle. In an example embodiment, a differential braking force may be applied as part of an acceleration event that includes a speed reduction. The differential braking force may be applied to cause a greater braking force associated with at least one wheel assembly relative to at least another wheel assembly that is also receiving a braking force. A larger braking force may be applied to enable a friction force determination to be determined using a larger braking force while avoiding or minimizing interference with passengers in the vehicle. For example, the vehicle computing system may apply a larger braking force to one wheel in order to cause the corresponding tire to slip or slide relative to the driving surface. The larger braking force may be increased over time until slip or sliding of the tire is detected. A smaller braking force may be applied to the other wheels so that no slip or sliding is experienced. In this way, passengers may not perceive the slip or sliding associated with the larger braking force.

[0168] At 706 and 708, operational data associated with wheel-based actions during the detected acceleration event is obtained. At 706, wheel rotation data associated with the wheel assembly receiving the larger braking force may be received. The wheel rotation data may indicate the degree of rotation of the wheel during tire slippage or the degree of rotation within the entire larger braking force application. Determining the wheel rotation data at 706 is optional. In some examples, the wheel rotation data may be determined using one or more wheel encoder sensors. At step 708, braking force data associated with the wheel assembly receiving the larger braking force may be obtained. The braking force data may indicate the braking force applied to the wheel assembly during tire slippage, or may indicate the braking force applied within the entire application of the larger braking force. In some examples, the braking force data may be measured by a sensor of the brake assembly. For example, the sensor may measure the output of the motor that applies the input braking force. Examples of braking force include input braking force and the resulting wheel torque associated with the input braking force.

[0169] At 710, the maximum wheel rotation of the wheel assembly receiving the greater braking force is determined. At 712, the maximum braking force of the wheel assembly receiving the greater braking force is determined.

[0170] At 714, the friction of the driving surface is determined based at least in part on the maximum wheel rotation and / or the maximum braking force. For example, a calculation may be performed in response to a greater wheel rotation or a greater maximum braking force resulting in a higher friction estimate for the surface. A wheel may be expected to rotate more under braking pressure when contacting a higher friction surface than when contacting a lower friction surface. Similarly, a smaller braking force may be expected to cause the tire to slip when contacting a lower friction surface. In an example embodiment, data indicative of the friction of the driving surface is generated at 714.

[0171] Fig.14 750 is a flow chart depicting an example process 750 for determining friction associated with propulsion-based wheel motion during a detected acceleration event in accordance with an example embodiment of the disclosed technology. In some examples, process 750 may be performed by friction estimation system 150 of vehicle computing system 102. Fig.14 Specific examples are depicted in which differential propulsion forces are applied. However, it should be noted that differential propulsion forces are not required. For example, one or more input propulsion forces may be applied to one or more wheel assemblies as part of an action based on propelling the wheels. In some examples, a single propulsion force is applied to one wheel assembly. In other examples, a single propulsion force is applied to multiple wheel assemblies. In still other examples, multiple propulsion forces may be applied.

[0172] At 752, one or more requests for wheel-based propulsion actions are issued to the vehicle control system. In an example embodiment, the request for wheel-based actions may be issued in response to a request for friction determination and detection of an acceleration event. More specifically, in some embodiments, the friction estimation system 150 may issue one or more requests to the vehicle controller 106 to apply differential propulsion forces at two or more wheel assemblies of the autonomous vehicle. The vehicle controller 106 may issue one or more requests and / or commands to initiate wheel-based actions to an appropriate vehicle control system, such as a propulsion system. For example, the friction detection controller 402 may issue a request for differential propulsion actions as the vehicle increases speed when leaving a stop sign or intersection.

[0173] At 754, a differential propulsion force is applied to at least two wheel assemblies of the vehicle. In an example embodiment, a differential propulsion force may be applied as part of an event that includes an increase in speed. The differential propulsion force may be applied to cause a greater propulsion force associated with at least one wheel assembly relative to at least another wheel assembly that is also receiving propulsion force. A larger propulsion force may be applied to enable the use of a larger propulsion force to determine friction force determination while avoiding or minimizing interference with passengers in the vehicle. For example, the vehicle computing system may apply a larger propulsion force to one wheel in order to cause the corresponding tire to slip or slide relative to the driving surface. The larger propulsion force may increase over time until the slip or slide of the tire is detected. A smaller propulsion force may be applied to the other wheels so that no slip or slide is experienced. In this way, the passenger may not perceive the slip or slide associated with the larger propulsion force. In some examples, one or more wheels may receive a braking force when the differential propulsion force is applied.

[0174] At 756 and 758, operation data associated with wheel-based actions during detected acceleration events are obtained. At 756, wheel rotation data may be received, such as wheel rotation data associated with a wheel assembly receiving a larger propulsion force. The wheel rotation data may indicate the degree of rotation of the wheel during tire slippage or the degree of rotation within the entire larger propulsion force application. Determining the wheel rotation data at 706 is optional. In some examples, the wheel rotation data may be determined using one or more wheel encoder sensors. At 758, propulsion force data associated with a wheel assembly receiving a larger propulsion force may be obtained. The propulsion force data may indicate the propulsion force applied to the wheel assembly during tire slippage or may indicate the propulsion force applied within the entire application of a larger braking force. In some examples, the propulsion force data may be measured by a sensor of the propulsion assembly. In other examples, the propulsion force data may be determined based on the output of the propulsion system, such as the amount of rotational force applied to the wheel. Examples of propulsion force include input propulsion force and the resulting wheel torque associated with the input propulsion force.

[0175] At 760, the maximum wheel rotation of the wheel assembly receiving the greater propulsive force is determined. At 762, the maximum propulsive force of the wheel assembly receiving the greater propulsive force is determined.

[0176] At 764, the friction of the driving surface is determined based at least in part on the maximum wheel rotation and / or the maximum propulsive force. For example, a calculation may be performed in response to a lower wheel rotation or a lower maximum wheel torque resulting in a higher friction estimate of the surface. It may be expected that the wheel will rotate more under the same propulsive force when contacting a lower friction surface (rather than contacting a higher friction surface). Similarly, it may be expected that a smaller propulsive force will cause the tire to slip when contacting a lower friction surface (rather than contacting a higher friction surface). In an example embodiment, data indicating the friction of the driving surface is generated at 764.

[0177] Fig.15 Depicted is a block diagram of an example computing system 1000 according to an example embodiment of the invention. Example computing system 1000 includes a computing system 1002 and a machine learning computing system 1030 communicatively coupled via a network 1080.

[0178] In some embodiments, as part of the operation of the vehicle, the computing system 1002 may perform a friction determination process, provide data indicative of a friction estimate, and / or use friction data. In some embodiments, the computing system 1002 may perform friction estimates using a machine learning model. In some embodiments, the computing system 1002 may be included in an autonomous or non-autonomous vehicle. For example, the computing system 1002 may be on a vehicle. In some embodiments, the computing system 1002 may be used to implement the vehicle computing system 102. In other embodiments, the computing system 1002 may not be located on a vehicle. For example, the computing system 1002 may operate offline to obtain sensor data and perform friction estimates. The computing system 1002 may include one or more distinct physical computing devices.

[0179] The computing system 1002 includes one or more processors 1012 and memory 1014. The one or more processors 1012 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be one processor or multiple processors operatively connected. The memory 1014 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, one or more memory devices, flash memory devices, etc., and combinations thereof.

[0180] The memory 1014 may store information that may be accessed by the one or more processors 1012. For example, the memory 1014 (e.g., one or more non-transitory computer-readable storage media, memory devices) may store data 1016 that may be obtained, received, accessed, written, manipulated, created, and / or stored. The data 1016 may include, for example, images or other sensor data captured by one or more sensors, machine learning models, etc., as described herein. In some implementations, the computing device 1002 may obtain the data from one or more memory devices that are remote from the computing system 1002.

[0181] The memory 1014 may also store computer-readable instructions 1018 that may be executed by one or more processors 1012. The instructions 1018 may be software written in any suitable programming language or may be implemented in hardware. Additionally or alternatively, the instructions 1018 may be a separate thread that is executed logically and / or virtually on the processor 1012.

[0182] For example, the memory 1014 may store instructions 1018 that, when executed by the one or more processors 1012, cause the one or more processors 1012 to perform any of the operations and / or functions described herein, including, for example, generating a machine learning model, generating an indication of friction, etc.

[0183] According to one aspect of the invention, computing system 1002 may store or include one or more machine learning models 1010. As an example, machine learning model 1010 may be or may otherwise include various machine learning models, such as, for example, a neural network (e.g., a deep neural network) or other types of models, including linear models and / or nonlinear models. Example neural networks include feedforward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks.

[0184] In some implementations, computing system 1002 may receive one or more machine learning models 1010 from machine learning computing system 1030 via network 1080, and may store one or more machine learning models 1010 in memory 1014. Computing system 1002 may then use or otherwise implement one or more machine learning models 1010 (e.g., via processor 1012). In particular, computing system 1002 may implement machine learning model 1010 to determine friction data based on sensor data.

[0185] The machine learning computing system 1030 includes one or more processors 1032 and memory 1034. The one or more processors 1032 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be one processor or multiple processors operably connected. The memory 1034 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, one or more memory devices, flash memory devices, etc., and combinations thereof. In some embodiments, the machine learning computing system 1030 may be used to implement the vehicle computing system 102.

[0186] The memory 1034 may store information that may be accessed by the one or more processors 1032. For example, the memory 1034 (e.g., one or more non-transitory computer-readable storage media, memory devices) may store data 1036 that may be obtained, received, accessed, written, manipulated, created, and / or stored. The data 1036 may include, for example, machine learning models and flow charts, as described herein. In some implementations, the machine learning computing device 1030 may obtain data from one or more memory devices remote from the machine learning computing system 1030.

[0187] The memory 1034 may also store computer-readable instructions 1038 that may be executed by one or more processors 1032. The instructions 1038 may be software written in any suitable programming language or may be implemented in hardware. Additionally or alternatively, the instructions 1038 may be a separate thread that is executed logically and / or virtually on the processor 1032.

[0188] For example, memory 1034 may store instructions 1038 that, when executed by one or more processors 1032, cause one or more processors 1032 to perform any of the operations and / or functions described herein, including, for example, determining friction data and controlling the operations and / or functions of the autonomous vehicle based on data indicating the friction of a driving surface associated with the autonomous vehicle.

[0189] In some embodiments, the machine learning computing system 1030 includes one or more server computing devices. If the machine learning computing system 1030 includes multiple server computing devices, such server computing devices may operate according to various computing architectures, including, for example, sequential computing architectures, parallel computing architectures, or some combination thereof.

[0190] In addition to or in lieu of machine learning model 1010 at computing system 1002, machine learning computing system 1030 may include one or more machine learning models 1040. As an example, machine learning model 1040 may be or may otherwise include a variety of machine learning models, such as, for example, a neural network (e.g., a deep neural network) or other types of models, including linear models and / or nonlinear models. Example neural networks include feedforward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks.

[0191] As an example, machine learning computing system 1030 can communicate with computing system 1002 according to a client-server relationship. For example, machine learning computing system 1030 can implement machine learning model 1040 to provide a network service to computing system 1002. For example, the network service can provide an indication of a friction force estimate in response to sensor data and / or other data received from the autonomous vehicle.

[0192] Thus, machine learning model 1010 may be located and used at computing system 1002 , and / or machine learning model 1040 may be located and used at machine learning computing system 1030 .

[0193] In some embodiments, the machine learning computing system 1030 and / or the computing system 1002 may train the machine learning model 1010 and / or 1040 using a model trainer 1060. The model trainer 1060 may train the machine learning model 1010 and / or 1040 using one or more training or learning algorithms. One example training technique is back propagation error. In some embodiments, the model trainer 1060 may perform supervised training techniques using a set of labeled training data. In other embodiments, the model trainer 1060 may perform unsupervised training techniques using a set of unlabeled training data. The model trainer 1060 may perform several generalization techniques to improve the generalization ability of the trained model. Generalization techniques include weight decay, dropout, or other techniques.

[0194] In particular, model trainer 1060 can train machine learning models 1010 and / or 1040 based on a set of training data 1062. Training data 1062 can include, for example, ground truth data, annotations including sensor data portions, and / or vehicle state data. Model trainer 1060 can be implemented in hardware, firmware, and / or software that controls one or more processors.

[0195] In some examples, model trainer 160 may train machine learning models 1010 and / or 1040 configured to generate data indicating friction associated with a driving surface. In some examples, machine learning models 1010 and / or 1040 are trained using sensor data that has been labeled or otherwise annotated as having a correspondence to a surface having a particular friction associated therewith. By way of example, collected sensor data associated with a surface having measured friction may be labeled to indicate that it corresponds to a particular level of friction or friction measurement. In some instances, the labeling may be a simple annotation that the sensor data corresponds to an active training data set.

[0196] The computing system 1002 may also include a network interface 1024 for communicating with one or more systems or devices, including systems or devices located remotely from the computing system 1002. The network interface 1024 may include any circuits, components, software, etc. for communicating with one or more networks (e.g., 1080). In some implementations, the network interface 1024 may include, for example, one or more of a communication controller, a receiver, a transceiver, a transmitter, a port, a conductor, software, and / or hardware for communicating data. Similarly, the machine learning computing system 1030 may include a network interface 1064.

[0197] The network 1080 may be any type of network or combination of networks that allows communication between devices. In some embodiments, the network may include one or more of a local area network, a wide area network, the Internet, a secure network, a cellular network, a mesh network, a peer-to-peer communication link, and / or some combination thereof, and may include any number of wired or wireless links. Communication via the network 1080 may be achieved, for example, via a network interface using any type of protocol, protection scheme, encoding, format, encapsulation, etc.

[0198] Fig.15 An example computing system 1000 that can be used to implement the present invention is described. Other computing systems can also be used. For example, in some embodiments, computing system 1002 can include model trainer 1060 and training data 1062. In such embodiments, machine learning model 1010 can be trained at computing system 1002 and used locally on computing system 1002. As another example, in some embodiments, computing system 1002 is connected to other computing systems.

[0199] In addition, components illustrated and / or discussed as included in one of computing systems 1002 or 1030 may instead be included in the other of computing systems 1002 or 1030. Such configurations may be implemented without departing from the scope of the present invention. The use of computer-based systems allows a large number of possible configurations, combinations, and divisions of tasks and functionality between and among components. Computer-implemented operations may be performed on a single component or across multiple components. Computer-implemented tasks and / or operations may be performed sequentially or in parallel. Data and instructions may be stored in a single memory device or stored across multiple memory devices.

[0200] The techniques discussed herein refer to servers, databases, software applications, and other computer-based systems, and also to the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a large number of possible configurations, combinations, and divisions of tasks and functionality between and among components. For example, the processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0201] Although the subject matter of the present invention is described with respect to various specific example embodiments of the subject matter of the present invention, each example is provided by way of explanation and does not limit the present invention. After understanding the foregoing, those skilled in the art can easily produce substitutions, changes and equivalents to such embodiments. Therefore, it should be easily understood by those skilled in the art that the present invention does not exclude the inclusion of such modifications, changes and / or additions to the subject matter of the present invention. For example, a feature described or described as part of one embodiment can be used together with another embodiment to produce yet another embodiment. Therefore, it is desired that the present invention encompasses such substitutions, changes and equivalents.

Claims

1. A computing system comprising: one or more processors; and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising: issuing a request by a friction detection controller of an autonomous vehicle for a friction estimate associated with a surface on which the autonomous vehicle is traveling; detecting at least one of a planned acceleration, deceleration, or stop associated with the autonomous vehicle based at least in part on at least one of map data, motion planning data, or route planning data associated with the autonomous vehicle; In response to the request, performing a wheel-based action, wherein the wheel-based action includes providing an input torque at a steering system of the autonomous vehicle when the autonomous vehicle is stopped, and the input torque increases over time; obtaining operational data describing the wheel-based action, wherein the operational data indicates at least one of the input torque, a steering force caused by the input torque, or a steering displacement caused by the input torque; determining the friction force estimate based at least in part on the operational data, including determining data indicative of a peak steering force based on analyzing changes in the steering force relative to changes in the steering displacement; and The autonomous vehicle is controlled based at least in part on the friction estimate.

2. The computing system of claim 1 , wherein controlling the autonomous vehicle based at least in part on the friction force estimate comprises at least one of: generating a motion plan based at least in part on the friction estimate; or A route plan is generated based at least in part on the friction estimate.

3. The computing system of claim 2, further comprising: generating at least one motion planning constraint based at least in part on the friction estimate; and At least one route planning constraint is generated based at least in part on the friction estimate.

4. A computing system according to claim 1, wherein detecting at least one of the acceleration, the deceleration, or the stopping associated with the autonomous vehicle is based at least in part on motion planning data associated with the autonomous vehicle.

5. The computing system of claim 1 , wherein the operations further comprise: Data indicative of a confidence level associated with the friction force estimate is determined.

6. The computing system of claim 5, wherein the operations further comprise: The confidence level is reduced based on a time since the operational data was obtained.

7. The computing system of claim 1, wherein the wheel-based motion is associated with movement of at least one tire of the autonomous vehicle relative to the surface.

8. The computing system of claim 7, wherein: The steering displacement includes at least one of a wheel rotation or a steering angle.

9. The computing system of claim 7, wherein: The wheel-based action includes providing one or more input braking forces to a braking system of the autonomous vehicle as part of reducing a speed of the autonomous vehicle; The operational data includes data indicative of at least one of the input braking force, a wheel torque associated with at least one of the input braking forces, or a wheel rotation associated with at least one of the input braking forces; and Determining the friction force estimate is based on the data indicative of at least one of the input braking force, the wheel torque, or the wheel rotation.

10. The computing system of claim 9, wherein: The one or more input braking forces increase over time; and The operations further include determining an amount of the one or more input braking forces sufficient to cause a tire corresponding to the first wheel assembly to slip.

11. The computing system of claim 9, wherein the operations further comprise rotating between a plurality of wheel assemblies receiving the one or more input braking forces.

12. The computing system of claim 7, wherein: The wheel-based action includes providing one or more input propulsion forces to a propulsion system of the autonomous vehicle; The operational data includes data indicative of at least one of the input propulsive force, a wheel torque associated with at least one of the input propulsive force, or a wheel rotation associated with at least one of the input propulsive force; and Determining the friction force estimate is based on the data indicative of at least one of the input propulsion force, the wheel torque, or the wheel rotation.

13. The computing system of claim 1, wherein detecting at least one of the acceleration, the deceleration, or the stop associated with the autonomous vehicle is based at least in part on map data associated with the autonomous vehicle.

14. The computing system of claim 1, wherein: Detecting at least one of the acceleration, the deceleration, or the stopping associated with the autonomous vehicle is based at least in part on sensor data.

15. The computing system of claim 1, wherein: Determining the friction estimate includes determining whether the friction associated with the surface satisfies one or more thresholds.

16. An autonomous vehicle, comprising: one or more processors; and One or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: issuing, by a friction detection controller of the autonomous vehicle, a request for a friction estimate associated with a surface on which the autonomous vehicle is traveling; detecting at least one of a planned acceleration, deceleration, or stop associated with an autonomous vehicle based at least in part on at least one of map data, motion planning data, or route planning data associated with the autonomous vehicle; In response to the request, performing a wheel-based action, wherein the wheel-based action includes providing an input torque at a steering system of the autonomous vehicle when the autonomous vehicle is stopped, and the input torque increases over time; obtaining operational data describing the wheel-based action, wherein the operational data indicates at least one of the input torque, a steering force caused by the input torque, or a steering displacement caused by the input torque; determining the friction force estimate based at least in part on the operational data, including determining data indicative of a peak steering force based on analyzing changes in the steering force relative to changes in the steering displacement; and At least one constraint of the autonomous vehicle is generated based at least in part on the friction estimate.

17. An autonomous vehicle according to claim 16, wherein the at least one constraint includes at least one of a motion planning constraint, a map constraint, or an operating constraint.

18. A computer-implemented method comprising: issuing a request by a friction detection controller of an autonomous vehicle for a friction estimate associated with a surface on which the autonomous vehicle is traveling; detecting at least one of a planned acceleration, deceleration, or stop associated with the autonomous vehicle based at least in part on at least one of map data, motion planning data, or route planning data associated with the autonomous vehicle; In response to the request, performing a wheel-based action, wherein the wheel-based action includes providing an input torque at a steering system of the autonomous vehicle when the autonomous vehicle is stopped, and the input torque increases over time; obtaining operational data describing the wheel-based action, wherein the operational data indicates at least one of the input torque, a steering force caused by the input torque, or a steering displacement caused by the input torque; determining the friction force estimate based at least in part on the operational data, including determining data indicative of a peak steering force based on analyzing changes in the steering force relative to changes in the steering displacement; and The autonomous vehicle is controlled based at least in part on the friction estimate.

19. A computing system comprising: one or more processors; and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising: detecting a stop associated with the vehicle; initiating a steering action of the vehicle during the stopping, the steering action associated with movement of at least one tire of the vehicle relative to a driving surface, wherein the steering action includes providing an input torque at a steering system of the vehicle while the vehicle is stopped; obtaining operational data associated with the steering action during the stop of the vehicle, wherein the operational data indicates at least one of the input torque, a steering force caused by the input torque, or a steering displacement caused by the input torque; determining a friction force associated with the driving surface based at least in part on the peak steering force; and Data indicative of the friction associated with the driving surface is generated.

20. The computing system of claim 19, wherein: The input torque increases over time; and Determining the friction force includes determining data indicative of the peak steering force based on analyzing changes in the steering force relative to changes in the steering displacement.

21. The computing system of claim 20, wherein: The steering displacement includes at least one of a wheel rotation or a steering angle.

22. The computing system of claim 19, wherein: The stop is a future stop associated with the vehicle; and Detecting the future stop is based on at least one of map data, motion planning data, or route planning data associated with the vehicle.

23. The computing system of claim 19, wherein: Detecting the stop is based at least in part on sensor data indicative of the stop.

24. The computing system of claim 19, wherein: Determining the friction force associated with the driving surface includes determining whether the friction force associated with the driving surface satisfies one or more thresholds.

25. The computing system of claim 19, wherein the operations further comprise: Data indicative of a confidence level associated with the data indicative of the friction force is determined.

26. The computing system of claim 25, wherein the operations further comprise: The confidence level is reduced based on a time since the operational data was obtained.

27. The computing system of claim 19, wherein the operations further comprise: A user notification is generated at the vehicle based on the data indicative of the friction force.

28. The computing system of claim 19, wherein the operations further comprise: A signal including the data indicative of the friction forces associated with the driving surface is transmitted to one or more remote computing systems.

29. The computing system of claim 28, wherein: The vehicle is an autonomous vehicle; and The operations further include controlling the autonomous vehicle based at least in part on the data indicative of the friction associated with the driving surface.

30. The computing system of claim 29, wherein controlling the autonomous vehicle based at least in part on the data indicative of the friction force comprises at least one of: generating a motion plan based at least in part on said data indicative of said friction force; or A route plan is generated based at least in part on the data indicative of the friction.

31. A vehicle comprising: one or more processors; and One or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: detecting a stop associated with the vehicle; initiating a steering action of the vehicle during the stopping, the steering action associated with movement of at least one tire of the vehicle relative to a driving surface, wherein the steering action includes providing an input torque at a steering system of the vehicle while the vehicle is stopped; and obtaining operational data associated with the steering action during the stop of the vehicle, wherein the operational data indicates at least one of the input torque, a steering force caused by the input torque, or a steering displacement caused by the input torque; determining a friction force associated with the driving surface based at least in part on the peak steering force; and Data indicative of the friction associated with the driving surface is generated.

32. A computer-implemented method comprising: detecting a stop associated with the vehicle; initiating a steering action of the vehicle during the stopping, the steering action associated with movement of at least one tire of the vehicle relative to a driving surface, wherein the steering action includes providing an input torque at a steering system of the vehicle while the vehicle is stopped; and obtaining operational data associated with the steering action during the stop of the vehicle, wherein the operational data indicates at least one of the input torque, a steering force caused by the input torque, or a steering displacement caused by the input torque; determining a friction force associated with the driving surface based at least in part on the peak steering force; and Data indicative of the friction associated with the driving surface is generated.

Images