System and method for navigating at a safe distance - Patents.com

Abstract

A system and method for navigating a host vehicle is disclosed. In one embodiment, at least one processing device may be programmed to receive an image representing an environment of the host vehicle, determine a planned navigation operation for the host vehicle, analyze the image to identify a target vehicle having a heading toward the host vehicle, and determine a next-state distance between the host vehicle and the target vehicle that will occur if the planned navigation operation is performed. The at least one processing device may further determine a stopping distance for the host vehicle based on a braking rate, maximum acceleration capacity, and current speed of the host vehicle, determine a stopping distance for the target vehicle based on a braking rate, maximum acceleration capacity, and current speed of the target vehicle, and perform the planned navigation operation if the determined next-state distance is greater than the sum of the stopping distances of the host vehicle and the target vehicle.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 718,554, filed August 14, 2018, U.S. Provisional Patent Application No. 62 / 724,355, filed August 29, 2018, U.S. Provisional Patent Application No. 62 / 772,366, filed November 28, 2018, and U.S. Provisional Patent Application No. 62 / 777,914, filed December 11, 2018. All of the above applications are incorporated herein by reference in their entireties.

[0002] The present disclosure relates generally to autonomous vehicle navigation. Additionally, the present disclosure relates to a system and method for navigating subject to potential accident liability constraints. [Background technology]

[0003] As technology continues to evolve, the goal of a fully autonomous vehicle capable of navigating on the road becomes more realistic. An autonomous vehicle may need to consider various factors and make appropriate decisions based on those factors to safely and accurately reach an intended destination. For example, an autonomous vehicle may need to process and interpret visual information (e.g., information captured from a camera), information from radar or lidar, and may also use information obtained from other sources (e.g., GPS devices, speed sensors, accelerometers, suspension sensors, etc.). At the same time, to navigate to a destination, an autonomous vehicle may need to identify its position within a particular road (e.g., a particular lane within a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, observe traffic signals and signs, proceed from one road to another at appropriate intersections or interchanges, and respond to any other conditions that occur or develop during the operation of the vehicle. Furthermore, the navigation system may have to comply with certain imposed constraints. In some cases, those constraints may relate to the interaction between the host vehicle and one or more other objects, such as other vehicles or pedestrians. In other cases, the constraints may relate to responsibility rules that must be followed when performing one or more navigation operations for the host vehicle.

[0004] In the field of autonomous driving, there are two important considerations for a viable autonomous vehicle system. The first consideration is the standardization of safety assurances, including the requirements that all self-driving vehicles must meet to ensure safety, and how those requirements can be verified. The second consideration is scalability, because engineering solutions that cause rising costs may not scale to millions of cars and may hinder widespread or even less widespread adoption of autonomous vehicles. Thus, there is a need for interpretable mathematical models for safety assurances and the design of systems that can scale to millions of cars and still comply with the safety assurance requirements. Summary of the Invention

[0005] Embodiments according to the present disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation features. For example, according to embodiments of the present disclosure, the disclosed system may include one, two, or more cameras that monitor the vehicle's environment. The disclosed system may provide a navigation response, for example, based on an analysis of images captured by one or more of the cameras. The navigation response may also take into account other data, including, for example, Global Positioning System (GPS) data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data.

[0006] In one embodiment, a system for navigating a host vehicle is disclosed. The system may include at least one processing device programmed to: receive at least one image representing an environment of the host vehicle from the image capture device; determine a planned navigation operation for achieving a navigation goal of the host vehicle based on at least one driving policy; analyze the at least one image to identify a target vehicle within the environment of the host vehicle, where the target vehicle's heading is toward the host vehicle; determine a next state distance between the host vehicle and the target vehicle that will occur if the planned navigation operation is performed; determine a host vehicle braking rate, a host vehicle maximum acceleration capacity, and a current speed of the host vehicle; determine a current stopping distance of the host vehicle based on the host vehicle braking rate, the host vehicle maximum acceleration capacity, and the current speed of the host vehicle; determine the target vehicle's current speed, the target vehicle maximum acceleration capacity, and the target vehicle braking rate; determine the target vehicle's stopping distance based on the target vehicle braking rate, the target vehicle maximum acceleration capacity, and the current speed of the target vehicle; and perform the planned navigation operation if the determined next state distance is greater than a sum of the host vehicle stopping distance and the target vehicle stopping distance.

[0007] In one embodiment, a method for navigating a host vehicle is disclosed. The method may include receiving at least one image representing an environment of the host vehicle from an image capture device, determining a planned navigation operation for achieving a navigation goal of the host vehicle based on at least one driving policy, analyzing the at least one image to identify a target vehicle within the environment of the host vehicle, the target vehicle having a heading toward the host vehicle, determining a next state distance between the host vehicle and the target vehicle that will occur if the planned navigation operation is performed, determining a host vehicle braking rate, a host vehicle maximum acceleration capacity, and a current speed of the host vehicle, determining a current stopping distance of the host vehicle based on the host vehicle braking rate, the host vehicle maximum acceleration capacity, and the current speed of the host vehicle, determining a current speed of the target vehicle, a target vehicle maximum acceleration capacity, and a target vehicle braking rate, determining a stopping distance of the target vehicle based on the target vehicle braking rate, the target vehicle maximum acceleration capacity, and the current speed of the target vehicle, and performing the planned navigation operation if the determined next state distance is greater than a sum of the stopping distance of the host vehicle and the stopping distance of the target vehicle.

[0008] In one embodiment, a system for navigating a host vehicle is disclosed, the system includes receiving at least one image representative of an environment of the host vehicle from an image capture device, determining a planned navigation operation for achieving a navigation goal of the host vehicle based on at least one driving policy, analyzing the at least one image to identify a target vehicle within the environment of the host vehicle, determining a next state lateral distance between the host vehicle and the target vehicle that would occur if the planned navigation operation were performed, determining a maximum yaw rate capability of the host vehicle, a maximum change in turning radius capability of the host vehicle, and a current lateral speed of the host vehicle, and determining a maximum yaw rate capability of the host vehicle, a maximum change in turning radius capability of the host vehicle, and a current lateral speed of the host vehicle. The navigation system may include at least one processing device programmed to: determine a lateral braking distance of the host vehicle based on the current lateral speed of the host vehicle, a maximum change in turning radius capability of the target vehicle, and a current lateral speed of the host vehicle; determine a lateral braking distance of the target vehicle based on the current lateral speed of the target vehicle, a maximum yaw rate capability of the target vehicle, and a maximum change in turning radius capability of the target vehicle; determine a lateral braking distance of the target vehicle based on the current lateral speed of the target vehicle, a maximum yaw rate capability of the target vehicle, and a maximum change in turning radius capability of the target vehicle; and perform the planned navigation operation if the determined next state lateral distance is greater than a sum of the lateral braking distance of the host vehicle and the lateral braking distance of the target vehicle.

[0009] In one embodiment, a method for navigating a host vehicle is disclosed, the method includes receiving at least one image representative of an environment of the host vehicle from an image capture device, determining a planned navigation maneuver for achieving a navigation goal of the host vehicle based on at least one driving policy, analyzing the at least one image to identify a target vehicle within the environment of the host vehicle, determining a next state lateral distance between the host vehicle and the target vehicle that would occur if the planned navigation maneuver were performed, determining a maximum yaw rate capability of the host vehicle, a maximum change in turning radius capability of the host vehicle, and a current lateral speed of the host vehicle, and determining a maximum yaw rate capability of the host vehicle. The method may include determining a lateral braking distance of the host vehicle based on yaw rate capability, a maximum change in turning radius capability of the host vehicle, and a current lateral speed of the host vehicle; determining a current lateral speed of the target vehicle, a maximum yaw rate capability of the target vehicle, and a maximum change in turning radius capability of the target vehicle; determining a lateral braking distance of the target vehicle based on the current lateral speed of the target vehicle, a maximum yaw rate capability of the target vehicle, and a maximum change in turning radius capability of the target vehicle; and performing a planned navigation operation if the determined lateral distance of the next state is greater than a sum of the lateral braking distance of the host vehicle and the lateral braking distance of the target vehicle.

[0010] In one embodiment, a system for navigating a host vehicle near a pedestrian crossing is disclosed. The system may include at least one processing device programmed to: receive at least one image representing an environment of the host vehicle from an image capture device, detect a representation of a pedestrian crossing in the at least one image based on an analysis of the at least one image, determine whether a representation of a pedestrian appears in the at least one image based on an analysis of the at least one image, detect a presence of a traffic light in the environment of the host vehicle, determine whether the detected traffic light is associated with the host vehicle and the pedestrian crossing, determine a state of the detected traffic light, determine a proximity of the pedestrian to the detected pedestrian crossing when a representation of a pedestrian appears in the at least one image, determine a planned navigation operation for navigating the host vehicle to the detected pedestrian crossing based on at least one driving policy, where the determination of the planned navigation operation is further based on the determined state of the detected traffic light and the determined proximity of the pedestrian to the detected pedestrian crossing, and cause one or more actuator systems of the host vehicle to perform the planned navigation operation.

[0011] In one embodiment, a method for navigating a host vehicle near a pedestrian crossing is disclosed. The method may include receiving at least one image representing an environment of the host vehicle from an image capture device, detecting a representation of the pedestrian crossing in the at least one image based on an analysis of the at least one image, determining whether a representation of a pedestrian appears in the at least one image based on an analysis of the at least one image, detecting a presence of a traffic light in the environment of the host vehicle, determining whether the detected traffic light is associated with the host vehicle and the pedestrian crossing, determining a state of the detected traffic light, determining a proximity of the pedestrian to the detected pedestrian crossing when a representation of a pedestrian appears in the at least one image, determining a planned navigation operation for navigating the host vehicle to the detected pedestrian crossing based on at least one driving policy, where the determination of the planned navigation operation is further based on the determined state of the detected traffic light and the determined proximity of the pedestrian to the detected pedestrian crossing, and causing one or more actuator systems of the host vehicle to perform the planned navigation operation.

[0012] In one embodiment, a method for navigating a host vehicle near a pedestrian crossing is disclosed. The method may include receiving at least one image representing an environment of the host vehicle from an image capture device, detecting a start and end position of the pedestrian crossing, determining whether a pedestrian is present near the pedestrian crossing based on an analysis of the at least one image, detecting a presence of a traffic light in the environment of the host vehicle, determining whether the traffic light is associated with the host vehicle and the pedestrian crossing, determining a state of the traffic light, determining a navigation operation of the host vehicle near the detected pedestrian crossing based on the association of the traffic light, the determined state of the traffic light, the presence of a pedestrian near the pedestrian crossing, a selected minimum distance between the pedestrian and either the start or end position of the pedestrian crossing, a motion vector of the pedestrian, and causing one or more actuator systems of the host vehicle to perform the navigation operation.

[0013] According to other disclosed embodiments, a non-transitory computer-readable storage medium may store program instructions executable by at least one processing device and performing any of the steps and / or methods described herein.

[0014] The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the scope of the claims. [Brief description of the drawings]

[0015] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various disclosed embodiments.

[0016] [Figure 1] 1 is a diagrammatic representation of an exemplary system according to a disclosed embodiment.

[0017] [Figure 2A] 1 is a side view representation of an exemplary vehicle including a system according to disclosed embodiments.

[0018] [Figure 2B] 2B is a top view representation of the vehicle and system shown in FIG. 2A according to a disclosed embodiment.

[0019] [Figure 2C] 1 is a top view representation of another embodiment of a vehicle including a system according to the disclosed embodiments.

[0020] [Figure 2D] 1 is a top view representation of yet another embodiment of a vehicle including a system according to disclosed embodiments.

[0021] [Figure 2E] 1 is a top view representation of yet another embodiment of a vehicle including a system according to disclosed embodiments.

[0022] [Figure 2F]1 is a diagrammatic representation of an exemplary vehicle control system in accordance with a disclosed embodiment.

[0023] [Figure 3A] 1 is a diagrammatic representation of the interior of a vehicle including a rearview mirror and a user interface of a vehicle imaging system according to disclosed embodiments.

[0024] [Figure 3B] 1 is a diagram of an example camera mount configured to be positioned behind a rearview mirror, facing a vehicle windshield, in accordance with a disclosed embodiment.

[0025] [Figure 3C] 3C is a diagram of the camera mount shown in FIG. 3B from a different perspective, according to a disclosed embodiment.

[0026] [Figure 3D] 1 is a diagram of an example camera mount configured to be positioned behind a rearview mirror, facing a vehicle windshield, in accordance with a disclosed embodiment.

[0027] [Figure 4] FIG. 1 illustrates an example block diagram of a memory configured to store instructions for performing one or more operations in accordance with the disclosed embodiments.

[0028] [Figure 5A] 1 is a flowchart illustrating an example process for generating one or more navigational responses based on monocular image analysis, according to disclosed embodiments.

[0029] [Figure 5B] 1 is a flowchart illustrating an example process for detecting one or more vehicles and / or pedestrians in a set of images, according to disclosed embodiments.

[0030] [Figure 5C]1 is a flowchart illustrating an example process for detecting road markings and / or lane geometry information in a set of images, according to disclosed embodiments.

[0031] [Figure 5D] 1 is a flowchart illustrating an example process for detecting traffic lights in a set of images, according to disclosed embodiments.

[0032] [Figure 5E] 1 is a flowchart of an example process for generating one or more navigational responses based on a vehicle path, according to disclosed embodiments.

[0033] [Figure 5F] 4 is a flowchart illustrating an example process for determining whether a leading vehicle is changing lanes, according to disclosed embodiments.

[0034] [Figure 6] 1 is a flowchart illustrating an example process for generating one or more navigational responses based on stereo image analysis, according to disclosed embodiments.

[0035] [Figure 7] 1 is a flowchart illustrating an example process for generating one or more navigational responses based on an analysis of three sets of images, according to disclosed embodiments.

[0036] [Figure 8] 1 is a block diagram representation of modules that may be implemented by one or more specifically programmed processing devices of a navigation system for an autonomous vehicle according to disclosed embodiments.

[0037] [Figure 9] 1 is a graph of navigation options in accordance with a disclosed embodiment;

[0038] [Figure 10] 1 is a graph of navigation options in accordance with a disclosed embodiment;

[0039] [Figure 11A] 1 illustrates a schematic diagram of navigation options for a host vehicle within a merge area according to a disclosed embodiment. [Figure 11B] 1 illustrates a schematic diagram of navigation options for a host vehicle within a merge area according to a disclosed embodiment. [Figure 11C] 1 illustrates a schematic diagram of navigation options for a host vehicle within a merge area according to a disclosed embodiment.

[0040] [Figure 11D] 1 illustrates a diagrammatic representation of a double merge scenario in accordance with a disclosed embodiment;

[0041] [Figure 11E] 13 illustrates a graph of potentially useful options in a double merge scenario in accordance with disclosed embodiments.

[0042] [Figure 12] 1 illustrates an illustration of a representative image captured of a host vehicle's environment along with potential navigation constraints in accordance with a disclosed embodiment;

[0043] [Figure 13] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment.

[0044] [Figure 14] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment.

[0045] [Figure 15] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment.

[0046] [Figure 16] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment.

[0047] [Figure 17A] 1 illustrates a diagram of a host vehicle navigating into a roundabout in accordance with a disclosed embodiment; [Figure 17B] 1 illustrates a diagram of a host vehicle navigating into a roundabout in accordance with a disclosed embodiment;

[0048] [Figure 18] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment.

[0049] [Figure 19] 1 illustrates an example of a host vehicle traveling on a multi-lane highway, according to disclosed embodiments.

[0050] [Figure 20A] 1 illustrates an example of a vehicle cutting in front of another vehicle, according to disclosed embodiments. [Figure 20B] 1 illustrates an example of a vehicle cutting in front of another vehicle, according to disclosed embodiments.

[0051] [Figure 21] 1 illustrates an example of a vehicle following another vehicle, according to disclosed embodiments.

[0052] [Figure 22] 1 illustrates an example of a vehicle exiting a parking lot and merging onto a potentially busy road, according to disclosed embodiments.

[0053] [Figure 23] 1 illustrates a vehicle traveling on a road, according to a disclosed embodiment;

[0054] [Figure 24A] 1 illustrates an example scenario according to disclosed embodiments. [Figure 24B] 1 illustrates an example scenario according to disclosed embodiments. [Figure 24C] 1 illustrates an example scenario according to disclosed embodiments. [Figure 24D] 1 illustrates an example scenario according to disclosed embodiments.

[0055] [Diagram 25] 1 illustrates an example scenario according to disclosed embodiments.

[0056] [Figure 26] 1 illustrates an example scenario according to disclosed embodiments.

[0057] [Figure 27] 1 illustrates an example scenario according to disclosed embodiments.

[0058] [Figure 28A] 1 illustrates an example scenario of a vehicle following another vehicle, according to disclosed embodiments. [Figure 28B] 1 illustrates an example scenario of a vehicle following another vehicle, according to disclosed embodiments.

[0059] [Figure 29A] 1 illustrates an example of a fault in an interruption scenario in accordance with disclosed embodiments. [Figure 29B] 1 illustrates an example of a fault in an interruption scenario in accordance with disclosed embodiments.

[0060] [Figure 30A] 1 illustrates an example of a fault in an interruption scenario in accordance with disclosed embodiments. [Figure 30B] 1 illustrates an example of a fault in an interruption scenario in accordance with disclosed embodiments.

[0061] [Figure 31A] 1 illustrates an example of an error in a drift scenario according to disclosed embodiments. [Figure 31B]1 illustrates an example of an error in a drift scenario according to disclosed embodiments. [Figure 31C] 1 illustrates an example of an error in a drift scenario according to disclosed embodiments. [Figure 31D] 1 illustrates an example of an error in a drift scenario according to disclosed embodiments.

[0062] [Figure 32A] 1 illustrates an example of a fault in a two-way traffic scenario according to disclosed embodiments. [Figure 32B] 1 illustrates an example of a fault in a two-way traffic scenario according to disclosed embodiments.

[0063] [Figure 33A] 1 illustrates an example of a fault in a two-way traffic scenario according to disclosed embodiments. [Figure 33B] 1 illustrates an example of a fault in a two-way traffic scenario according to disclosed embodiments.

[0064] [Figure 34A] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments. [Figure 34B] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments.

[0065] [Figure 35A] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments. [Figure 35B] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments.

[0066] [Figure 36A] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments. [Figure 36B] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments.

[0067] [Figure 37A] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments. [Figure 37B] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments.

[0068] [Figure 38A] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments. [Figure 38B] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments.

[0069] [Figure 39A] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments. [Figure 39B] 1 illustrates an example of a fault in a route priority scenario in accordance with disclosed embodiments.

[0070] [Figure 40A] 1 illustrates an example of a fault in a traffic light scenario, according to disclosed embodiments. [Figure 40B] 1 illustrates an example of a fault in a traffic light scenario, according to disclosed embodiments.

[0071] [Figure 41A] 1 illustrates an example of a fault in a traffic light scenario, according to disclosed embodiments. [Figure 41B] 1 illustrates an example of a fault in a traffic light scenario, according to disclosed embodiments.

[0072] [Figure 42A] 1 illustrates an example of a fault in a traffic light scenario, according to disclosed embodiments. [Figure 42B] 1 illustrates an example of a fault in a traffic light scenario, according to disclosed embodiments.

[0073] [Figure 43A]1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 43B] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 43C] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments.

[0074] [Figure 44A] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 44B] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 44C] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments.

[0075] [Figure 45A] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 45B] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 45C] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments.

[0076] [Figure 46A] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 46B] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 46C] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments. [Figure 46D] 1 illustrates an example of a vulnerable road user (VRU) scenario, according to disclosed embodiments.

[0077] [Figure 47A] FIG. 1 is a diagram of fault times and appropriate responses according to disclosed embodiments.

[0078] [Figure 47B] FIG. 13 is an illustration of route priorities for routes of different geometries in accordance with disclosed embodiments.

[0079] [Figure 47C] 13A-13C are diagrams of vertical ordering on routes of different geometries according to disclosed embodiments.

[0080] [Figure 47D] FIG. 13 is an illustration of safe longitudinal distance between vehicles according to disclosed embodiments.

[0081] [Figure 47E] 1 is a diagram of a situation in which a vehicle cannot predict the path of another vehicle, in accordance with a disclosed embodiment;

[0082] [Fig.47F] FIG. 1 is an illustration of route priority at a traffic light according to a disclosed embodiment;

[0083] [Figure 47G] FIG. 2 is a diagram of an exemplary unstructured route, consistent with disclosed embodiments.

[0084] [Fig. 47H] 13A-13C are diagrams of exemplary lateral behavior in unstructured conditions according to disclosed embodiments;

[0085] [Fig.47I] 1 is a diagram of exposure and negligence times in a concealed region according to disclosed embodiments;

[0086] [Figure 48A] 1 illustrates an example of a scenario of two vehicles traveling in opposite directions, according to disclosed embodiments.

[0087] [Figure 48B] 1 illustrates an example of a target vehicle traveling toward a host vehicle, according to disclosed embodiments.

[0088] [Figure 49] 1 illustrates an example of a host vehicle maintaining a safe longitudinal distance in accordance with disclosed embodiments.

[0089] [Figure 50A] 1 provides a flowchart illustrating an exemplary process for maintaining a safe vertical distance, according to disclosed embodiments. [Figure 50B] 1 provides a flowchart illustrating an exemplary process for maintaining a safe vertical distance, according to disclosed embodiments.

[0090] [Figure 51A] 1 illustrates an example of a scenario of two vehicles spaced laterally from each other, according to disclosed embodiments.

[0091] [Figure 51B] 1 illustrates an example of a host vehicle maintaining a safe lateral distance in accordance with disclosed embodiments.

[0092] [Figure 52A] 1 illustrates an example of a host vehicle performing a planned navigation operation, according to disclosed embodiments.

[0093] [Figure 52B] 1 illustrates an example of a host vehicle determining whether to perform a navigation operation.

[0094] [Figure 53A] 1 provides a flowchart illustrating an exemplary process for maintaining a safe lateral distance, according to disclosed embodiments. [Figure 53B] 1 provides a flowchart illustrating an exemplary process for maintaining a safe lateral distance, according to disclosed embodiments.

[0095] [Figure 54] 1 is a schematic diagram of a road including a pedestrian crossing, according to a disclosed embodiment;

[0096] [Figure 55A] 1 is a schematic diagram of possible navigation actions performed by a vehicle traveling along a road, according to a disclosed embodiment; [Figure 55B] 1 is a schematic diagram of possible navigation actions performed by a vehicle traveling along a road, according to a disclosed embodiment;

[0097] [Figure 55C] 1 illustrates an example of estimating distance from a pedestrian to a crosswalk according to a disclosed embodiment. [Fig. 55D] 1 illustrates an example of estimating distance from a pedestrian to a crosswalk according to a disclosed embodiment.

[0098] [Figure 56] 1 is a flowchart illustrating a process for navigating a vehicle near a pedestrian crossing according to a disclosed embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0099] The following detailed description refers to the accompanying drawings. Whenever possible, the same reference numbers are used in the drawings and the following description to refer to the same or similar parts. Although several exemplary embodiments are described herein, modifications, adaptations and other implementations are possible. For example, substitutions, additions or modifications may be made to the components shown in the drawings, and the exemplary methods described herein may be modified by substituting, reordering, deleting or adding steps of the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. Instead, the appropriate scope is defined by the appended claims.

[0100] Autonomous Vehicle Overview

[0101] As used throughout this disclosure, the term "autonomous vehicle" refers to a vehicle that can perform at least one navigation change without driver input. A "navigation change" refers to one or more changes in steering, braking, or acceleration / deceleration of the vehicle. To be autonomous, a vehicle need not be fully automatic (e.g., fully operable without a driver or driver input). Rather, an autonomous vehicle includes a vehicle that can operate under driver control during certain periods of time and without driver control during other periods of time. An autonomous vehicle can also include a vehicle that controls only some aspects of vehicle navigation, such as steering (e.g., to maintain a vehicle course between vehicle lane constraints), or controls some steering actions under certain circumstances (but not under all circumstances), but leaves other aspects (e.g., braking or braking under certain circumstances) to the driver. In some cases, an autonomous vehicle may handle some or all aspects of braking, speed control, and / or steering of the vehicle.

[0102] Because human drivers typically rely on visual cues and observations to control the vehicle, the transportation infrastructure is built accordingly, with lane markings, traffic signs, and traffic lights designed to provide visual information to the driver. In light of these design features of the transportation infrastructure, an autonomous vehicle may include a camera and a processing unit that analyzes visual information captured from the vehicle's environment. The visual information may include, for example, images depicting components of the transportation infrastructure (e.g., lane markings, traffic signs, traffic lights, etc.) and other obstacles (e.g., other vehicles, pedestrians, debris, etc.) observable by the driver. Additionally, an autonomous vehicle may use stored information when navigating, such as information that provides a model of the vehicle's environment. For example, the vehicle may use GPS data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data to provide information related to the vehicle's environment while the vehicle is traveling, and the vehicle (and other vehicles) may use the information to locate itself in the model. Some vehicles may also be capable of vehicle-to-vehicle communication, sharing information, peer vehicle changes of hazards or changes in the vehicle's surroundings, etc.

[0103] System Overview

[0104] FIG. 1 is a block diagram representation of a system 100 according to an exemplary embodiment disclosed. The system 100 may include various components depending on the requirements of a particular implementation. In some embodiments, the system 100 may include a processing unit 110, an image acquisition unit 120, a position sensor 130, one or more memory units 140, 150, a map database 160, a user interface 170, and a wireless transceiver 172. The processing unit 110 may include one or more processing devices. In some embodiments, the processing unit 110 may include an application processor 180, an image processor 190, or any other suitable processing device. Similarly, the image acquisition unit 120 may include any number of image acquisition devices and components depending on the requirements of a particular application. In some embodiments, the image acquisition unit 120 may include one or more image capture devices (e.g., cameras, CCDs, any other type of image sensor), such as an image capture device 122, an image capture device 124, an image capture device 126, etc. System 100 may also include a data interface 128 that communicatively connects processing unit 110 to image acquisition unit 120. For example, data interface 128 may include any wired and / or wireless link for transmitting image data acquired by image acquisition unit 120 to processing unit 110.

[0105] Wireless transceiver 172 may include one or more devices configured to exchange transmissions with one or more networks (e.g., cellular, Internet, etc.) over a wireless interface through the use of radio frequencies, infrared frequencies, magnetic fields, or electric fields. Wireless transceiver 172 may transmit and / or receive data using any known standard (e.g., Wi-Fi, Bluetooth, Bluetooth Smart, 802.15.4, ZigBee, etc.). Such transmissions may include communications from the host vehicle to one or more remotely located servers. Such transmissions may include communications (one-way or two-way) between the host vehicle and one or more target vehicles in the host vehicle's environment (e.g., to facilitate coordinating the host vehicle's navigation in light of or with target vehicles in the host vehicle's environment), as well as broadcast transmissions to unspecified recipients in the vicinity of the transmitting vehicle.

[0106] Both application processor 180 and image processor 190 may include various types of hardware-based processing devices. For example, either or both of application processor 180 and image processor 190 may include a microprocessor, a preprocessor (such as an image preprocessor), a graphics processor, a central processing unit (CPU), support circuits, a digital signal processor, an integrated circuit, memory, or any other type of device suitable for executing applications and processing and analyzing images. In some embodiments, application processor 180 and / or image processor 190 may include any type of single-core or multi-core processor, mobile device microcontroller, central processing unit, etc. A variety of processing devices may be used, including, for example, processors available from manufacturers such as Intel®, AMD®, etc., and may include a variety of architectures (e.g., x86 processor, ARM®, etc.).

[0107] In some embodiments, application processor 180 and / or image processor 190 may include any of the EyeQ series of processors available from Mobileye®. These processor designs include multiple processing units, each with a local memory and instruction set. Such processors may include video inputs that receive image data from multiple image sensors, and may also include video output capabilities. In one example, the EyeQ2® uses 90 nm-micron technology operating at 332 MHz. The EyeQ2® architecture consists of two floating-point hyper-threaded 32-bit RISC CPUs (MIPS32® 34K® cores), five Vision Computation Engines (VCEs), three Vector Microcode Processors (VMPs®), a Denali 64-bit mobile DDR controller, a 128-bit internal acoustic interconnect, dual 16-bit video input and 18-bit video output controllers, a 16-channel DMA, and several peripherals. The MIPS34K CPU manages the five VCEs, three VMPs™ and DMA, a second MIPS34K CPU and multi-channel DMA, and other peripherals. The five VCEs, three VMPs, and the MIPS34K CPU can perform intensive vision calculations required by feature-rich bundle applications. In another example, the disclosed embodiments may use the EyeQ3, a third generation processor that is six times more powerful than the EyeQ2. In another example, the EyeQ4 and / or EyeQ5 may be used with the disclosed embodiments. Of course, newer or future EyeQ processing devices may be used with the disclosed embodiments.

[0108] Any of the processing devices disclosed herein can be configured to perform a particular function. Configuring a processing device, such as any of the described EyeQ processors or other controllers or microprocessors, to perform a particular function may include programming computer-executable instructions and providing the processing device for execution during operation of the processing device. In some embodiments, configuring a processing device may include directly programming architectural instructions into the processing device. In other embodiments, configuring a processing device may include storing executable instructions on a memory accessible by the processing device during operation. For example, the processing device may access the memory during operation to retrieve and execute the stored instructions. In any case, a processing device configured to perform the sensing, image analysis, and / or navigation functions disclosed herein represents a dedicated hardware-based system that controls multiple hardware-based components of a host vehicle.

[0109] 1 shows two separate processing devices included in processing unit 110, more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to accomplish the tasks of application processor 180 and image processor 190. In other embodiments, these tasks may be performed by three or more processing devices. Furthermore, in some embodiments, system 100 may include one or more of processing units 110 without including other components, such as image acquisition unit 120.

[0110] The processing unit 110 may include various types of devices. For example, the processing unit 110 may include various devices such as a controller, an image preprocessor, a central processing unit (CPU), support circuits, digital signal processors, integrated circuits, memory, or any other type of device that processes and analyzes images. The image preprocessor may include a video processor that captures, digitizes, and processes images from an image sensor. The CPU may include any number of microcontrollers or microprocessors. The support circuits may be any number of circuits commonly known in the art, including cache, power, clock, and input / output circuits. The memory may store software that, when executed by the processor, controls the operation of the system. The memory may include databases and image processing software. The memory may include any number of random access memories, read-only memories, flash memories, disk drives, optical storage devices, tape storage devices, removable storage devices, and other types of storage devices. In one example, the memory may be separate from the processing unit 110. In another example, the memory may be integrated into the processing unit 110.

[0111] Each memory 140, 150 may contain software instructions that, when executed by a processor (e.g., application processor 180 and / or image processor 190), may control the operation of various aspects of system 100. These memory units may include various databases and image processing software as well as trained systems, such as neural networks or deep neural networks. The memory units may include random access memory, read-only memory, flash memory, disk drives, optical storage devices, tape storage devices, removable storage devices, and / or any other type of storage device. In some embodiments, memory units 140, 150 may be separate from application processor 180 and / or image processor 190. In other embodiments, these memory units may be integrated into application processor 180 and / or image processor 190.

[0112] Position sensor 130 may include any type of device suitable for determining a location associated with at least one component of system 100. In some embodiments, position sensor 130 may include a GPS receiver. Such a receiver may determine the location and velocity of a user by processing signals broadcast by Global Positioning System satellites. Position information from position sensor 130 may be provided to application processor 180 and / or image processor 190.

[0113] In some embodiments, system 100 may include components such as a speed sensor (e.g., a speedometer) for measuring the speed of vehicle 200. System 100 may also include one or more accelerometers (single-axis or multi-axis) for measuring the acceleration of vehicle 200 along one or more axes.

[0114] The memory units 140, 150 may contain a database or any other organized format of data indicating the locations of known landmarks. Sensory information of the environment (images, radar signals, depth information from lidar or stereo processing of two or more images, etc.) may be processed along with location information such as GPS coordinates and vehicle ego motion to determine the vehicle's current location relative to known landmarks and to refine the vehicle's location. Certain aspects of this technology are included in a location technology known as REM™, sold by the assignee of the present application.

[0115] User interface 170 may include any device suitable for providing information or receiving input from one or more users of system 100. In some embodiments, user interface 170 may include user input devices including, for example, a touch screen, a microphone, a keyboard, a pointer device, a track wheel, a camera, a knob, buttons, etc. Using such input devices, a user may be able to provide information input or commands to system 100 by typing instructions or information, providing voice commands, selecting on-screen menu options using buttons, pointer or eye tracking functions, or through any other suitable technique for communicating information to system 100.

[0116] User interface 170 may comprise one or more processing devices configured to provide information to or receive information from a user and process that information, e.g., for use by application processor 180. In some embodiments, such processing devices may execute instructions to recognize and track eye movements, receive and interpret voice commands, recognize and interpret touches and / or gestures made on a touchscreen, respond to keyboard entries or menu selections, etc. In some embodiments, user interface 170 may include a display, a speaker, a haptic device, and / or any other device that provides output information to a user.

[0117] Map database 160 may include any type of database that stores map data useful to system 100. In some embodiments, map database 160 may include data related to the locations in a reference coordinate system of various items, including roads, water features, geographic features, businesses, points of interest, restaurants, gas stations, etc. Map database 160 may store not only the locations of such items, but also descriptors related to those items, including, for example, names associated with any of the stored features. In some embodiments, map database 160 may be physically located with other components of system 100. Alternatively or additionally, map database 160 or portions thereof may be located remotely with respect to other components of system 100 (e.g., processing unit 110). In such embodiments, information from map database 160 may be downloaded to a network (e.g., via a cellular network and / or the Internet, etc.) via a wired or wireless data connection. In some cases, map database 160 may store a sparse data model that includes polynomial representations of specific road features (e.g., lane marks) or a target trajectory of the host vehicle. The map database 160 may also include stored representations of various recognized landmarks that may be used to determine or update the known position of the host vehicle relative to the target trajectory. The landmark representations may include data fields such as the landmark type and the landmark location, among other potential identifiers.

[0118] Image capture devices 122, 124, and 126 may each include any type of device suitable for capturing at least one image from an environment. Furthermore, any number of image capture devices may be used to obtain images for input to the image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. Image capture devices 122, 124, and 126 are further described below with reference to Figures 2B-2E.

[0119] One or more cameras (e.g., image capture devices 122, 124, and 126) may be part of a detection block included on the vehicle. A variety of other sensors may be included in the detection block, and any or all of the sensors may be utilized to develop a sensed navigation state of the vehicle. In addition to cameras (forward facing, side facing, rear facing, etc.), other sensors may be included in the detection block, such as radar, lidar, acoustic sensors, etc. Additionally, the detection block may include one or more components configured to communicate, transmit, and receive information related to the vehicle's environment. For example, such components may include a wireless transceiver (RF, etc.) that may receive sensor-based information or any other type of information related to the host vehicle's environment from a source located remotely relative to the host vehicle. Such information may include sensor output information or related information received from a vehicle system other than the host vehicle. In some embodiments, such information may include information received from a remote computing device, a centralized server, etc. Additionally, the cameras may take many different configurations, such as a single camera unit, multiple cameras, a camera cluster, long FOV, short FOV, wide angle, fisheye, etc.

[0120] System 100 or various components of system 100 may be incorporated into a variety of different platforms. In some embodiments, system 100 may be included in vehicle 200, as shown in FIG. 2A. For example, vehicle 200 may include processing unit 110 and any other components of system 100, as described above with respect to FIG. 1. In some embodiments, vehicle 200 may include only a single image capture device (e.g., a camera), while in other embodiments, such as those discussed in connection with FIGS. 2B-2E, multiple image capture devices may be used. For example, as shown in FIG. 2A, either of image capture devices 122 and 124 of vehicle 200 may be part of an ADAS (advanced driver assistance system) imaging set.

[0121] The image capture device included in vehicle 200 as part of image acquisition unit 120 may be located in any suitable location. In some embodiments, as shown in Figures 2A-2E and 3A-3C, image capture device 122 may be located near the rearview mirror. This location may provide a line of sight similar to that of the driver of vehicle 200 and may assist in determining what the driver can and cannot see. Although image capture device 122 may be located anywhere near the rearview mirror, placing image capture device 122 on the driver's side of the mirror may further assist in capturing an image representative of the driver's field of view and / or line of sight.

[0122] Other locations for the image capture devices of the image acquisition unit 120 may also be used. For example, the image capture device 124 may be located on or in the bumper of the vehicle 200. Such a location may be particularly suitable for an image capture device having a wide field of view. The line of sight of an image capture device located in the bumper may be different from the line of sight of the driver, and thus the bumper image capture device and the driver do not always see the same object. The image capture devices (e.g., image capture devices 122, 124, and 126) may also be located in other locations. For example, the image capture devices may be located on one or both side mirrors of the vehicle 200, on the roof of the vehicle 200, on the hood of the vehicle 200, on the trunk of the vehicle 200, on the sides of the vehicle 200, mounted on, positioned behind, or positioned in front of any window of the vehicle 200, mounted on or near the front and / or rear lights of the vehicle 200, etc.

[0123] In addition to the image capture device, vehicle 200 may include various other components of system 100. For example, processing unit 110 may be integrated into the vehicle's engine control unit (ECU) or may be included in vehicle 200 separate from the ECU. Vehicle 200 may also be equipped with a position sensor 130, such as a GPS receiver, and vehicle 200 may also include a map database 160 and memory units 140 and 150.

[0124] As described above, the wireless transceiver 172 may transmit and / or receive data via one or more networks (e.g., a cellular network, the Internet, etc.). For example, the wireless transceiver 172 may upload data collected by the system 100 to one or more servers and download data from one or more servers. Through the wireless transceiver 172, the system 100 may receive updates to the data stored in the map database 160, the memory 140, and / or the memory 150, for example, periodically or on demand. Similarly, the wireless transceiver 172 may upload any data from the system 100 (e.g., images captured by the image acquisition unit 120, data received by the position sensor 130, other sensors or vehicle control systems, etc.) and / or any data processed by the processing unit 110 to one or more servers.

[0125] System 100 may upload data to a server (e.g., the cloud) based on a privacy level setting. For example, system 100 may implement a privacy level setting that regulates or limits the types of data (including metadata) that may uniquely identify the vehicle and / or the driver / owner of the vehicle that are sent to the server. Such settings may be set by a user via wireless transceiver 172, may be initialized by factory default settings, or may be set by data received by wireless transceiver 172, for example.

[0126] In some embodiments, system 100 may upload data according to a "high" privacy level, under a configuration setting, system 100 may transmit data (e.g., location information associated with a route, captured images, etc.) without any details about a particular vehicle and / or driver / owner. For example, when uploading data according to a "high" privacy level, system 100 may transmit data such as captured images and / or limited location information associated with a route that does not include the vehicle identification number (VIN) or the name of the driver or owner of the vehicle, but instead.

[0127] Other privacy levels are contemplated. For example, system 100 may transmit data to the server according to a "medium" privacy level, which may include additional information not included under a "high" privacy level, such as the make and / or model of the vehicle and / or vehicle type (e.g., passenger car, sport utility vehicle, truck, etc.). In some embodiments, system 100 may upload data according to a "low" privacy level. Under a "low" privacy level setting, system 100 may upload and include sufficient data to uniquely identify a particular vehicle, owner / driver, and / or some or all of the route traveled by the vehicle. Such "low" privacy level data may include, for example, one or more of the VIN, driver / owner name, starting point of the vehicle prior to departure, intended destination of the vehicle, make and / or model of the vehicle, type of vehicle, etc.

[0128] FIG 2A is a side view representation of an exemplary vehicle imaging system according to a disclosed embodiment. FIG 2B is a top view representation of the embodiment shown in FIG 2A. As shown in FIG 2B, the disclosed embodiment may show a vehicle 200 including a system 100 within the body having a first image capture device 122 positioned near a rearview mirror and / or near a driver of the vehicle 200, a second image capture device 124 positioned on or within a bumper area (e.g., one of the bumper areas 210) of the vehicle 200, and a processing unit 110.

[0129] As shown in Figure 2C, both image capture devices 122 and 124 may be positioned near the rearview mirror and / or near the driver of vehicle 200. Additionally, while two image capture devices 122 and 124 are shown in Figures 2B and 2C, it should be understood that other embodiments may include three or more image capture devices. For example, in the embodiment shown in Figures 2D and 2E, a first image capture device 122, a second image capture device 124, and a third image capture device 126 are included in system 100 of vehicle 200.

[0130] 2D, image capture device 122 may be positioned near a rearview mirror and / or near the driver of vehicle 200, and image capture devices 124 and 126 may be positioned on a bumper area (e.g., one of bumper areas 210) of vehicle 200. Also, as shown in FIG. 2E, image capture devices 122, 124 and 126 may be positioned near a rearview mirror and / or near the driver's seat of vehicle 200. The disclosed embodiments are not limited to any particular number and configuration of image capture devices, and image capture devices may be positioned in any suitable location within and / or on vehicle 200.

[0131] It should be understood that the disclosed embodiments are not limited to vehicles and may be applicable in other contexts, and it should also be understood that the disclosed embodiments are not limited to a particular type of vehicle 200 and may be applicable to all types of vehicles, including cars, trucks, trailers, and other types of vehicles.

[0132] The first image capture device 122 may include any suitable type of image capture device. The image capture device 122 may include an optical axis. In one example, the image capture device 122 may include an Aptina M9V024 WVGA sensor with a global shutter. In other embodiments, the image capture device 122 may provide a resolution of 1280×960 pixels and may include a rolling shutter. The image capture device 122 may include various optical elements. In some embodiments, one or more lenses may be included to provide, for example, a desired focal length and field of view of the image capture device. In some embodiments, a 6 mm lens or a 12 mm lens may be associated with the image capture device 122. In some embodiments, the image capture device 122 may be configured to capture an image having a desired field of view (FOV) 202, as shown in FIG. 2D. For example, image capture device 122 may be configured to have a regular FOV, such as in the range of 40 degrees to 56 degrees, including a 46 degree FOV, a 50 degree FOV, a 52 degree FOV, or degrees greater than 52 degrees. Alternatively, image capture device 122 may be configured to have a narrow FOV, such as in the range of 23 degrees to 40 degrees, such as a 28 degree FOV or a 36 degree FOV. In addition, image capture device 122 may be configured to have a wide FOV, such as in the range of 100 degrees to 180 degrees. In some embodiments, image capture device 122 may include a wide angle bumper camera or a bumper camera with up to 180 degrees FOV. In some embodiments, image capture device 122 may be a 7.2 Mpixel image capture device with an aspect ratio of about 2:1 (e.g., H×V=3800×1900 pixels) with a horizontal FOV of about 100 degrees. Such an image capture device may be used in place of a three-dimensional image capture device configuration. Due to large lens distortion, the vertical FOV of such an image capture device can be much less than 50 degrees in implementations in which the image capture device uses a radially symmetric lens, for example, such lenses are not radially symmetric, thereby allowing a vertical FOV greater than 50 degrees with a horizontal FOV of 100 degrees.

[0133] The first image capture device 122 may acquire a plurality of first images of a scene associated with the vehicle 200. The plurality of first images may each be acquired as a series of image scan lines, which may be captured using a rolling shutter. Each scan line may include a plurality of pixels.

[0134] The first image capture device 122 may have a scan rate associated with the acquisition of each of the first series of image scan lines. The scan rate may refer to the rate at which the image sensor is able to acquire image data associated with each pixel included in a particular scan line.

[0135] Image capture devices 122, 124, and 126 may include any suitable type and number of image sensors, including, for example, CCD sensors or CMOS sensors. In one embodiment, a CMOS image sensor may be utilized with a rolling shutter whereby each pixel in a row is read one at a time, and the scanning of the rows proceeds row by row until the entire image frame is captured. In some embodiments, the rows may be captured sequentially from top to bottom for the frame.

[0136] In some embodiments, one or more of the image capture devices disclosed herein (e.g., image capture devices 122, 124, and 126) may constitute a high-resolution imager and may have a resolution of greater than 5 Mpixels, greater than 7 Mpixels, greater than 10 Mpixels, or more.

[0137] The use of a rolling shutter can result in pixels in different rows being exposed and captured at different times, which can cause skew and other image artifacts in the captured image frame. On the other hand, if the image capture device 122 is configured to operate with a global or synchronous shutter, all pixels can be exposed for the same amount of time during a common exposure period. As a result, image data in a frame collected from a system utilizing a global shutter represents a snapshot of the entire FOV (such as FOV 202) at a particular time. Conversely, when applying a rolling shutter, each row in a frame is exposed and data is captured at a different time. Thus, moving objects can appear distorted with an image capture device having a rolling shutter. This phenomenon is described in more detail below.

[0138] The second image capture device 124 and the third image capture device 126 may be any type of image capture device. Like the first image capture device 122, each of the image capture devices 124 and 126 may include an optical axis. In one embodiment, each of the image capture devices 124 and 126 may include an Aptina M9V024 WVGA sensor with a global shutter. Alternatively, each of the image capture devices 124 and 126 may include a rolling shutter. Like the image capture device 122, the image capture devices 124 and 126 may be configured to include various lenses and optical elements. In some embodiments, the lenses associated with the image capture devices 124 and 126 may provide the same FOV (such as FOV 202) as the FOV associated with the image capture device 122, or a narrower FOV (such as FOVs 204 and 206). For example, image capture devices 124 and 126 may have a FOV of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less than 20 degrees.

[0139] Image capture devices 124 and 126 may acquire multiple second and third images of a scene associated with vehicle 200. Each of the multiple second and third images may be acquired as a second and third series of image scan lines, which may be captured using a rolling shutter. Each scan line or row may have a number of pixels. Image capture devices 124 and 126 may have second and third scan rates associated with acquiring each image scan line included in the second and third series.

[0140] Each image capture device 122, 124, and 126 may be positioned in any suitable location and in any suitable orientation relative to vehicle 200. The relative positions of image capture devices 122, 124, and 126 may be selected to assist in fusing together information obtained from the image capture devices. For example, in some embodiments, the FOV associated with image capture device 124 (FOV 204) may partially or completely overlap with the FOV associated with image capture device 122 (e.g., FOV 202) and the FOV associated with image capture device 126 (e.g., FOV 206).

[0141] Image capture devices 122, 124, and 126 may be positioned on vehicle 200 at any suitable relative height. In one example, there may be a height difference between image capture devices 122, 124, and 126, which may provide sufficient parallax information to enable stereo analysis. For example, as shown in FIG. 2A, the two image capture devices 122 and 124 are at different heights. There may also be a lateral displacement difference between image capture devices 122, 124, and 126, which may provide additional parallax information for stereo analysis by processing unit 110, for example. The lateral displacement difference may be expressed as d x In some embodiments, a forward or rearward displacement (e.g., range displacement) may exist between image capture devices 122, 124, 126. For example, image capture device 122 may be positioned 0.5 to 2 meters or more behind image capture device 124 and / or image capture device 126. With this type of displacement, one of the image capture devices may be able to cover a potential blind spot of the other image capture device.

[0142] Image capture device 122 may have any suitable resolution capability (e.g., the number of pixels associated with the image sensor), and the resolution of the image sensor associated with image capture device 122 may be higher, lower, or the same as the resolution of the image sensors associated with image capture devices 124 and 126. In some embodiments, the image sensors associated with image capture device 122 and / or image capture devices 124 and 126 may have a resolution of 640×480, 1024×768, 1280×960, or any other suitable resolution.

[0143] The frame rate (e.g., the rate at which the image capture device acquires a set of pixel data for one image frame before moving on to capture pixel data associated with the next image frame) may be controllable. The frame rate associated with image capture device 122 may be higher, lower, or the same as the frame rate associated with image capture devices 124 and 126. The frame rates associated with image capture devices 122, 124, and 126 may depend on a variety of factors that may affect the timing of the frame rate. For example, one or more of image capture devices 122, 124, and 126 may include a selectable pixel delay period imposed before or after acquisition of image data associated with one or more pixels of an image sensor in image capture devices 122, 124, and / or 126. In general, image data corresponding to each pixel may be acquired according to the device's clock rate (e.g., one pixel per clock cycle). Additionally, in embodiments including a rolling shutter, one or more of image capture devices 122, 124, and 126 may include a selectable horizontal blanking period that is imposed before or after acquisition of image data associated with a row of pixels of an image sensor in image capture devices 122, 124, and / or 126. Additionally, one or more of image capture devices 122, 124, and / or 126 may include a selectable vertical blanking period that is imposed before or after acquisition of image data associated with an image frame of image capture devices 122, 124, and 126.

[0144] These timing controls may enable the frame rates associated with image capture devices 122, 124, and 126 to be synchronized even if the line scan rate of each image capture device is different. Additionally, as discussed in more detail below, among other factors (e.g., image sensor resolution, maximum line scan rate, etc.), these selectable timing controls may enable the synchronization of image capture from areas where the FOV of image capture device 122 overlaps with one or more of the FOVs of image capture devices 124 and 126, even if the field of view of image capture device 122 differs from the FOV of image capture devices 124 and 126.

[0145] Frame rate timing for image capture devices 122, 124, and 126 may depend on the resolution of the associated image sensors. For example, if one device includes an image sensor with a resolution of 640×480 and the other device includes an image sensor with a resolution of 1280×960, assuming similar line scan rates for both devices, capturing a frame of image data from the sensor with the higher resolution will require a longer period of time.

[0146] Another factor that may affect the timing of image data acquisition at image capture devices 122, 124, and 126 is the maximum line scan rate. For example, acquisition of a row of image data from an image sensor included in image capture devices 122, 124, and 126 requires some minimum amount of time. Assuming that no pixel delay period is added, this minimum amount of time to acquire a row of image data will be related to the maximum line scan rate of a particular device. Devices that offer higher maximum line scan rates have the potential to provide higher frame rates than devices with lower maximum line scan rates. In some embodiments, one or both of image capture devices 124 and 126 may have a maximum line scan rate that is higher than the maximum line scan rate associated with image capture device 122. In some embodiments, the maximum line scan rate of image capture devices 124 and / or 126 may be 1.25 times, 1.5 times, 1.75 times, or 2 times or more than the maximum line scan rate of image capture device 122.

[0147] In another embodiment, image capture devices 122, 124, and 126 may have the same maximum line scan rate, but image capture device 122 may operate at a scan rate that is equal to or less than its maximum scan rate. The system may be configured such that one or both of image capture devices 124 and 126 operate at a line scan rate that is equal to the line scan rate of image capture device 122. In other examples, the system may be configured such that the line scan rate of image capture devices 124 and / or 126 may be 1.25 times, 1.5 times, 1.75 times, or 2 times or more than the line scan rate of image capture device 122.

[0148] In some embodiments, image capture devices 122, 124, and 126 may be asymmetric. That is, the image capture devices may include cameras with different fields of view (FOV) and focal lengths. The fields of view of image capture devices 122, 124, and 126 may include, for example, any desired area relative to the environment of vehicle 200. In some embodiments, one or more of image capture devices 122, 124, and 126 may be configured to acquire image data from an environment in front of vehicle 200, an environment behind vehicle 200, an environment on either side of vehicle 200, or a combination thereof.

[0149] Further, the focal length associated with each image capture device 122, 124 and / or 126 may be selectable (e.g., by inclusion of an appropriate lens, etc.) such that each device captures images of objects at a desired distance range from vehicle 200. For example, in some embodiments, image capture devices 122, 124 and 126 may capture images of close-by objects within a few meters of the vehicle. Image capture devices 122, 124, 126 may also be configured to capture images of objects at greater distances from the vehicle (e.g., 25 m, 50 m, 100 m, 150 m or more). Further, the focal lengths of image capture devices 122, 124, and 126 may be selected such that one image capture device (e.g., image capture device 122) can capture images of objects relatively close to the vehicle (e.g., within 10 m or within 20 m), while the other image capture devices (e.g., image capture devices 124 and 126) can capture images of objects farther away from vehicle 200 (e.g., greater than 20 m, greater than 50 m, greater than 100 m, greater than 150 m, etc.).

[0150] According to some embodiments, the FOV of one or more of image capture devices 122, 124, and 126 may have a wide angle. For example, it may be advantageous to have an FOV of 140 degrees, particularly for image capture devices 122, 124, and 126 that may be used to capture images of areas near vehicle 200. For example, image capture device 122 may be used to capture images of areas to the right or left of vehicle 200, and in such embodiments, it may be desirable for image capture device 122 to have a wide FOV (e.g., at least 140 degrees).

[0151] The field of view associated with each of the image capture devices 122, 124, and 126 may depend on the respective focal length. For example, as the focal length increases, the corresponding field of view decreases.

[0152] Image capture devices 122, 124, and 126 may be configured to have any suitable field of view. In one particular example, image capture device 122 may have a horizontal FOV of 46 degrees, image capture device 124 may have a horizontal FOV of 23 degrees, and image capture device 126 may have a horizontal FOV between 23 and 46 degrees. In another example, image capture device 122 may have a horizontal FOV of 52 degrees, image capture device 124 may have a horizontal FOV of 26 degrees, and image capture device 126 may have a horizontal FOV between 26 and 52 degrees. In some embodiments, the ratio between the FOV of image capture device 122 and the FOV of image capture device 124 and / or image capture device 126 may vary from 1.5 to 2.0. In other embodiments, the ratio may vary from 1.25 to 2.25.

[0153] System 100 may be configured such that the field of view of image capture device 122 at least partially or completely overlaps the field of view of image capture device 124 and / or image capture device 126. In some embodiments, system 100 may be configured such that the fields of view of image capture devices 124 and 126, for example, fall within the field of view of image capture device 122 (e.g., are smaller than the field of view of image capture device 122) and share a common center with the field of view of image capture device 122. In other embodiments, image capture devices 122, 124, and 126 may capture adjacent FOVs or may have partially overlapping FOVs. In some embodiments, the fields of view of image capture devices 122, 124, and 126 may be aligned such that the center of image capture device 124 and / or 126, which has a narrower FOV, may be located in the bottom half of the field of view of device 122, which has a wider FOV.

[0154] 2F is a diagrammatic representation of an exemplary vehicle control system according to disclosed embodiments. As shown in FIG. 2F, vehicle 200 may include a throttle system 220, a brake system 230, and a steering system 240. System 100 may provide inputs (e.g., control signals) to one or more of throttle system 220, brake system 230, and steering system 240 via one or more data links (e.g., any wired and / or wireless link or link that transmits data). For example, based on analysis of images acquired by image capture devices 122, 124, and / or 126, system 100 may provide control signals to one or more of throttle system 220, brake system 230, and steering system 240 to navigate vehicle 200 (e.g., by causing it to accelerate, turn, lane shift, etc.). Additionally, system 100 may receive inputs indicative of operating conditions of vehicle 200 (e.g., speed, whether vehicle 200 is braking and / or turning, etc.) from one or more of throttle system 220, braking system 230, and steering system 24. Further details are provided below in conjunction with Figures 4-7.

[0155] As shown in FIG. 3A , the vehicle 200 may also include a user interface 170 for interacting with a driver or passenger of the vehicle 200. For example, the user interface 170 in a vehicle application may include a touch screen 320, knobs 330, buttons 340, and a microphone 350. The driver or passenger of the vehicle 200 may also interact with the system 100 using a steering wheel (e.g., located on or near the steering column of the vehicle 200, including, for example, a turn signal handle), buttons (e.g., located on the steering wheel of the vehicle 200), and the like. In some embodiments, the microphone 350 may be positioned adjacent to the rearview mirror 310. Similarly, in some embodiments, the image capture device 122 may be located near the rearview mirror 310. In some embodiments, the user interface 170 may also include one or more speakers 360 (e.g., speakers of a vehicle audio system). For example, the system 100 may provide various notifications (e.g., alerts) via the speaker 360.

[0156] 3B-3D are diagrams of an exemplary camera mount 370 configured to be positioned behind a rearview mirror (e.g., rearview mirror 310) and opposite a vehicle windshield, according to disclosed embodiments. As shown in FIG. 3B, camera mount 370 may include image capture devices 122, 124, and 126. Image capture devices 124 and 126 may be positioned behind glare shield 380, which may directly contact the windshield and may include a film and / or composition of anti-reflective materials. For example, glare shield 380 may be positioned to be aligned opposite a windshield having a matching slope. In some embodiments, each of image capture devices 122, 124, and 126 may be positioned behind glare shield 380, for example, as shown in FIG. 3D. The disclosed embodiments are not limited to any particular configuration of image capture devices 122, 124, and 126, camera mount 370, and glare shield 380. FIG. 3C is a view of the camera mount 370 shown in FIG. 3B from the front.

[0157] As will be appreciated by those skilled in the art having the benefit of this disclosure, many variations and / or modifications may be made to the above disclosed embodiments. For example, not all components are essential to the operation of system 100. Furthermore, any component may be located in any suitable portion of system 100, and the components may be rearranged in various configurations while still providing the functionality of the disclosed embodiments. Thus, the configurations discussed above are examples, and regardless of the configuration described above, system 100 may provide a wide range of functionality for analyzing the surroundings of vehicle 200 and navigating vehicle 200 in response to the analysis.

[0158] As discussed in more detail below, in accordance with various disclosed embodiments, system 100 may provide various features related to autonomous driving and / or driver assistance technology. For example, system 100 may analyze image data, location data (e.g., GPS location information), map data, speed data, and / or data from sensors included in vehicle 200. System 100 may collect data for analysis, for example, from image capture unit 120, location sensor 130, and other sensors. Additionally, system 100 may analyze the collected data to identify whether vehicle 200 should take a particular action and then automatically take the determined action without human intervention. For example, when vehicle 200 navigates without human participation, system 100 may automatically control braking, acceleration, and / or steering of vehicle 200 (e.g., by sending control signals to one or more of throttle system 220, braking system 230, and steering system 240). Additionally, system 100 may analyze the collected data and issue warnings and / or alerts to vehicle occupants based on the analysis of the collected data. Further details regarding various embodiments provided by system 100 are provided below.

[0159] Forward-looking multi-imaging system

[0160] As discussed above, system 100 may provide driver assistance features using a multi-camera system. The multi-camera system may use one or more cameras facing toward the front of the vehicle. In other embodiments, the multi-camera system may include one or more cameras facing toward the side of the vehicle or toward the rear of the vehicle. In one embodiment, for example, system 100 may use a two-camera imaging system, where a first camera and a second camera (e.g., image capture devices 122 and 124) may be positioned at the front and / or side of the vehicle (e.g., vehicle 200). Other camera configurations are consistent with the disclosed embodiments, and the configurations disclosed herein are examples. For example, system 100 may include any number of camera configurations (e.g., one, two, three, four, five, six, seven, eight, etc.). Additionally, system 100 may include a "cluster" of cameras. For example, a cluster of cameras (including any suitable number of cameras, e.g., 1, 4, 8, etc.) can face forward relative to the vehicle, or can face in any other direction (e.g., rearward, sideways, diagonally, etc.). Thus, system 100 may include multiple clusters of cameras, with each cluster oriented in a particular direction to capture images from a particular region of the vehicle's environment.

[0161] The first camera may have a field of view that is larger, smaller, or partially overlapping that of the second camera. Additionally, the first camera may be connected to a first image processor to perform monocular image analysis of the image provided by the first camera, and the second camera may be connected to a second image processor to perform monocular image analysis of the image provided by the second camera. The outputs (e.g., processed information) of the first and second image processors may be combined. In some embodiments, the second image processor may receive images from both the first and second cameras to perform stereo analysis. In another embodiment, the system 100 may use a three-camera imaging system, where each camera has a different field of view. Thus, such a system may make decisions based on information derived from objects at various distances both in front of and to the sides of the vehicle. References to monocular image analysis may refer to cases where image analysis is performed based on images captured from a single viewpoint (e.g., a single camera). Stereo image analysis may refer to cases where image analysis is performed based on two or more images captured with one or more image capture parameters changed. For example, captured images suitable for performing stereoscopic image analysis may include images captured from two or more different positions, images captured from different fields of view, images captured using different focal lengths, images captured with parallax information, etc.

[0162] For example, in one embodiment, system 100 may implement a three-camera configuration using image capture devices 122-126. In such a configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees or other value selected from the range of about 20-45 degrees, etc.), image capture device 124 may provide a wide field of view (e.g., 150 degrees or other value selected from the range of about 100-180 degrees), and image capture device 126 may provide a medium field of view (e.g., 46 degrees or other value selected from the range of about 35-60 degrees). In some embodiments, image capture device 126 may operate as a main or primary camera. Image capture devices 122-126 may be positioned substantially side-by-side (e.g., 6 cm apart) behind rearview mirror 310. Additionally, in some embodiments, one or more of image capture devices 122-126 may be mounted behind glare shield 380 that is flush with the windshield of vehicle 200, as discussed above. Such a shield may operate to minimize the effect of any reflections from the interior of the vehicle on the image capture devices 122-126.

[0163] In another embodiment, as discussed above in connection with Figures 3B and 3C, the wide field of view camera (e.g., image capture device 124 in the example above) may be mounted lower than the narrow main field of view camera (e.g., image capture devices 122 and 126 in the example above). This configuration may provide a free line of sight from the wide field of view camera. To reduce reflections, the camera may be mounted near the windshield of vehicle 200 and may include a polarizer to attenuate reflected light.

[0164] A three-camera system may provide certain performance characteristics. For example, some embodiments may include the ability to verify the detection of an object by one camera based on the detection results from another camera. In the three-camera configuration discussed above, processing unit 110 may include, for example, three processing devices (e.g., three EyeQ series processor chips as discussed above), each processing device dedicated to processing images captured by one or more of image capture devices 122-126.

[0165] In a three camera system, the first processing device may receive images from both the main camera and the narrow FOV camera and may perform vision processing of the narrow FOV camera to detect, for example, other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the first processing device may calculate pixel discrepancies between the images from the main camera and the narrow camera and create a 3D reconstruction of the environment of the vehicle 200. The first processing device may then combine the 3D reconstruction with 3D map data or 3D information calculated based on information from another camera.

[0166] The second processing device may receive images from the main camera and perform vision processing to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Furthermore, the second processing device may calculate the camera displacement, calculate pixel discrepancies between successive images based on the displacement, and create a 3D reconstruction (e.g., structure-from-motion) of the scene. The second processing device may send the structure-from-motion based on the 3D reconstruction to the first processing device and combine the structure-from-motion with the stereoscopic 3D image.

[0167] The third processing device may receive images from the wide FOV camera and process the images to detect vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. The third processing device may further execute additional processing instructions to analyze the images and identify moving objects in the images, such as vehicles changing lanes, pedestrians, etc.

[0168] In some embodiments, having image-based information streams captured and processed independently may provide an opportunity to provide redundancy in the system, such as by using a first image capture device and images processed from that device to verify and / or supplement information obtained by capturing and processing image information from at least a second image capture device.

[0169] In some embodiments, system 100 may use two image capture devices (e.g., image capture devices 122 and 124) in providing navigation assistance to vehicle 200, and a third image capture device (e.g., image capture device 126) may be used to provide redundancy and verify analysis of data received from the other two image capture devices. For example, in such a configuration, image capture devices 122 and 124 may provide images for stereo analysis by system 100 for navigating vehicle 200, while image capture device 126 may provide images for monocular analysis by system 100 to provide redundancy and verification of information derived based on images captured from image capture devices 123 and / or image capture device 124. That is, image capture device 126 (and corresponding processing device) may be viewed as providing a redundant subsystem that provides a check on the analysis derived from image capture devices 122 and 124 (e.g., to provide an automatic emergency braking (AEB) system). Additionally, in some embodiments, redundancy and validation of received data can be supplemented based on information received from one or more sensors (e.g., radar, lidar, acoustic sensors, information received from one or more transceivers outside the vehicle, etc.).

[0170] Those skilled in the art will recognize that the above camera configurations, camera placements, camera numbers, camera locations, etc. are merely exemplary. These components, etc. described for an overall system, may be assembled and used in a variety of different configurations without departing from the scope of the disclosed embodiments. Further details regarding the use of multi-camera systems to provide driver assistance and / or autonomous vehicle functions follow below.

[0171] 4 is an example functional block diagram of memory 140 and / or 150 that may be stored / programmed with instructions to perform one or more operations in accordance with the disclosed embodiments. Although reference is made below to memory 140, those skilled in the art will recognize that instructions may be stored in memory 140 and / or 150.

[0172] As shown in FIG. 4, memory 140 may store monocular image analysis module 402, stereo image analysis module 404, velocity and acceleration module 406, and navigation response module 408. The disclosed embodiments are not limited to any particular configuration of memory 140. Furthermore, application processor 180 and / or image processor 190 may execute instructions stored in any of modules 402-408 included in memory 140. Those skilled in the art will appreciate that references to processing unit 110 in the following discussion may refer to application processor 180 and image processor 190 individually or collectively. Thus, any steps of the following processes may be performed by one or more processing devices.

[0173] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis of a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensory information (e.g., information from radar) to perform the monocular image analysis. As described below in connection with FIGS. 5A-5D, the monocular image analysis module 402 may include instructions to detect a set of features in the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazards, and any other features associated with the vehicle's environment. Based on the analysis, the system 100 may cause one or more navigational responses in the vehicle 200 (e.g., via the processing unit 110), such as turns, lane shifts, and acceleration changes, as discussed below in connection with the navigation response module 408.

[0174] In one embodiment, the monocular image analysis module 402 may store instructions (e.g., computer vision software) that, when executed by the processing unit 110, perform monocular image analysis of a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensory information (e.g., information from radar, lidar, etc.) to perform the monocular image analysis. As described below in connection with FIGS. 5A-5D, the monocular image analysis module 402 may include instructions for detecting a set of features in the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazards, and any other features associated with the vehicle's environment. Based on the analysis, the system 100 may cause one or more navigational responses in the vehicle 200, such as a turn, lane shift, acceleration change, etc., as discussed below in connection with determining a navigational response (e.g., by the processing unit 110).

[0175] In one embodiment, stereo image analysis module 404 may store instructions (e.g., computer vision software) that, when executed by processing unit 110, perform stereo image analysis of a first and second set of images acquired by a combination of image capture devices selected from image capture devices 122, 124, and 126. In some embodiments, processing unit 110 may combine information from the first and second sets of images with additional sensory information (e.g., information from radar) to perform stereo image analysis. For example, stereo image analysis module 404 may include instructions to perform stereo image analysis based on a first set of images acquired by image capture device 124 and a second set of images acquired by image capture device 126. As described below in connection with FIG. 6, stereo image analysis module 404 may include instructions to detect a set of features in the first and second sets of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and hazards. Based on the analysis, processing unit 110 may cause one or more navigational responses in vehicle 200, such as turns, lane shifts, and acceleration changes, as described below in connection with navigation response module 408. Additionally, in some embodiments, stereo image analysis module 404 may implement techniques related to trained systems (such as neural networks or deep neural networks) or untrained systems.

[0176] In one embodiment, the speed and acceleration module 406 may store software configured to analyze data received from one or more computational and electromechanical devices in the vehicle 200 configured to alter the speed and / or acceleration of the vehicle 200. For example, the processing unit 110 may execute instructions associated with the speed and acceleration module 406 to calculate a target speed of the vehicle 200 based on data derived from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include, for example, a target position, speed and / or acceleration, the position and / or speed of the vehicle 200 relative to nearby vehicles, pedestrians or road objects, and the position information of the vehicle 200 relative to lane markings of the road, etc. Additionally, the processing unit 110 may calculate a target speed of the vehicle 200 based on sensory input (e.g., information from a radar) and input from other systems of the vehicle 200, such as the throttle system 220, the braking system 230 and / or the steering system 240 of the vehicle 200. Based on the calculated target speed, the processing unit 110 may send electronic signals to the throttle system 220, the braking system 230 and / or the steering system 240 of the vehicle 200 to trigger a change in speed and / or acceleration, for example by physically reducing the brakes or accelerator of the vehicle 200.

[0177] In one embodiment, the navigation response module 408 may store software executable by the processing unit 110 to determine a desired navigation response based on data derived from execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include position and speed information associated with nearby vehicles, pedestrians, and road objects, target position information for the vehicle 200, and the like. Additionally, in some embodiments, the navigation response may be based (partially or fully) on map data, a predetermined position of the vehicle 200, and / or a relative speed or relative acceleration between the vehicle 200 and one or more objects detected from execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. The navigation response module 408 may also determine a desired navigation response based on sensory inputs (e.g., information from a radar) and inputs from other systems of the vehicle 200, such as the throttle system 220, the braking system 230, and the steering system 240 of the vehicle 200. Based on the desired navigational response, processing unit 110 may trigger the desired navigational response by sending electronic signals to throttle system 220, braking system 230, and steering system 240 of vehicle 200 to, for example, turn the steering wheel of vehicle 200 to achieve a predetermined angle of rotation. In some embodiments, processing unit 110 may use the output of navigational response module 408 (e.g., the desired navigational response) as an input to the execution of velocity and acceleration module 406 to calculate a speed change of vehicle 200.

[0178] Additionally, any of the modules disclosed herein (e.g., modules 402, 404, and 406) may implement techniques relating to trained systems (such as neural networks or deep neural networks) or untrained systems.

[0179] FIG. 5A is a flow chart illustrating an example process 500A for generating one or more navigational responses based on monocular image analysis, according to disclosed embodiments. In step 510, processing unit 110 may receive a plurality of images via data interface 128 between processing unit 110 and image acquisition unit 120. For example, a camera (such as image capture device 122 having field of view 202) included in image acquisition unit 120 may capture a plurality of images of an area in front of vehicle 200 (or, for example, to the side or rear of the vehicle) and transmit them to processing unit 110 via a data connection (e.g., digital, wired, USB, wireless, Bluetooth, etc.). Processing unit 110 may execute monocular image analysis module 402 to analyze the plurality of images in step 520, as described in further detail below in connection with FIGS. 5B-5D. By performing the analysis, processing unit 110 may detect a set of features in the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, and traffic lights.

[0180] Processing unit 110 may also execute monocular image analysis module 402 in step 520 to detect various road hazards, such as truck tire parts, fallen road signs, loose cargo, and small animals. Road hazards may vary in structure, shape, size, and color, making such hazards more difficult to detect. In some embodiments, processing unit 110 may execute monocular image analysis module 402 to perform multi-frame analysis on multiple images to detect road hazards. For example, processing unit 110 may estimate camera movement between consecutive image frames and calculate pixel discrepancies between frames to build a 3D map of the road. Processing unit 110 may then use the 3D map to detect the road surface and hazards present on the road surface.

[0181] In step 530, processing unit 110 may execute navigation response module 408 to cause vehicle 200 to make one or more navigation responses based on the analysis performed in step 520 and the techniques described above in connection with FIG. 4. The navigation responses may include, for example, a turn, a lane shift, and an acceleration change. In some embodiments, processing unit 110 may use data derived from execution of speed and acceleration module 406 to cause one or more navigation responses. Furthermore, multiple navigation responses may be performed simultaneously, sequentially, or any combination thereof. For example, processing unit 110 may cause vehicle 200 to cross a lane and then, for example, accelerate, for example, by sequentially sending control signals to steering system 240 and throttle system 220 of vehicle 200. Alternatively, processing unit 110 may cause vehicle 200 to brake and simultaneously shift lanes, for example, by simultaneously sending control signals to braking system 230 and steering system 240 of vehicle 200.

[0182] FIG. 5B is a flow chart illustrating an example process 500B for detecting one or more vehicles and / or pedestrians in a set of images according to the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to perform process 500B. In step 540, processing unit 110 may identify a set of candidate objects representing vehicles and / or pedestrians that may be present. For example, processing unit 110 may scan one or more images, compare the images to one or more predefined patterns, and identify locations within each image that may contain a target object (e.g., a vehicle, a pedestrian, or parts thereof). The predefined patterns may be specified to achieve a low rate of "false hits" and a low rate of "misses." For example, processing unit 110 may use a low similarity threshold to the predefined patterns to identify candidate objects as possible vehicles or pedestrians. By doing so, processing unit 110 may be able to reduce the probability of missing (e.g., not identifying) a candidate object representing a vehicle or pedestrian.

[0183] At step 542, processing unit 110 may filter the set of candidate objects to eliminate certain candidates (e.g., irrelevant or less relevant objects) based on classification criteria. Such criteria may be derived from various characteristics associated with object types stored in a database (e.g., a database stored in memory 140). The characteristics may include the object's shape, size, texture, and position (e.g., relative to vehicle 200), etc. Thus, processing unit 110 may use one or more sets of criteria to reject false candidates from the set of candidate objects.

[0184] At step 544, processing unit 110 may analyze multiple image frames to identify whether objects in the set of candidate images represent vehicles and / or pedestrians. For example, processing unit 110 may track detected candidate objects across successive frames and accumulate per-frame data associated with the detected objects (e.g., size, position relative to vehicle 200, etc.). Additionally, processing unit 110 may estimate parameters of the detected objects and compare the per-frame position data of the objects to predicted positions.

[0185] In step 546, processing unit 110 may construct a set of measurements of the detected objects. Such measurements may include, for example, position, velocity, and acceleration values ​​(relative to vehicle 200) associated with the detected objects. In some embodiments, processing unit 110 may construct the measurements based on estimation techniques using a series of time-based observations, such as a Kalman filter or linear quadratic estimation (LQE), and / or modeling data available for different object types (e.g., cars, trucks, pedestrians, bicycles, road signs, etc.). A Kalman filter may be based on a measurement of the scale of the object, where the scale measurement is proportional to the time to impact (e.g., the amount of time it takes vehicle 200 to reach the object). Thus, by performing steps 540-546, processing unit 110 may identify vehicles and pedestrians appearing in the set of captured images and derive information (e.g., position, velocity, size) associated with the vehicles and pedestrians. Based on the identification and derived information, processing unit 110 may cause one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.

[0186] In step 548, processing unit 110 may perform optical flow analysis of one or more images to reduce the probability of detecting "false hits" and missing candidate objects representing vehicles or pedestrians. Optical flow analysis may refer to, for example, analyzing movement patterns, separate from road surface movement, for vehicle 200 in one or more images associated with other vehicles and pedestrians. Processing unit 110 may calculate candidate object movement by observing different positions of the object across multiple image frames captured at different times. Processing unit 110 may use the position and time values ​​as inputs to a mathematical model to calculate candidate object movement. Thus, optical flow analysis may provide another way to detect vehicles and pedestrians in the vicinity of vehicle 200. Processing unit 110 may perform optical flow analysis in combination with steps 540-546 to provide redundancy for detecting vehicles and pedestrians and increase the reliability of system 100.

[0187] FIG. 5C is a flow chart illustrating an example process 500C for detecting road marks and / or lane geometry information in a set of images according to the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to perform process 500C. In step 550, processing unit 110 may detect a set of objects by scanning one or more images. To detect segments of lane marks, lane geometry information, and other related road marks, processing unit 110 may filter the set of objects to exclude those determined to be irrelevant (e.g., small holes, small rocks, etc.). In step 552, processing unit 110 may group together segments detected in step 550 that belong to the same road mark or lane mark. Based on the grouping, processing unit 110 may develop a model, such as a mathematical model, to represent the detected segments.

[0188] In step 554, processing unit 110 may construct a set of measurements associated with the detected segments. In some embodiments, processing unit 110 may create a projection of the detected segments from the image plane to the real-world plane. The projection may be characterized using a third-order polynomial with coefficients corresponding to physical properties such as position, slope, curvature, and curvature derivatives of the detected road. In generating the projection, processing unit 110 may take into account road surface changes and pitch and roll rates associated with vehicle 200. In addition, processing unit 110 may model road height by analyzing the position and motion cues present on the road surface. Furthermore, processing unit 110 may estimate pitch and roll rates associated with vehicle 200 by tracking a set of feature points in one or more images.

[0189] In step 556, processing unit 110 may perform a multi-frame analysis, for example, by tracking the detection segment across successive image frames and accumulating frame-by-frame data associated with the detection segment. When processing unit 110 performs a multi-frame analysis, the set of measurements constructed in step 554 may become more reliable and may be associated with an increasingly higher degree of confidence. Thus, by performing steps 550-556, processing unit 110 may identify road marks appearing in the set of captured images and derive lane geometry information. Based on the identification and derived information, processing unit 110 may cause one or more navigational responses in vehicle 200, as described above in connection with FIG. 5A.

[0190] In step 558, processing unit 110 may further develop a safety model of vehicle 200 in the vehicle's surroundings, taking into account additional information sources. Processing unit 110 may use the safety model to define situations in which system 100 may safely execute autonomous control of vehicle 200. To develop the safety model, in some embodiments, processing unit 110 may consider the positions and movements of other vehicles, detected road edges and barriers, and / or general road shape descriptions extracted from map data (such as data from map database 160). By taking into account additional information sources, processing unit 110 may provide redundancy in detecting road marks and lane geometry, increasing the reliability of system 100.

[0191] FIG. 5D is a flow chart illustrating an example process 500D for detecting traffic lights in a set of images according to disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to perform process 500D. In step 560, processing unit 110 may scan the set of images and identify objects that appear at locations in the images that are likely to contain traffic lights. For example, processing unit 110 may filter the identified objects to build a set of candidate objects that excludes objects that are unlikely to correspond to traffic lights. The filtering may be based on various characteristics associated with traffic lights, such as shape, size, texture, and location (e.g., relative to vehicle 200). Such characteristics may be based on many examples of traffic lights and traffic control signals and may be stored in a database. In some embodiments, processing unit 110 may perform a multi-frame analysis on the set of candidate objects reflecting possible traffic lights. For example, processing unit 110 may track the candidate objects across consecutive image frames, estimate the real-world positions of the candidate objects, and filter out objects that are moving (and therefore unlikely to be traffic lights). In some embodiments, processing unit 110 may perform color analysis on the candidate objects to identify the relative location of the detected colors represented within the potential traffic lights.

[0192] In step 562, processing unit 110 may analyze the geometry of the intersection. The analysis may be based on any combination of (i) the number of lanes detected on either side of vehicle 200, (ii) marks (such as arrow marks) detected on the road, and (iii) a description of the intersection extracted from map data (such as data from map database 160). Processing unit 110 may perform the analysis using information derived from execution of monocular analysis module 402. In addition, processing unit 110 may identify a correspondence between traffic lights detected in step 560 and lanes appearing in the vicinity of vehicle 200.

[0193] As vehicle 200 approaches the intersection, in step 564, processing unit 110 may update a confidence associated with the analyzed intersection geometry and the detected traffic lights. For example, the number of traffic lights estimated to appear at the intersection compared to the number that actually appear at the intersection may affect the confidence. Thus, based on the confidence, processing unit 110 may delegate control to the driver of vehicle 200 to improve safety conditions. By performing steps 560-564, processing unit 110 may identify traffic lights appearing in the set of captured images and analyze the intersection geometry information. Based on the identification and analysis, processing unit 110 may cause one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.

[0194] 5E is a flowchart of an example process 500E for generating one or more navigation responses in vehicle 200 based on a vehicle path, according to disclosed embodiments. In step 570, processing unit 110 may construct an initial vehicle path associated with vehicle 200. The vehicle path may be represented using a set of points represented by coordinates (x,y), with distances d between two points in the set of points being expressed as i may be in the range of 1 to 5 meters. In one embodiment, processing unit 110 may construct an initial vehicle path using two polynomials, such as left and right road polynomials. Processing unit 110 may calculate the geometric midpoint between the two polynomials and, if there is a predefined offset (offset 0 may correspond to driving in the center of the lane), offset each point in the resulting vehicle path by the predefined offset (e.g., smart lane offset). The offset may be in a direction perpendicular to the segment between any two points in the vehicle path. In another embodiment, processing unit 110 may use one polynomial and an estimated lane width to offset each point in the vehicle path by half the estimated lane width plus a predefined offset (e.g., smart lane offset).

[0195] In step 572, processing unit 110 may update the vehicle path constructed in step 570. Processing unit 110 may update the vehicle path by calculating the distance d k is the distance d i A higher resolution may be used to reconstruct the vehicle path constructed in 570 such that the distance d k may be in the range of 0.1 to 0.3 meters. Processing unit 110 may reconstruct the vehicle path using a parabolic spline algorithm, which may result in a cumulative distance vector S corresponding to the total length of the vehicle path (i.e., based on the set of points representing the vehicle path).

[0196] In step 574, the processing unit 110 calculates the look-ahead point ((x l ,z l ) in coordinates. The processing unit 110 may extract look-ahead points from the cumulative distance vector S, and the look-ahead points may be associated with a look-ahead distance and a look-ahead time. The look-ahead distance may have a lower bound range of 10-20 meters and may be calculated as the product of the speed of the vehicle 200 and the look-ahead time. For example, as the speed of the vehicle 200 decreases, the look-ahead distance may also decrease (e.g., until a lower bound is reached). The look-ahead time, which may range from 0.5 to 1.5 seconds, may be inversely proportional to the gain of one or more control loops associated with producing a navigation response in the vehicle 200, such as a heading error tracking control loop. For example, the gain of the heading error tracking control loop may depend on the bandwidth of the yaw rate loop, the steering actuator loop, and the vehicle lateral dynamics, etc. Thus, the higher the gain of the heading error tracking control loop, the shorter the look-ahead time.

[0197] In step 576, processing unit 110 may determine a heading error and yaw rate command based on the look-ahead point identified in step 574. Processing unit 110 may calculate the arctangent of the look-ahead point, e.g., arctan(x l / z l) to determine the heading error. Processing unit 110 may determine the yaw rate command as the product of the heading error and a high-level control gain. The high-level control gain may be equal to (2 / look ahead time) if the look ahead distance is not at a lower limit. If the look ahead distance is at a lower limit, the high-level control gain may be equal to (2*velocity of vehicle 200 / look ahead distance).

[0198] 5F is a flow chart illustrating an example process 500F for determining whether a leading vehicle is changing lanes, according to the disclosed embodiments. In step 580, processing unit 110 may determine navigation information associated with a leading vehicle (e.g., a vehicle traveling in front of vehicle 200). For example, processing unit 110 may determine a position, a velocity (e.g., direction and speed), and / or an acceleration of the leading vehicle using the techniques described above in connection with FIGs. 5A and 5B. Processing unit 110 may also determine one or more road polynomials, lookahead points (associated with vehicle 200), and / or a snail trail (e.g., a set of points describing a path taken by a leading vehicle) using the techniques described above in connection with FIGs. 5E.

[0199] In step 582, processing unit 110 may analyze the navigation information identified in step 580. In one embodiment, processing unit 110 may calculate a distance (e.g., along the trail) between the snail trail and the road polynomial. If the difference in this distance along the trail exceeds a predetermined threshold (e.g., 0.1-0.2 meters for straight roads, 0.3-0.4 meters for gently curving roads, and 0.5-0.6 meters for sharply curving roads), processing unit 110 may determine that the leading vehicle is likely changing lanes. If multiple vehicles are detected to be traveling in front of vehicle 200, processing unit 110 may compare the snail trails associated with each vehicle. Based on the comparison, processing unit 110 may determine that a vehicle whose snail trail does not match the snail trails of the other vehicles is likely changing lanes. Processing unit 110 may further compare the curvature of the snail trail (associated with the leading vehicle) to the expected curvature of the road segment along which the leading vehicle is traveling. The expected curvature may be extracted from map data (e.g., data from map database 160), road polynomials, snail trails of other vehicles and prior knowledge about the road, etc. If the difference between the snail trail curvature and the expected curvature of the road segment exceeds a predetermined threshold, processing unit 110 may determine that the leading vehicle is likely changing lanes.

[0200] In another embodiment, processing unit 110 may compare the leading vehicle's instantaneous position to the look-ahead point (associated with vehicle 200) over a particular time period (e.g., 0.5-1.5 seconds). If the difference in distance and the cumulative sum of discrepancies between the leading vehicle's instantaneous position and the look-ahead point during the particular time period exceeds a predetermined threshold (e.g., 0.3-0.4 meters for straight roads, 0.7-0.8 meters for gently curving roads, and 1.3-1.7 meters for sharply curving roads), processing unit 110 may determine that the leading vehicle is likely changing lanes. In another embodiment, processing unit 110 may analyze the geometry of the snail trail by comparing the lateral distance traveled along the trail to the expected curvature of the snail trail. The expected radius of curvature is calculated: (δz 2 +δ x 2 ) / 2 / (δ x ) in which δ x represents the lateral movement distance, and δ z represents the longitudinal travel distance. If the difference between the lateral travel distance and the expected curvature exceeds a predetermined threshold (e.g., 500-700 meters), processing unit 110 may determine that the leading vehicle is likely changing lanes. In another embodiment, processing unit 110 may analyze the position of the leading vehicle. If the position of the leading vehicle obscures the road polynomial (e.g., the leading vehicle overlaps the road polynomial), processing unit 110 may determine that the leading vehicle is likely changing lanes. If the position of the leading vehicle is such that another vehicle is detected ahead of the leading vehicle and the snail trails of the two vehicles are not parallel, processing unit 110 may determine that the (closer) leading vehicle is likely changing lanes.

[0201] In step 584, processing unit 110 may determine whether leading vehicle 200 is changing lanes based on the analysis performed in step 582. For example, processing unit 110 may make that determination based on a weighted average of the individual analyses performed in step 582. Under such a scheme, for example, a determination by processing unit 110 that the leading vehicle is likely changing lanes based on a particular type of analysis may be assigned a value of "1" (with a "0" representing a determination that the leading vehicle is unlikely to be changing lanes). Different weights may be assigned to different analyses performed in step 582, and the disclosed embodiments are not limited to any particular combination of analyses and weights. Furthermore, in some embodiments, the analysis may utilize a trained system (e.g., a machine learning or deep learning system) that may, for example, estimate a future path beyond the vehicle's current location based on images captured at the current location.

[0202] 6 is a flow chart illustrating an example process 600 for generating one or more navigational responses based on stereo image analysis, according to disclosed embodiments. In step 610, processing unit 110 may receive a first and second plurality of images via data interface 128. For example, a camera included in image acquisition unit 120 (such as image capture devices 122 and 124 having fields of view 202 and 204) may capture and transmit a first and second plurality of images of an area in front of vehicle 200 to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first and second plurality of images via two or more data interfaces. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0203] In step 620, processing unit 110 may execute stereo image analysis module 404 to perform stereo image analysis of the first and second plurality of images to create a 3D map of the road in front of the vehicle and detect features in the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and road hazards. The stereo image analysis may be performed similar to the steps described above in connection with FIGS. 5A-5D. For example, processing unit 110 may execute stereo image analysis module 404 to detect candidate objects (e.g., vehicles, pedestrians, road marks, traffic lights, road hazards, etc.) in the first and second plurality of images, filter out a subset of the candidate objects based on various criteria, perform a multi-frame analysis, construct measurements, and identify confidence levels for the remaining candidate objects. In performing the above steps, processing unit 110 may consider information from both the first and second plurality of images, rather than information from only one set of images. For example, processing unit 110 may analyze differences in pixel-level data (or other subsets of data from the two streams of captured images) of a candidate object that appears in both the first and second multiple images. As another example, processing unit 110 may estimate the position and / or velocity (e.g., relative to vehicle 200) of a candidate object by observing that an object appears in one of the multiple images but not in the other, or other differences that may exist for an object that appears in the two image streams. For example, the position, velocity, and / or acceleration relative to vehicle 200 may be determined based on the trajectory, location, movement characteristics, etc. of features associated with the object that appears in one or both of the image streams.

[0204] In step 630, processing unit 110 may execute navigation response module 408 to generate one or more navigational responses in vehicle 200 based on the analysis performed in step 620 and the techniques described above in connection with FIG. 4. The navigational responses may include, for example, turns, lane shifts, acceleration changes, speed changes, braking, etc. In some embodiments, processing unit 110 may generate one or more navigational responses using data derived from execution of speed and acceleration module 406. Furthermore, multiple navigational responses may be performed simultaneously, sequentially, or any combination thereof.

[0205] FIG. 7 is a flow chart illustrating an example process 700 for generating one or more navigation responses based on an analysis of three sets of images, according to disclosed embodiments. In step 710, processing unit 110 may receive the first, second, and third plurality of images via data interface 128. For example, cameras included in image acquisition unit 120 (such as image capture devices 122, 124, and 126 having fields of view 202, 204, and 206) may capture and transmit the first, second, and third plurality of images of an area in front of and / or to the side of vehicle 200 to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first, second, and third plurality of images via three or more data interfaces. For example, each of image capture devices 122, 124, and 126 may have an associated data interface that communicates data to processing unit 110. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0206] In step 720, processing unit 110 may analyze the first, second, and third pluralities of images to detect features in the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and road hazards. The analysis may be performed similar to the steps described above in connection with FIGS. 5A-5D and 6. For example, processing unit 110 may perform monocular image analysis on each of the first, second, and third pluralities of images (e.g., via execution of monocular image analysis module 402 and based on the steps described above in connection with FIGS. 5A-5D). Alternatively, processing unit 110 may perform stereo image analysis on the first and second pluralities of images, the second and third pluralities of images, and / or the first and third pluralities of images (e.g., via execution of stereo image analysis module 404 and based on the steps described above in connection with FIG. 6). Processed information corresponding to the analysis of the first, second, and / or third pluralities of images may be combined. In some embodiments, processing unit 110 may perform a combination of monocular image analysis and stereo image analysis. For example, processing unit 110 may perform monocular image analysis on the first plurality of images (e.g., via execution of monocular image analysis module 402) and perform stereo image analysis on the second and third plurality of images (e.g., via execution of stereo image analysis module 404). The configuration of image capture devices 122, 124, and 126 - including their respective positions and fields of view 202, 204, and 206 - may affect the type of analysis performed on the first, second, and third plurality of images. The disclosed embodiments are not limited to a particular configuration of image capture devices 122, 124, and 126 or the type of analysis performed on the first, second, and third plurality of images.

[0207] In some embodiments, processing unit 110 may perform tests on system 100 based on the images acquired and analyzed in steps 710 and 720. Such tests may provide an indicator of the overall performance of system 100 with a particular configuration of image capture devices 122, 124, and 126. For example, processing unit 110 may identify the rate of "false hits" (e.g., when system 100 incorrectly determines the presence of a vehicle or pedestrian) and "misses."

[0208] At step 730, processing unit 110 may cause one or more navigation responses in vehicle 200 based on information derived from two of the first, second, and third pluralities of images. The selection of two of the first, second, and third pluralities of images may depend on various factors, such as, for example, the number, type, and size of objects detected in each of the plural images. Processing unit 110 may make the selection based on the quality and resolution of the images, the effective field of view reflected in the images, the number of captured frames, and the extent to which one or more objects of interest actually appear in the frames (e.g., the percentage of frames in which the object appears, the proportion in which the object appears in each such frame, etc.), etc.

[0209] In some embodiments, processing unit 110 may select information derived from two of the first, second, and third pluralities of images by identifying the extent to which information derived from one image source is consistent with information derived from the other image source. For example, processing unit 110 may combine processed information derived from each of image capture devices 122, 124, and 126 (whether monocular analysis, stereo analysis, or any combination of the two) to identify visual indicators (e.g., lane markings, detected vehicles and / or their positions and / or paths, detected traffic lights, etc.) that are consistent across images captured from each of image capture devices 122, 124, and 126. Processing unit 110 may also filter out information that is inconsistent across captured images (e.g., vehicles changing lanes, lane models showing vehicles too close to vehicle 200, etc.). Thus, processing unit 110 may select information derived from two of the first, second, and third pluralities of images based on identifying consistent and inconsistent information.

[0210] The navigational responses may include, for example, turns, lane shifts, and acceleration changes. Processing unit 110 may generate one or more navigational responses based on the analysis performed in step 720 and the techniques described above in connection with FIG. 4. Processing unit 110 may also generate one or more navigational responses using data derived from execution of velocity and acceleration module 406. In some embodiments, processing unit 110 may generate one or more navigational responses based on a relative position, relative velocity, and / or relative acceleration between vehicle 200 and an object detected in any of the first, second, and third multiple images. The multiple navigational responses may be performed simultaneously, sequentially, or any combination thereof.

[0211] Reinforcement learning and trained navigation systems

[0212] The following sections discuss autonomous driving, along with systems and methods for achieving autonomous control of a vehicle, whether the autonomous control of the vehicle is fully autonomous (self-driving vehicle) or partially autonomous (e.g., one or more drivers assist the system or function). As shown in FIG. 8, the autonomous driving task can be divided into three main modules, including a sensing module 801, a driving policy module 803, and a control module 805. In some embodiments, the modules 801, 803, and 805 can be stored in the memory unit 140 and / or the memory unit 150 of the system 100, and / or the modules 801, 803, and 805 (or portions thereof) can be stored remotely from the system 100 (e.g., stored in a server accessible to the system 100, for example, via the wireless transceiver 172). Furthermore, any of the modules disclosed herein (e.g., modules 801, 803, and 805) can implement techniques related to trained systems (such as neural networks or deep neural networks) or untrained systems.

[0213] The sensing module 801, which may be implemented using the processing unit 110, may handle various tasks related to sensing navigation conditions in the host vehicle's environment. Such tasks may depend on inputs from various sensors and sensing systems associated with the host vehicle. Those inputs may include images or image streams from one or more on-board cameras, GPS location information, accelerometer output, user feedback, user input to one or more user interface devices, radar, lidar, etc. Sensing, which may include data from cameras and / or any other available sensors along with map information, may be collected, analyzed, and organized into a "sensing state" that describes information extracted from a scene in the host vehicle's environment. Sensing states may include sensing information related to target vehicles, lane markings, pedestrians, traffic lights, road geometry, lane shape, obstacles, distance to other objects / vehicles, relative speed, relative acceleration, among other potential sensing information. Supervised machine learning may be implemented to generate sensing state outputs based on the sensing data provided to the sensing module 801. The output of the sensing module may represent the sensed navigation “state” of the host vehicle, which may be sent to the driving policy module 803.

[0214] Although the sensed states may be developed based on image data received from one or more cameras or image sensors associated with the host vehicle, the sensed states used for navigation may be developed using any suitable sensor or combination of sensors. In some embodiments, the sensed states may be developed without using captured image data. Indeed, any of the navigation principles described herein may be applicable to sensed states developed based on captured image data as well as sensed states developed using other non-image-based sensors. The sensed states may also be determined by sources external to the host vehicle. For example, the sensed states may be developed in whole or in part based on information received from a source remote from the host vehicle (e.g., based on sensor information, processed state information, etc. shared from other vehicles, shared from a central server, or any other source of information related to the host vehicle's navigation state).

[0215] The driving policy module 803, which will be described in more detail below and may be implemented using the processing unit 110, may implement a desired driving policy for determining one or more navigation actions to be taken by the host vehicle in response to sensed navigation conditions. If there are no other agents (e.g., target vehicles or pedestrians) in the host vehicle's environment, the sensed conditions input to the driving policy module 803 may be processed in a relatively straightforward manner. If the sensed conditions require negotiation with one or more other agents, the task becomes more complex. Techniques used to generate the output of the driving policy module 803 may include reinforcement learning (discussed in more detail below). The output of the driving policy module 803 may include at least one navigation action for the host vehicle, and may include a desired acceleration (which may lead to an updated speed of the host vehicle), a desired yaw rate of the host vehicle, a desired trajectory, among other potential desired navigation actions.

[0216] Based on the output from the driving policy module 803, the control module 805, which may also be implemented using the processing unit 110, may develop control instructions for one or more actuators or controlled devices associated with the host vehicle. Such actuators and devices may include an accelerator, one or more steering controls, brakes, signal transmitters, displays, or any other actuators or devices that may be controlled as part of a navigation operation associated with the host vehicle. Aspects of control theory may be used to generate the output of the control module 805. To implement the desired navigation goals or requirements of the driving policy module 803, the control module 805 may be responsible for developing and outputting instructions to the controllable components of the host vehicle.

[0217] Returning to the driving policy module 803, in some embodiments, the driving policy module 803 can be implemented using a trained system trained by reinforcement learning. In other embodiments, the driving policy module 803 can be implemented without machine learning methods by "manually" dealing with various scenarios that may arise during autonomous navigation using specified algorithms. However, such an approach, while feasible, may result in a driving policy that is too simple and may lack the flexibility of a trained system based on machine learning. A trained system may be better equipped to handle complex navigation situations, be better able to determine whether a taxi is parked or stopped to pick up or drop off passengers, be better able to determine whether a pedestrian is about to cross the road in front of the host vehicle, be better able to balance the unexpected behavior of other drivers against self-protection, be better able to navigate busy roads containing target vehicles and / or pedestrians, be better able to determine when to suspend certain navigation rules or augment other rules, be better able to anticipate undetected but expected conditions (e.g., whether a pedestrian appears from behind a car or obstacle), etc. Trained systems based on reinforcement learning may be better equipped to deal with continuous and high-dimensional state spaces as well as continuous action spaces.

[0218] Training the system using reinforcement learning may include learning a driving policy to map from sensed states to navigation actions. The driving policy is a function π:S→A, where S is a set of states,

number

[0219] The system can be trained by exposing it to various navigation conditions, having the system apply a policy, and providing rewards (based on a reward function designed to reward desired navigation behavior). Based on the reward feedback, the system can "learn" the policy and become trained in producing the desired navigation behavior. For example, the learning system can learn the current state s t ∈S and set the policy

number

[0220] The goal of reinforcement learning (RL) is to find a policy π. At time t, the state s t It is located in the t A reward function r that measures the instantaneous quality of t However, at time t, action a tTaking an action affects the environment and therefore the value of future states. As a result, when deciding which action to take, one should not only consider the current reward but also future rewards. In some cases, if the system determines that a higher reward can be realized in the future by taking a choice that has a lower reward now, then it should take that action even if that particular action is associated with a lower reward than another choice available. To formalize this, suppose that a policy π and an initial state s are

number

[0221]

number

[0222] Rather than restricting the horizon to T, we can discount future rewards and define the following for some fixed γ ∈ (0,1):

[0223]

number

[0224] In any case, the optimal policy is

[0225]

number

[0226] where the expectation is over the initial state s.

[0227] There are several possible methodologies for training a driving policy system. For example, the system can use imitation techniques (e.g., behavior cloning) that learn from state / action pairs, where the actions are selected by a good agent (e.g., a human) in response to a particular observed state. Suppose a human driver is observed. The observations provide a basis for training the driving policy system, i.e., (s t ,a t ) format (s t is the state, and a t We can obtain, observe, and use many examples of π(s t )≒a t We can use supervised learning to learn the policy π such that ||π(s t )-a t It is often impossible to learn functions where || is very small. Moreover, even small errors can accumulate over time to produce large errors.

[0228] Another technique that can be used is policy-based learning, where policies are expressed in parametric form and can be directly optimized using a suitable optimization technique (e.g., stochastic gradient descent).

number

[0229] The system can also be trained by value-based learning (learning the Q-function or V-function). The optimal value function V * We assume that we can learn a good approximation for . An optimal policy can be constructed (e.g., by using the Bellman equation). Some versions of value-based learning can be performed offline (called "off-policy" training). Some disadvantages of value-based approaches can be attributed to their strong reliance on Markov assumptions and required approximations of complex functions (it can be harder to approximate a value function than to directly approximate a policy).

[0230] Another technique may include model-based learning and planning (learning the probabilities of state transitions and solving an optimization problem to find the optimal V). A combination of these techniques can also be used to train a learning system. In this approach, the dynamics of the process, i.e. (s t ,a t ) and the next state s t+1 Once this function is learned, we can solve an optimization problem to find the policy π whose value is optimal. This is called "planning". One advantage of this approach is that the learning part is supervised, and we can t ,a t ,s t+1) can be applied offline by observing the learning process. As with the "imitation" approach, one disadvantage of this approach can be that small errors in the learning process can accumulate, resulting in a poorly performing policy.

[0231] Another approach for training the driving policy module 803 may include decomposing the driving policy function into semantically significant components. Doing so allows for manual implementation of parts of the policy, which may guarantee the safety of the policy, and for other parts of the policy to be implemented using reinforcement learning methods, which may enable adaptability to many scenarios, human-like balance between defensive / aggressive behavior, and human-like negotiation with other drivers. From a technical point of view, reinforcement learning methods can combine several methodologies to provide a tractable training procedure, where most of the training can be performed using recorded data or a self-built simulator.

[0232] In some embodiments, training of the driving policy module 803 can utilize an "option" mechanism. To illustrate this, consider a simple scenario of driving policy for a two-lane highway. In a direct RL approach, we define the state as

number

[0233] Autonomous Cruise Control (ACC) policy, ACC :S→A: This policy always outputs a yaw rate of 0 and only changes speed to provide smooth, accident-free driving.

[0234] ACC+Left Policy, o L:S → A: The longitudinal commands in this policy are the same as the ACC commands. The yaw rate is a simple implementation of centering the vehicle towards the center of the left lane while ensuring safe lateral movements (e.g., no moving left if there is a car on the left).

[0235] ACC+Right Policy, o R :S→A:o L Same as, but the vehicle may be centered towards the center of the right lane.

[0236] These policies can be called "alternatives". The policy that depends on these "alternatives" and selects the alternatives is o : We can learn S → O, where O is the set of available options. In one case, O = {o ACC ,o L ,o R For all s for which} holds,

number

[0237] In practice, a policy function can be decomposed into a choice graph 901 as shown in FIG. 9. Another example of a choice graph 1000 is shown in FIG. 10. A choice graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There is a special node called the root node 903 of the graph. This node has no input nodes. The decision process starts from the root node and traverses the graph until it reaches a "leaf" node that points to a node that has no output decision line. As shown in FIG. 9, the leaf nodes can include, for example, nodes 905, 907, and 909. When a leaf node is encountered, the driving policy module 803 can output acceleration and steering commands associated with the desired navigation behavior associated with the leaf node.

[0238] For example, an interior node such as nodes 911, 913, 915, etc. may implement a policy that selects a child from among its available choices. The set of available children of an interior node includes all of the nodes that are related to the particular interior node by decision lines. For example, interior node 913, shown in Figure 9 as "Merge," includes three child nodes 909, 915, and 917 ("Stay," "Overtake Right," and "Overtake Left," respectively) that are each connected to node 913 by decision lines.

[0239] By allowing nodes to adjust their position in the hierarchy of the graph of alternatives, flexibility of the decision-making system can be obtained. For example, any of the nodes can be allowed to declare itself as "critical". Each node can implement a function "is critical" that outputs "true" if the node is within the critical section of its policy implementation. For example, the node responsible for the takeover can declare itself as critical during the operation. This can impose a constraint on the set of available children of node u, which can include all nodes v that are children of node u and for which there exists a path from v to a leaf node that passes through all nodes designated as critical. Such an approach can on the one hand allow the declaration of a desired path on the graph at each time step, while on the other hand it can preserve the stability of the policy, especially while the critical parts of the policy are being implemented.

[0240] By defining a graph of options, the problem of learning a driving policy π:S→A can be decomposed into the problem of defining a policy for each node of the graph, where the policy at the internal node should be selected among the available child nodes. For some of the nodes, the individual policies can be implemented manually (e.g., by an if-then type algorithm that specifies a set of actions depending on the observed states), while for others, the policies can be implemented using a trained system built by reinforcement learning. The choice between a manual approach or a trained / learned approach can depend on the safety aspects related to the task and its relative simplicity. The graph of options can be constructed in such a way that some of the nodes are easily implemented, while other nodes can rely on trained models. Such an approach can ensure the safe operation of the system.

[0241] The following discussion provides further details regarding the role of the choice graph of Figure 9 within the driving policy module 803. As discussed above, the input to the driving policy module is a "sensed state" outlining, for example, a map of the environment obtained from available sensors. The output of the driving policy module 803 is a set of desires (optionally together with a set of hard constraints) that define a trajectory as a solution to an optimization problem.

[0242] As mentioned above, the graph of options represents a hierarchical set of decisions organized as a DAG. There is a special node called the "root" of the graph. The root node is the only node that has no incoming edges (e.g., decision lines). The decision process starts from the root node and traverses the graph until a "leaf" node is reached, i.e., a node that has no outgoing edges. Each internal node should implement a policy to choose one of its available children. Every leaf node should implement a policy that defines a set of desires (e.g., a set of navigational objectives for the host vehicle) based on the entire path from the root to the leaf. The set of desires, together with a set of hard constraints that are directly defined based on the sensed conditions, establish an optimization problem whose solution is the trajectory of the vehicle. The hard constraints can be used to further increase the safety of the system, and the desires can be used to provide driving comfort and human-like driving behavior of the system. The trajectory provided as a solution to the optimization problem thus defines the commands to be given to the steering, braking and / or engine actuators to achieve the trajectory.

[0243] Returning to FIG. 9 , the option graph 901 represents an option graph for a two-lane highway with a merging lane (meaning that at some point the third lane merges into the right or left lane of the highway). The root node 903 first determines if the host vehicle is in a simple road scenario or approaching a merging scenario. This is an example of a decision that can be made based on the sensing conditions. The simple road node 911 includes three child nodes: a stay node 909, a left overtaking node 917, and a right overtaking node 915. Staying refers to a situation where the host vehicle wants to continue traveling in the same lane. The stay node is a leaf node (no outgoing edges / lines). Thus, the stay node defines a set of wishes. The first wish defined by this node may include, for example, a desired lateral position as close as possible to the center of the current lane of travel. There may also be a wish to navigate smoothly (e.g., within a predefined or allowable maximum acceleration). The stay node may also define how the host vehicle reacts to other vehicles. For example, a stationary node can survey detected target vehicles and assign each a semantic meaning that can be translated into components of a trajectory.

[0244] Various semantic meanings can be assigned to target vehicles in the host vehicle's environment. For example, in some embodiments, the semantic meanings can include any of the following indications: 1) Not involved: indicates that the detected vehicle in the scene is not currently involved; 2) Next lane: indicates that the detected vehicle is in an adjacent lane and should maintain an appropriate offset relative to that vehicle (the exact offset can be calculated in an optimization problem that constructs a trajectory given the desires and hard constraints, and can possibly be vehicle dependent, and the leaf of the choice graph that stays on sets the semantic type of the target vehicle that defines the desire for the target vehicle); 3) Give way: the host vehicle attempts to give way to the detected target vehicle (especially if the host vehicle determines that the target vehicle is likely to cut in to the host vehicle's lane), e.g., by slowing down; 4) Lead: the host vehicle attempts to accept and comply with the right of way, e.g., by accelerating; 5) Follow: the host vehicle wishes to follow this target vehicle and maintain a smooth drive; 6) Overtake left / right: this means that the host vehicle wants to initiate a lane change to the left or right lane. Left overtaking node 917 and right overtaking node 915 are interior nodes that have not yet determined a wish.

[0245] The next node in the options graph 901 is a gap selection node 919. This node may be responsible for selecting a gap between two target vehicles in a particular target lane that the host vehicle wishes to enter. By selecting a node of the form IDj, for some value of j, the host vehicle arrives at a leaf that specifies a desire for the trajectory optimization problem, e.g., a maneuver that the host vehicle wishes to perform to reach the selected gap. Such a maneuver may include first accelerating / braking in the current lane and proceeding to the target lane at a suitable time to enter the selected gap. If the gap selection node 919 cannot find a suitable gap, it proceeds to an abort node 921 that defines a desire to return to the center of the current lane and cancel the overtaking.

[0246] Returning to the merging node 913, as the host vehicle approaches a merge, the host vehicle has several options that may depend on the particular situation. For example, as shown in FIG 11A, the host vehicle 1105 is traveling along a two-lane road without detecting another target vehicle in the main or merging lane 1111 of the two-lane road. In this situation, the driving policy module 803 may select the remain node 909 upon reaching the merging node 913. That is, if it does not detect a target vehicle as merging onto the road, it may be desirable to remain in one's current lane.

[0247] 11B, the situation is slightly different, where the host vehicle 1105 detects one or more target vehicles 1107 entering the main road 1112 from the merging lane 1111. In this situation, when the driving policy module 803 faces the merging node 913, the driving policy module 803 may decide to initiate a left overtaking maneuver to avoid the merging situation.

[0248] In FIG. 11C , the host vehicle 1105 encounters one or more target vehicles 1107 entering the main road 1112 from a merging lane 1111. The host vehicle 1105 also detects a target vehicle 1109 moving in a lane adjacent to the host vehicle's lane. The host vehicle also detects one or more target vehicles 1110 moving in the same lane as the host vehicle 1105. In this situation, the driving policy module 803 may decide to adjust the speed of the host vehicle 1105 to yield to the target vehicle 1107 and move ahead of the target vehicle 1115. This may be achieved, for example, by proceeding to the gap selection node 919, which selects the gap between ID0 (vehicle 1107) and ID1 (vehicle 1115) as the appropriate merging gap. The appropriate gap of the merging situation then defines the objective of the trajectory planner optimization problem.

[0249] As discussed above, a node in the graph of options can declare itself as "critical", and such a declaration can ensure that the option selected passes through the critical node. Formally, each node can implement a function IsCritical. After making a forward pass from root to leaf on the graph of options and solving the trajectory planner optimization problem, a backward pass can be made from the leaf to the root. Along this backward pass, the IsCritical function of all nodes in the path can be called, and a list of all critical nodes can be saved. In the forward path corresponding to the next time slot, the driving policy module 803 can be requested to select a path from the root node to the leaf node that passes through all critical nodes.

[0250] 11A-11C can be used to illustrate the potential benefits of this approach. For example, in a situation where an overtaking maneuver is initiated and the driving policy module 803 reaches the leaf corresponding to IDk, it is undesirable to select, for example, the stay node 909 if the host vehicle is in the middle of the overtaking maneuver. To avoid this jumpiness, the IDj node can designate itself as critical. During the maneuver, the success of the trajectory planner can be monitored, and the function IsCritical returns a "true" value if the overtaking maneuver proceeds as intended. This approach can ensure that the overtaking maneuver continues in the next time frame (rather than jumping to another potentially inconsistent maneuver before completing the initially selected maneuver). On the other hand, if the monitoring of the maneuver indicates that the selected maneuver is not proceeding as intended or if the maneuver becomes unnecessary or impossible, the function IsCritical can return a "false" value. This can enable the gap selection node to select a different gap in the next time frame, or to abort the overtaking maneuver entirely. On the one hand, this approach may enable the declaration of a desired path through a graph of alternatives at each time step, while on the other hand, it may help promote stability of the policy during critical parts of the execution.

[0251] The strict constraints, which will be described in more detail below, can be distinguished from the navigation aspirations. For example, the strict constraints can ensure safe driving by applying an additional layer of filtering of the planned navigation operation. The involved strict constraints, which can be manually programmed and defined, can be determined from the sensed state, rather than by using a trained system built on reinforcement learning. However, in some embodiments, the trained system can learn the applicable strict constraints to be applied and followed. Such an approach can encourage the driving policy module 803 to arrive at a selected action that already complies with the applicable strict constraints, thereby reducing or eliminating selected actions that may require later modification to comply with the applicable strict constraints. Nevertheless, as a redundant safety measure, the strict constraints can be applied to the output of the driving policy module 803 even if the driving policy module 803 is trained to take into account the given strict constraints.

[0252] There are many examples of potential strict constraints. For example, a strict constraint may be defined in relation to guard rails at the edge of the road. Under no circumstances is the host vehicle allowed to cross the guard rail. Such a rule would cause a strict lateral constraint on the trajectory of the host vehicle. Another example of a strict constraint may include road bumps (e.g., speed control bumps), which may cause a strict constraint on the driving speed before or while crossing the bump. Strict constraints may be considered safety-critical and therefore may be defined manually, rather than relying exclusively on a trained system to learn the constraints during training.

[0253] In contrast to strict constraints, the goal of aspirations may be to enable or achieve a comfortable driving experience. As discussed above, an example of aspiration may include a goal to position the host vehicle in a lateral position within the lane that corresponds to the center of the host vehicle's lane. Another aspiration may include the ID of a gap to move into. Note that the host vehicle does not need to be strictly in the center of the lane, and instead aspiration to be as close to the center of the lane as possible can ensure that the host vehicle is likely to move to the center of the lane if it deviates from the center of the lane. Aspirations may not be safety-critical. In some embodiments, aspirations may require negotiation with other drivers and pedestrians. One approach to constructing aspirations may utilize a graph of options, and the policies implemented within at least some nodes of the graph may be based on reinforcement learning.

[0254] For nodes of the graph of choices 901 or 1000 that are implemented as nodes that are trained based on learning, the training process may include decomposing the problem into a supervised learning phase and a reinforcement learning phase. In the supervised learning phase:

number

number

number

number

[0255] A key element that can be provided in some scenarios is a differentiable path from future losses / rewards back to a decision on action. In a choice graph structure, the implementation of choices including safety constraints is usually not differentiable. To overcome this issue, the selection of children at nodes of the learned policy can be probabilistic. That is, the node can output a probability vector p, which assigns a probability to be used in selecting each of the children of a particular node. Suppose a node has k children, a (1) ,...,a (k) Let be the action of the path from each child to the leaf. Thus, the resulting predicted action is

number

number

[0256] s t , a t Given,

number

[0257] Additionally, in some embodiments, the system may implement a multi-agent approach. For example, the system may consider data from various sources and / or imagery captured from multiple angles. Additionally, some disclosed embodiments may result in energy savings because prediction of events that do not directly involve the host vehicle but may have an impact on the host vehicle may be considered, and prediction of events that may result in unpredictable situations involving other vehicles may be a consideration (e.g., radar may "see through" to unavoidable events that will impact the preceding vehicle and the host vehicle, as well as predicting the high likelihood of such events).

[0258] A trained system with imposed navigation constraints

[0259] In the context of autonomous driving, a critical concern is how to ensure that the learned policy of the trained navigation network is safe. In some embodiments, the driving policy system can be trained using constraints, so that the actions selected by the trained system may already take into account applicable safety constraints. In addition, in some embodiments, an additional layer of safety can be provided by passing the selected actions of the trained system through one or more strict constraints implicated by a particular sensed scene in the host vehicle's environment. Such an approach can ensure that the actions taken by the host vehicle are limited to those that are confirmed to satisfy applicable safety constraints.

[0260] At its core, the navigation system may include a learning algorithm based on a policy function that maps observed states to one or more desired actions. In some implementations, the learning algorithm is a deep learning algorithm. The desired actions may include at least one action expected to maximize the expected reward for the vehicle. In some cases, the actual action taken by the vehicle may correspond to one of the desired actions, while in other cases, the actual action taken may be determined based on the observed states, the one or more desired actions, and non-learning strict constraints (e.g., safety constraints) imposed on the learning navigation engine. These constraints may include non-driving zones surrounding various types of detected objects (e.g., target vehicles, pedestrians, static objects on or in the road, moving objects on or in the road, guard rails, etc.). In some cases, the size of the zone may vary based on the detected movement (e.g., speed and / or direction) of the detected object. Other constraints may include maximum travel speed when passing through pedestrian influence zones, maximum deceleration (to accommodate the spacing of target vehicles behind the host vehicle), mandatory stops at detected pedestrian crossings or railroad crossings, etc.

[0261] Strict constraints used with a system trained by machine learning can provide a degree of safety in autonomous driving that may exceed the degree of safety that can be obtained based on the output of the trained system alone. For example, a machine learning system can be trained using a set of desired constraints as training guidelines, and the trained system can then select actions that are configured with and adhere to the applicable navigation constraints in response to the detected navigation conditions. However, when selecting a navigation action, the trained system still has some flexibility, and therefore there may be at least some situations in which the action selected by the trained system may not strictly follow the relevant navigation constraints. Therefore, in order to require that the action selected strictly follows the relevant navigation constraints, non-machine learning components that ensure strict application of the relevant navigation constraints outside the learning / training framework can be used to combine, compare, filter, adjust, modify, etc. the output of the trained system.

[0262] The following discussion provides further details on the trained system and the potential benefits (especially from a safety perspective) that can be gained from combining the trained system with algorithmic components outside the training / learning framework. As discussed above, a reinforcement learning objective with a policy can be optimized by stochastic gradient ascent. The objective (e.g., expected reward) is

number

[0263] In machine learning scenarios, objectives that include expectations can be used. However, such objectives may not return actions that are strictly bound by navigation constraints without being constrained by those constraints. For example, for a trajectory that represents a rare "turn in the road" event (e.g., an accident) that should be avoided,

number

number

number

number

[0264]

number

number

number

number

[0265] Lemma: π o Let p be a policy, p and r be scalars, so that with probability p,

number

number

[0266]

number

[0267] holds, and the last approximation applies when r≧1 / p.

[0268] This explanation is in the format

number

number

number

number

number

number

number

number

[0269] The double merge navigation situation shown in FIG. 11D provides an example that further illustrates these concepts. In a double merge, vehicles approach the merge area 1130 from both the left and right sides. From each side, a decision can be made whether a vehicle, such as vehicle 1133 or vehicle 1135, will merge into the opposite lane of the merge area 1130. In congested traffic, successfully executing a double merge may require significant negotiation skill and experience, and may be difficult to perform in a heuristic or brute force manner by enumerating all possible trajectories that all agents in the scene may take. In this double merge example, a set of desires D suitable for the double merge operation can be defined. D is the Cartesian product of the following sets: D=[0, v max ]xLx{g,t,o} n where [0,v max

[0046] is the desired target speed of the host vehicle, L={1,1.5,2,2.5,3,3.5,4} is the desired lateral position in lane units, where integers indicate the center of the lane and fractions indicate the lane boundaries, and {g,t,o} are classification labels assigned to each of the other n vehicles. The other vehicles can be assigned "g" if the host vehicle should yield to them, "t" if the host vehicle should gain the right of way relative to them, or "o" if the host vehicle should maintain an offset distance relative to them.

[0270] The following is a set of desires (v,l,c 1 ,...,c n This explains how we can transform a set of trajectories (x 1 ,y1 ),...,(x k ,y k ), and (x i ,y i ) is the position (lateral, longitudinal) of the host vehicle (in egocentric units) at time τ·i. In some experiments, τ=0.1 seconds and k=10. Of course, other values ​​can be chosen. The cost assigned to a trajectory may include a weighted sum of the individual costs assigned to the desired speed, lateral position, and labels assigned to each of the other n vehicles.

[0271] The desired velocity v ∈ [0, v max ], the cost of the trajectory relative to the velocity

number

[0272] Given a desired lateral position, the cost associated with the desired lateral position

number

[0273] where dist(x,y,l) is the distance from point (x,y) to lane position l. Regarding the cost due to other vehicles, for any other vehicle, (x ' 1 ,y ' 1 ),...,(x ' k ,y ' k ) can represent other vehicles in the egocentric unit of the host vehicle, and i can be expressed as (x i ,y i ) and (x ' j ,y ' j) is small. If there is no such point, i can be set as i=∞. If another vehicle is classified as "giving way", it may be desirable for τi>τj+0.5, meaning that the host vehicle reaches that same point at least 0.5 seconds after the other vehicle reaches the trajectory intersection. A possible formula for converting the above constraint to a cost is [τ(ji)+0.5] + It is.

[0274] Similarly, if another car is classified as “taking the road”, it may be desirable for τj > τi + 0.5, which means that the cost [τ(ij) + 0.5] + If another vehicle is classified as "offset", it may be desirable for i = ∞, meaning that the host vehicle's trajectory and the offset vehicle's trajectory do not intersect. This condition can be translated into a cost by penalizing it with respect to the distance between the trajectories.

[0275] Assigning weights to each of these costs yields a single objective function π (T) Costs that promote smooth driving can be added to the objective. Hard constraints can be added to the objective to ensure the functional safety of the track. For example, (x i、 y i ) may be prohibited from going off the road, (x i、 y i ) is the time when |ij| is small and any other vehicle's trajectory point (x ' j、 y ' j ) for (x ' j、 y ' j ) may be prohibited from approaching.

[0276] In summary, policy θcan be decomposed into an agnostic mapping from states to a set of desires and a mapping from desires to actual trajectories. The latter mapping is not based on learning and can be implemented by solving an optimization problem whose costs depend on the desires and whose hard constraints can guarantee the functional safety of the policy.

[0277] The following exposition describes the mapping from an agnostic state to a set of desires. As explained above, to comply with functional safety, a system that relies solely on reinforcement learning must use reward-based learning.

number

[0278] For various reasons, the decision can be further decomposed into semantically meaningful components. For example, the size of D may be large and continuous. In the double confluence scenario described above with respect to FIG. 11D, D=[0, v max ]xLx{g,t,o} n ) holds. In addition, the gradient estimator is

number

[0279] Returning to the concept of a graph of options, FIG. 11E shows a graph of options that may represent the double-join scenario shown in FIG. 11D. As discussed above, a graph of options may represent hierarchical set decisions organized as a directed acyclic graph (DAG). There may be a special node in the graph called the "root" node 1140, which is the only node that has no incoming edges (e.g., decision lines). The decision process may start from the root node and traverse the graph until it reaches a "leaf" node, i.e., a node that has no outgoing edges. Each internal node may implement a policy function that chooses one child from among its available children. There may be a default mapping from the set of traversals on the graph of options to the set of desires D. In other words, a traversal on the graph of options may be automatically converted to a desire in D. Given a node v in the graph, a parameter vector θ v may specify a policy for selecting children of v. If θ is a set of all θ v If v is a node with θ v By traversing the graph from the root to the leaves, selecting child nodes using the policy defined by

number

[0280] In the double merge option graph 1139 of FIG. 11E, the root node 1140 may first determine whether the host vehicle is in a merge area (e.g., area 1130 of FIG. 11D) or whether instead the host vehicle is approaching a merge area and needs to prepare for a possible merge. In either case, the host vehicle may need to decide whether to change lanes (e.g., to the left or right) or whether to stay in the current lane. If the host vehicle decides to change lanes, the host vehicle may need to continue and determine whether conditions are suitable to perform a lane change maneuver (e.g., at “Go” node 1142). If it is not possible to change lanes, the host vehicle may attempt to “push” toward the desired lane (e.g., at node 1144) by aiming to be on a lane mark. Alternatively, the host vehicle may choose to “stay” in the same lane (e.g., at node 1146). This process may determine the lateral position of the host vehicle in a natural way. For example:

[0281] This may allow the desired lateral position to be determined in a natural way. For example, if the host vehicle changes lanes from lane 2 to lane 3, the "go" node may set the desired lateral position to 3, the "stay" node may set the desired lateral position to 2, and the "push" node may set the desired lateral position to 2.5. The host vehicle may then decide to keep the "same" speed (node ​​1148), to "accelerate" (node ​​1150), or to "slow down" (node ​​1152). The host vehicle may then enter a "chain" structure 1154 that looks at other vehicles and sets their semantic meanings to values ​​in the set {g,t,o}. This process may set aspirations for other vehicles. The parameters of all nodes in the chain may be shared (similar to recurrent neural networks).

[0282] A potential benefit of the choice is the interpretability of the results. Another potential benefit is that we can exploit the decomposable structure of the set D, and thus choose the policy at each node from a small number of possibilities. In addition, the structure may allow us to reduce the variance of the policy gradient estimator.

[0283] As discussed above, the length of an episode in a double merge scenario may be approximately T=250 steps. This value (or any other suitable value depending on the particular navigation scenario) may allow enough time to acknowledge the consequences of the host vehicle's actions (e.g., if the host vehicle decides to change lanes in preparation for a merge, the host vehicle acknowledges the benefit only after successfully completing the merge). On the other hand, driving dynamics require the host vehicle to make decisions at a fast enough frequency (e.g., 10 Hz in the above case).

[0284] The graph of options may allow the effective value of T to be reduced in at least two ways. First, given the high-level decisions, rewards can be defined for low-level decisions while taking into account shorter episodes. For example, if the host vehicle has already selected the "change lane" and "go" nodes, a policy for assigning semantic meaning to vehicles can be learned by looking at 2-3 second episodes (meaning T will be 20-30 instead of 250). Second, for high-level decisions (such as whether to change lanes or stay in the same lane), the host vehicle may not need to make a decision every 0.1 seconds. Instead, the host vehicle may be able to make decisions less frequently (e.g., every second), or may implement a "choice termination" function, where the gradient is calculated only after every termination of the choice. In either case, the effective value of T may be an order of magnitude smaller than its original value. Overall, the estimators at all nodes may rely on a value of T that is an order of magnitude smaller than the original 250 steps, which may immediately translate into smaller variance.

[0285] As discussed above, strict constraints can promote safer driving and there can be several different types of constraints. For example, strict static constraints can be determined directly from sensed conditions. These can include speed bumps, speed limits, road curvatures, intersections, etc. in the host vehicle's environment, which can involve one or more constraints on the vehicle's speed, heading, acceleration, braking (deceleration), etc. Strict static constraints can also include semantic free space, where the host vehicle is prohibited, for example, from going outside the free space and from navigating too close to physical barriers. Strict static constraints can also limit (e.g., prohibit) maneuvers that do not comply with various aspects of the vehicle's kinematic motion, for example, strict static constraints can be used to prohibit maneuvers that could lead to the host vehicle rolling over, skidding, or otherwise losing control.

[0286] Strict constraints can also be vehicle related. For example, a constraint can be used that requires the vehicle to keep at least 1 meter longitudinal distance to other vehicles and at least 0.5 meter lateral distance from other vehicles. Constraints can also be applied to avoid the host vehicle keeping on a collision course with one or more other vehicles. For example, time τ can be a measure of time based on a particular scene. The predicted trajectories of the host vehicle and one or more other vehicles from the current time to time τ can be considered. If the two trajectories intersect,

number

number

number

number

[0287] The time τ to track the trajectory of the host vehicle and one or more other vehicles can vary, except that in an intersection scenario where speeds may be low, τ can be longer and can be determined such that the host vehicle enters and exits the intersection in less than τ seconds.

[0288] Of course, applying strict constraints on the trajectories of the vehicles requires that their trajectories be predicted. For the host vehicle, predicting the trajectory may be relatively straightforward since the host vehicle generally already knows and actually plans its intended trajectory at any given time. For other vehicles, predicting their trajectories may be less straightforward. For other vehicles, the baseline calculations for determining the predicted trajectory may depend on the current speed and heading of the other vehicles, which may be determined, for example, based on an analysis of image streams captured by one or more cameras and / or other sensors (radar, lidar, acoustic, etc.) on board the host vehicle.

[0289] However, there may be some exceptions that simplify the problem or at least provide more confidence in the predicted trajectory of another vehicle. For example, for structured roads where there are lane indications and there may be yielding rules, the trajectory of the other vehicle may be based at least in part on the position of the other vehicle relative to the lane and based on applicable yielding rules. Thus, in some situations, when there is an observed lane structure, vehicles in adjacent lanes may be assumed to adhere to lane boundaries. That is, the host vehicle may assume that the vehicle in the adjacent lane will stay in its lane unless there is observed evidence (e.g., signal lights, strong lateral movement, movement across lane boundaries) that indicates that the vehicle in the adjacent lane will cut into the host vehicle's lane.

[0290] Other circumstances may also provide hints as to the expected trajectory of other vehicles: for example, at stop signs, traffic lights, roundabouts, etc., where the host vehicle may have the right-of-way, it can be assumed that other vehicles will respect that right-of-way, and therefore, unless there is evidence that the rule has been observed to be violated, it can be assumed that other vehicles will proceed along a trajectory that respects the right-of-way held by the host vehicle.

[0291] Hard constraints may also be applied with respect to pedestrians in the host vehicle's environment. For example, a buffer distance for pedestrians may be established such that the host vehicle is prohibited from navigating any closer than a specified buffer distance to any observed pedestrian. The buffer distance for pedestrians may be any suitable distance. In some embodiments, the buffer distance may be at least one meter to an observed pedestrian.

[0292] Similar to the vehicle situation, strict constraints can also be applied to the relative motion between a pedestrian and a host vehicle. For example, the pedestrian's trajectory (based on heading and speed) can be monitored against the host vehicle's predicted trajectory. Given a particular pedestrian's trajectory, for every point p on the trajectory, t(p) may represent the time it takes for the pedestrian to reach point p. To keep the required buffer distance of at least one meter from the pedestrian, t(p) must be greater than the time the host vehicle reaches point p (with a sufficient time difference so that the host vehicle passes in front of the pedestrian with a difference of at least one meter), or t(p) must be less than the time the host vehicle reaches point p (e.g., if the host vehicle brakes to yield to the pedestrian). Furthermore, in the latter example, a strict constraint may require that the host vehicle reaches point p at a time sufficiently slower than the pedestrian so that the host vehicle can pass behind the pedestrian and keep the required buffer distance of at least one meter. Of course, there may be exceptions to the pedestrian strict constraint. For example, if the host vehicle has the right-of-way or is moving very slowly and there is no observed evidence of pedestrians refusing to yield to the host vehicle or navigating toward the host vehicle, the strict pedestrian constraints can be relaxed (e.g., to a narrower buffer of at least 0.75 meters or 0.50 meters).

[0293] In some cases, constraints may be relaxed if it is determined that all cannot be satisfied. For example, in a situation where the road is too narrow to allow a desired distance (e.g., 0.5 meters) from both curves or from a curve and a parked vehicle, one or more of the constraints may be relaxed if there are mitigating reasons. For example, one may proceed slowly at 0.1 meters from a curve if there are no pedestrians (or other objects) on the sidewalk. In some embodiments, constraints may be relaxed if doing so improves the user experience. For example, to avoid potholes, constraints may be relaxed to allow the vehicle to navigate closer to lane edges, curves, or pedestrians than would normally be allowed. Furthermore, when determining which constraints to relax, in some embodiments, the one or more constraints that are decided to be relaxed are those that are deemed to have the least adverse effect on safety. For example, constraints on how close a vehicle can travel to a curve or concrete barrier may be relaxed before relaxing constraints dealing with proximity to other vehicles. In some embodiments, pedestrian constraints may be the last to be relaxed, or may never be relaxed depending on the circumstances.

[0294] FIG. 12 illustrates an example of a scene that may be captured and analyzed during navigation of a host vehicle. For example, the host vehicle may include a navigation system (e.g., system 100) as described above that may receive a plurality of images representing the host vehicle's environment from a camera (e.g., at least one of image capture device 122, image capture device 124, and image capture device 126) associated with the host vehicle. The scene illustrated in FIG. 12 is an example of one of the images that may be captured at time t from the environment of the host vehicle traveling in lane 1210 along predicted trajectory 1212. The navigation system may include at least one processing device (e.g., including any of the EyeQ processors or other devices described above) that is specifically programmed to receive a plurality of images and analyze the images to determine an action in response to the scene. Among other things, the at least one processing device may implement the sensing module 801, driving policy module 803, and control module 805 illustrated in FIG. 8. The sensing module 801 may be responsible for collecting and outputting image information collected from the camera and providing that information in the form of identified navigation states to the driving policy module 803, which may constitute a trained navigation system that has been trained by machine learning methods such as supervised learning and reinforcement learning. Based on the navigation state information provided by the sensing module 801 to the driving policy module 803, the driving policy module 803 may generate a desired navigation action to be performed by the host vehicle in response to the identified navigation states (e.g., by implementing the option graph approach described above).

[0295] In some embodiments, at least one processing device, for example using the control module 805, may directly translate the desired navigation behavior into navigation commands. However, in other embodiments, hard constraints may be applied to test the desired navigation behavior provided by the driving policy module 803 against various predefined navigation constraints that may be implicated by the scene and the desired navigation behavior. For example, if the driving policy module 803 outputs a desired navigation behavior that causes the host vehicle to follow a trajectory 1212, that navigation behavior may be tested against one or more hard constraints associated with various aspects of the host vehicle's environment. For example, the captured image 1201 may determine a curve 1213, a pedestrian 1215, a target vehicle 1217, and a static object (e.g., an upturned box) present in the scene. Each of these may be associated with one or more hard constraints. For example, the curve 1213 may be associated with a static constraint that prevents the host vehicle from navigating into or over the curve onto the sidewalk 1214. The curve 1213 may also be associated with a road barrier envelope that defines a predetermined distance (e.g., a buffer zone) extending from and along the curve (e.g., 0.1 meters, 0.25 meters, 0.5 meters, 1 meter, etc.) that defines a non-navigable area for the host vehicle. Of course, static constraints may be associated with other types of roadside boundaries (e.g., guard rails, concrete pillars, cones, pylons, any other type of roadside barrier).

[0296] It should be noted that distance and ranging can be determined by any suitable method. For example, in some embodiments, distance information can be provided by an on-board radar and / or lidar system. Alternatively or additionally, distance information can be derived by analyzing one or more images captured from the host vehicle's environment. For example, the number of pixels of a recognized object represented in an image can be determined and compared to a known field of view and focal length geometry of the image capture device to determine scale and distance. For example, velocity and acceleration can be determined by observing the change in scale between objects from image to image over a known time interval. This analysis can indicate how fast an object is moving away from or approaching the host vehicle, as well as the direction of movement towards or away from the host vehicle. A traversal speed can be determined by analyzing the position change of the object's X coordinate from one image to another over a known time period.

[0297] The pedestrian 1215 may be associated with a pedestrian envelope that defines a buffer zone 1216. In some cases, the imposed strict constraint may prohibit the host vehicle from navigating within a distance of 1 meter from the pedestrian 1215 (in any direction relative to the pedestrian). The pedestrian 1215 may also define the location of a pedestrian influence zone 1220. This influence zone may be associated with a constraint that limits the speed of the host vehicle within the influence zone. The influence zone may extend 5 meters, 10 meters, 20 meters, etc. from the pedestrian 1215. A different speed limit may be associated with each class of the influence zone. For example, within a zone between 1 meter and 5 meters from the pedestrian 1215, the host vehicle may be limited to a first speed (e.g., 10 mph, 20 mph, etc.) that may be less than the speed limit within the pedestrian influence zone that extends from 5 meters to 10 meters. Any class for the various stages of the influence zone may be used. In some embodiments, the first stage can be narrower than 1 meter to 5 meters and may span only 1 meter to 2 meters. In other embodiments, the first stage of the influence zone may extend from 1 meter (the boundary of the non-navigable zone around the pedestrian) to a distance of at least 10 meters. The second stage may then extend from 10 meters to at least about 20 meters. The second stage may be associated with a maximum travel speed of the host vehicle that exceeds the maximum travel speed associated with the first stage of the pedestrian influence zone.

[0298] One or more static object constraints may also be implicated by the detected scene in the host vehicle's environment. For example, in image 1201, at least one processing device may detect a static object, such as a box 1219 in the road. The detected static object may include various objects, such as trees, poles, road signs, and / or objects in the road. One or more predefined navigation constraints may be associated with the detected static object. For example, such constraints may include a static object envelope that defines a buffer zone around the object within which navigation of the host vehicle may be prohibited. At least a portion of the buffer zone may extend a predetermined distance from an edge of the detected static object. For example, in the scene represented by image 1201, a buffer zone of at least 0.1 meters, 0.25 meters, 0.5 meters, or more may be associated with box 1219, such that the host vehicle passes to the right or left of the box by at least some distance (e.g., the distance of the buffer zone) to avoid a collision with the detected static object.

[0299] The predefined hard constraints may also include one or more target vehicle constraints. For example, a target vehicle 1217 may be detected in the image 1201. One or more hard constraints may be used to ensure that the host vehicle does not collide with the target vehicle 1217. In some cases, the target vehicle envelope may be associated with a single buffer zone distance. For example, the buffer zone may be defined by a distance of one meter surrounding the target vehicle in all directions. The buffer zone may define an area extending at least one meter from the target vehicle within which the host vehicle is prohibited from navigating.

[0300] However, the envelope surrounding the target vehicle 1217 need not be defined by a fixed buffer distance. In some cases, the predefined strict constraints associated with the target vehicle (or any other movable object detected in the host vehicle's environment) may depend on the orientation of the host vehicle relative to the detected target vehicle. For example, in some cases, the distance of the longitudinal buffer zone (e.g., extending from the target vehicle to the front or rear of the host vehicle, such as when the host vehicle is traveling toward the target vehicle) may be at least 1 meter. The distance of the lateral buffer zone (e.g., extending from the target vehicle to either side of the host vehicle, such as when the host vehicle is traveling in the same or opposite direction as the target vehicle, such that the side of the host vehicle passes directly beside the side of the target vehicle) may be at least 0.5 meter.

[0301] As explained above, other constraints may come into play by detecting target vehicles or pedestrians in the host vehicle's environment. For example, the predicted trajectories of the host and target vehicles 1217 may be considered, and when the two trajectories intersect (e.g., at intersection 1230), a hard constraint may be:

number

[0302] Other hard constraints may also be used. For example, in at least some cases, a maximum deceleration rate of the host vehicle may be used. This maximum deceleration rate may be determined based on the detected distance to a target vehicle following the host vehicle (e.g., using images collected from a rear-facing camera). Hard constraints may include mandatory stops at detected pedestrian or railroad crossings or other applicable constraints.

[0303] If the analysis of the scene in the host vehicle's environment indicates that one or more predefined navigation constraints may be engaged, those constraints may be imposed on one or more planned navigation operations of the host vehicle. For example, if the analysis of the scene results in the driving policy module 803 returning a desired navigation operation, the desired navigation operation may be tested against the one or more engaged constraints. If it is determined that the desired navigation operation violates any aspect of the engaged constraints (e.g., if the desired navigation operation brings the host vehicle within 0.7 meters of the pedestrian 1215 when a predefined strict constraint requires the host vehicle to stay at least 1.0 meters from the pedestrian 1215), at least one modification to the desired navigation operation may be made based on the one or more predefined navigation constraints. Adjusting the desired navigation operation in this manner may result in the actual navigation operation of the host vehicle complying with the constraints engaged by the particular scene detected in the host vehicle's environment.

[0304] After determining the actual navigational behavior of the host vehicle, the navigational behavior can be implemented by causing adjustment of at least one navigation actuator of the host vehicle in response to the determined actual navigational behavior of the host vehicle, which may include at least one of a steering mechanism, a brake, or an accelerator of the host vehicle.

[0305] Prioritized Constraints

[0306] As explained above, various strict constraints can be used with the navigation system to ensure safe operation of the host vehicle. The constraints can include, among others, a minimum safe driving distance to a pedestrian, target vehicle, road barrier, or detected object, a maximum travel speed when passing within the influence zone of a detected pedestrian, or a maximum deceleration rate of the host vehicle. These constraints can be imposed by a trained system that is trained based on machine learning (supervised, reinforced, or a combination thereof), but can also be useful for untrained systems (e.g., using algorithms that directly address expected situations that arise within the scene of the host vehicle's environment).

[0307] In any case, there may be a hierarchy of constraints. In other words, some navigation constraints may take precedence over other navigation constraints. Thus, if a situation occurs where no navigation action is available that would satisfy all involved constraints, the navigation system may determine an available navigation action that will first realize the highest priority constraint. For example, the system may have the vehicle avoid pedestrians first, even if navigation to avoid pedestrians would cause a collision with another vehicle or object detected in the road. In another example, the system may have the vehicle ride up a curve to avoid pedestrians.

[0308] FIG. 13 shows a flow chart illustrating an algorithm for implementing a hierarchy of involved constraints determined based on an analysis of a scene in the host vehicle's environment. For example, in step 1301, at least one processor associated with the navigation system (e.g., an EyeQ processor, etc.) may receive a plurality of images representing the host vehicle's environment from an on-board camera of the host vehicle. In step 1303, a navigation state associated with the host vehicle may be determined by analyzing the one or more images representing the scene in the host vehicle's environment. For example, the navigation state may indicate, among other characteristics of the scene, that the host vehicle is traveling along a two-lane road 1210 as shown in FIG. 12, that a target vehicle 1217 is proceeding through an intersection ahead of the host vehicle, that a pedestrian 1215 is waiting to cross the road along which the host vehicle is traveling, and that there is an object 1219 ahead of the host vehicle's lane.

[0309] In step 1305, one or more navigation constraints implicated by the host vehicle's navigation state may be determined. For example, the at least one processing device may analyze a scene in the host vehicle's environment represented by one or more captured images and then determine one or more navigation constraints implicated by objects, vehicles, pedestrians, etc. recognized by image analysis of the captured images. In some embodiments, the at least one processing device may determine at least a first predefined navigation constraint and a second predefined navigation constraint implicated by the navigation state, where the first predefined navigation constraint may be different from the second predefined navigation constraint. For example, the first navigation constraint may relate to one or more target vehicles detected in the host vehicle's environment, and the second navigation constraint may relate to pedestrians detected in the host vehicle's environment.

[0310] In step 1307, the at least one processing device may determine a priority associated with the constraints identified in step 1305. In the illustrated example, a second predefined navigation constraint related to pedestrians may have a higher priority than a first predefined navigation constraint related to the target vehicle. Although the priority associated with a navigation constraint may be determined or assigned based on various factors, in some embodiments, the priority of a navigation constraint may relate to its relative importance from a safety perspective. For example, while it may be important to follow or satisfy all implemented navigation constraints in as many situations as possible, some constraints may be associated with stronger safety risks than other constraints and may therefore be assigned a higher priority. For example, a navigation constraint that requires the host vehicle to keep at least one meter away from pedestrians may have a higher priority than a constraint that requires the host vehicle to keep at least one meter away from the target vehicle. This may be because a collision with a pedestrian may have more severe consequences than a collision with another vehicle. Similarly, maintaining a gap between the host vehicle and the target vehicle may have a higher priority than constraints requiring the host vehicle to avoid boxes in the road, to drive below a certain speed over speed bumps, or to expose occupants of the host vehicle to less than maximum acceleration levels.

[0311] Although the driving policy module 803 is designed to maximize safety by satisfying navigation constraints implicated by a particular scene or navigation state, it may be physically impossible to satisfy all constraints implicated by a situation. In such a situation, the priority of each implicated constraint may be used to determine which of the implicated constraints should be satisfied first, as shown in step 1309. Continuing with the above example, in a situation where both the pedestrian gap constraint and the target vehicle gap constraint cannot be satisfied and only one of the constraints can be satisfied, the higher priority of the pedestrian gap constraint may result in the constraint being satisfied before attempting to maintain a gap to the target vehicle. Thus, in a normal situation, as shown in step 1311, if both the first predefined navigation constraint and the second predefined navigation constraint can be satisfied, the at least one processing device may determine a first navigation action for the host vehicle that satisfies both the first predefined navigation constraint and the second predefined navigation constraint based on the identified navigation state of the host vehicle. However, in other situations where all involved constraints cannot be satisfied, as shown in step 1313, if both the first predefined navigation constraint and the second predefined navigation constraint cannot be satisfied, the at least one processing device may determine a second navigation action for the host vehicle based on the identified navigation state that satisfies the second predefined navigation constraint (i.e., the constraint with a higher priority) but does not satisfy the first predefined navigation constraint (which has a lower priority than the second navigation constraint).

[0312] Next, in step 1315, the at least one processing device may cause adjustment of at least one navigation actuator of the host vehicle in response to the determined first navigation operation or the determined second navigation operation for the host vehicle to implement the determined navigation operation for the host vehicle. As with the previous example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0313] Relaxation of constraints

[0314] As discussed above, navigation constraints can be imposed for safety. The constraints can include, among others, a minimum safe driving distance to a pedestrian, a target vehicle, a road barrier, or a detected object, a maximum travel speed when passing within a detected pedestrian's influence zone, or a maximum deceleration rate of the host vehicle. These constraints can be imposed in a learning navigation system or a non-learning navigation system. In certain circumstances, these constraints can be relaxed. For example, if the host vehicle slows down or stops near a pedestrian and moves slowly to communicate an intent to pass the pedestrian, the pedestrian's response can be detected from the acquired images. If the pedestrian's response is to remain still or stop moving (and / or eye contact with the pedestrian is detected), it can be understood that the pedestrian recognizes the navigation system's intent to pass the pedestrian. In such circumstances, the system can relax one or more of the default constraints and enforce a less strict constraint (e.g., allowing the vehicle to navigate within 0.5 meters of the pedestrian rather than within the more strict 1 meter boundary).

[0315] FIG. 14 shows a flow chart for implementing control of a host vehicle based on relaxing one or more navigation constraints. In step 1401, at least one processing device may receive a plurality of images representing an environment of the host vehicle from a camera associated with the host vehicle. Analyzing the images in step 1403 may enable identifying a navigation state associated with the host vehicle. In step 1405, the at least one processor may determine navigation constraints associated with the navigation state of the host vehicle. The navigation constraints may include a first predefined navigation constraint that is implicated by at least one aspect of the navigation state. In step 1407, analyzing the plurality of images may determine the presence of at least one navigation constraint relaxation factor.

[0316] The navigation constraint mitigating factors may include any suitable indicator that one or more navigation constraints may be suspended, modified, or otherwise relaxed in at least one aspect. In some embodiments, the at least one navigation constraint mitigating factor may include a determination (based on image analysis) that the pedestrian's eyes are looking in the direction of the host vehicle. In that case, the pedestrian may be more safely assumed to recognize the host vehicle. As a result, there may be more confidence that the pedestrian will not engage in unexpected behavior that causes the pedestrian to move into the path of the host vehicle. Other constraint mitigating factors may also be used. For example, the at least one navigation constraint mitigating factor may include a pedestrian that is determined to be not moving (e.g., one that is estimated to be unlikely to enter the path of the host vehicle) or a pedestrian whose movement is determined to be slowing. The navigation constraint mitigating factors may also include more complex behaviors, such as a pedestrian that is determined to be not moving when the host vehicle stops and then resumes movement. In such a situation, the pedestrian may be assumed to understand that the host vehicle has the right-of-way, and the stopping pedestrian may indicate his or her intention to yield to the host vehicle. Other situations that may cause one or more constraints to be relaxed include the type of curb (e.g., a low curb or a gentle slope may allow the distance constraint to be relaxed), the absence of pedestrians or other objects on the sidewalk, vehicles that do not have their engines running may have a relaxed distance, or situations where pedestrians are facing away from and / or away from the area the host vehicle is heading into.

[0317] Upon identifying the presence of a navigation constraint relaxation factor (e.g., in step 1407), a second navigation constraint may be determined or developed in response to detecting the constraint relaxation factor. This second navigation constraint may differ from the first navigation constraint and may include at least one relaxed characteristic compared to the first navigation constraint. The second navigation constraint may include a newly generated constraint based on the first constraint, the newly generated constraint including at least one modification that relaxes the first constraint in at least one respect. Alternatively, the second constraint may constitute a predefined constraint that is less strict than the first navigation constraint in at least one respect. In some embodiments, this second constraint may be reserved for use only in situations where a constraint relaxation factor is identified in the host vehicle's environment. Whether the second constraint is newly generated or selected entirely or partially from a set of predefined constraints available, application of the second navigation constraint instead of the more strict first navigation constraint (which may be applied in the absence of detection of the associated navigation constraint relaxation factor) may be referred to as constraint relaxation and may be implemented in step 1409.

[0318] If at least one constraint relaxation factor is detected in step 1407 and at least one constraint is relaxed in step 1409, a navigation action for the host vehicle may be determined in step 1411. The navigation action for the host vehicle may be based on the identified navigation state and may satisfy the second navigation constraint. In step 1413, the navigation action may be implemented by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action.

[0319] As discussed above, the use of navigation constraints and relaxed navigation constraints can be used with a trained (e.g., by machine learning) navigation system or an untrained navigation system (e.g., a system programmed to respond with a predetermined action in response to a particular navigation situation). When using a trained navigation system, the availability of relaxed navigation constraints for a particular navigation situation may represent a mode switch from the response of the trained system to the response of the untrained system. For example, the trained navigation network may determine an original navigation action for the host vehicle based on a first navigation constraint. However, the action taken by the vehicle may be different from the navigation action that satisfies the first navigation constraint. Rather, the action taken may be an action developed by an untrained system that may satisfy a more relaxed second navigation constraint (e.g., in response to detecting a particular condition in the host vehicle's environment, such as the presence of a navigation constraint mitigating factor).

[0320] There are many examples of navigation constraints that may be relaxed in response to detecting constraint relaxation factors within the host vehicle's environment. For example, if a default navigation constraint includes a buffer zone associated with a detected pedestrian, and at least a portion of the buffer zone extends a predetermined distance from the detected pedestrian, a relaxed navigation constraint (either newly developed, recalled from memory from a predetermined set, or generated as a relaxed version of an existing constraint) may include a different or modified buffer zone. For example, the different or modified buffer zone may have a shorter distance to the pedestrian than the original or unmodified buffer zone to the detected pedestrian. As a result, if an appropriate constraint relaxation factor is detected within the host vehicle's environment, the host vehicle may be permitted to navigate closer to the detected pedestrian, taking into account the relaxed constraint.

[0321] The relaxed characteristics of the navigation constraints may include a reduced width of a buffer zone associated with at least one pedestrian as described above, but may also include a reduced width of a buffer zone associated with a target vehicle, a detected object, a roadside barrier, or any other object detected in the host vehicle's environment.

[0322] The at least one relaxed characteristic may also include other types of modifications in the navigation constraint characteristics. For example, the relaxed characteristic may include a speed increase associated with the at least one predefined navigation constraint. The relaxed characteristic may also include an increase in the maximum allowable deceleration / acceleration associated with the at least one predefined navigation constraint.

[0323] As noted above, constraints may be relaxed in certain circumstances, while navigation constraints may be augmented in other circumstances. For example, in some circumstances, the navigation system may determine that conditions justify augmenting the normal set of navigation constraints. Such augmentation may include adding new constraints to the set of predefined constraints or adjusting one or more aspects of the predefined constraints. This addition or adjustment may result in more conservative navigation relative to the set of predefined constraints that are applicable under normal driving conditions. Conditions that may justify augmenting constraints may include sensor failure, adverse environmental conditions (rain, snow, fog, or other conditions associated with reduced visibility or reduced vehicle traction), etc.

[0324] FIG. 15 shows a flowchart for implementing control of a host vehicle based on augmenting one or more navigation constraints. In step 1501, at least one processing device may receive a plurality of images representing an environment of the host vehicle from a camera associated with the host vehicle. Analyzing the images in step 1503 may enable identifying a navigation state associated with the host vehicle. In step 1505, the at least one processor may determine navigation constraints associated with the navigation state of the host vehicle. The navigation constraints may include a first predefined navigation constraint that is implicated by at least one aspect of the navigation state. In step 1507, analyzing the plurality of images may determine the presence of at least one navigation constraint augmentation factor.

[0325] The navigation constraints involved may include any of the navigation constraints discussed above (e.g., with respect to FIG. 12) or any other suitable navigation constraints. The navigation constraint augmentation factors may include any indicator that one or more navigation constraints may be supplemented / augmented in at least one aspect. The supplementation or augmentation of the navigation constraints may be done on a set-by-set basis (e.g., by adding a new navigation constraint to the set of predefined constraints) or on a constraint-by-constraint basis (e.g., modifying a particular constraint such that the modified constraint is more restrictive than the original, or adding a new constraint corresponding to the predefined constraint, where the new constraint is more restrictive in at least one aspect than the corresponding constraint). Additionally or alternatively, the supplementation or augmentation of the navigation constraints may refer to selecting from among the set of predefined constraints based on a hierarchy. For example, a set of augmented constraints may be provided for selection based on whether a navigation augmentation factor is detected in the environment of or for the host vehicle. Under normal conditions, where no augmentation factor is detected, the navigation constraints involved may be derived from the constraints applicable to the normal conditions. On the other hand, if one or more constraint augmentation factors are detected, the constraints involved can be derived from augmented constraints that are generated or predefined for the one or more augmentation factors, the augmented constraints being more restrictive in at least one aspect than the corresponding constraints applicable under normal conditions.

[0326] In some embodiments, the at least one navigation constraint augmentation factor may include detecting (e.g., based on image analysis) the presence of ice, snow, or water on a road surface in the host vehicle's environment. This determination may be based, for example, on detecting areas that are more reflective than expected on a dry road (e.g., indicative of ice or water on the road), white areas on the road surface indicating the presence of snow, shadows on the road consistent with the presence of longitudinal grooves on the road (e.g., tire tracks in snow), water droplets or small flakes of ice / snow on the host vehicle's windshield, or any other suitable indicator of the presence of water or ice / snow on the road surface.

[0327] At least one navigation constraint augmentation factor may also include detecting small grains on an exterior surface of the host vehicle's windshield. Such small grains may impair image quality of one or more image capture devices associated with the host vehicle. Although described with respect to the host vehicle's windshield in association with a camera mounted behind the host vehicle's windshield, detecting small grains on other surfaces associated with the host vehicle (e.g., a camera lens or lens cover, a headlight lens, a rear window, a taillight lens, or any other surface of the host vehicle visible to (or detected by a sensor of) an image capture device may also indicate the presence of a navigation constraint augmentation factor.

[0328] Navigation constraint augmentation factors may also be detected as characteristics of one or more image capture devices. For example, a detected degradation in image quality of one or more images captured by an image capture device (e.g., a camera) associated with the host vehicle may also constitute a navigation constraint augmentation factor. The degradation in image quality may be associated with a hardware failure or partial hardware failure associated with the image capture device or an assembly associated with the image capture device. Such degradation in image quality may also be caused by environmental conditions. For example, the presence of smoke, fog, rain, snow, etc. in the air surrounding the host vehicle may contribute to degradation in image quality with respect to roads, pedestrians, target vehicles, etc. that may be in the host vehicle's environment.

[0329] Navigation constraint augmentation factors may also relate to other aspects of the host vehicle. For example, in some circumstances, navigation constraint augmentation factors may include a detected failure or partial failure of a system or sensor associated with the host vehicle. Such augmentation factors may include, for example, detecting a failure or partial failure of a speed sensor, GPS receiver, accelerometer, camera, radar, lidar, brakes, tires, or any other system associated with the host vehicle that may affect the host vehicle's ability to navigate in conjunction with navigation constraints associated with the host vehicle's navigational state.

[0330] If the presence of a navigation constraint augmentation factor is identified (e.g., in step 1507), a second navigation constraint may be determined or developed in response to detecting the constraint augmentation factor. This second navigation constraint may differ from the first navigation constraint and may include at least one characteristic that is augmented with respect to the first navigation constraint. The second navigation constraint may be more restrictive than the first navigation constraint because detecting a constraint augmentation factor in the host vehicle's environment or associated with the host vehicle may indicate that at least one navigation capability of the host vehicle may be degraded compared to normal operating conditions. Such degraded capability may include degraded road traction (e.g., ice, snow or water on the road, degraded tire pressure, etc.), impaired visibility (e.g., rain, snow, dust, smoke, fog, etc. that reduces captured image quality), impaired detection capability (e.g., sensor failure or partial failure, degraded sensor performance, etc.), or any other degraded capability of the host vehicle to navigate in response to the detected navigation conditions.

[0331] If at least one constraint augmentation factor is detected in step 1507 and at least one constraint is augmented in step 1509, a navigation action for the host vehicle may be determined in step 1511. The navigation action for the host vehicle may be based on the identified navigation state and may satisfy the second navigation (i.e., augmented) constraint. In step 1513, the navigation action may be implemented by causing adjustment of at least one navigation actuator of the host vehicle in response to the determined navigation action.

[0332] As discussed above, the use of navigation constraints and augmented navigation constraints can be used with a trained (e.g., by machine learning) navigation system or an untrained navigation system (e.g., a system programmed to respond with a predetermined action in response to a particular navigation situation). When using a trained navigation system, the availability of an augmented navigation constraint for a particular navigation situation can represent a mode switch from the response of the trained system to the response of the untrained system. For example, the trained navigation network can determine an original navigation action for the host vehicle based on a first navigation constraint. However, the action taken by the vehicle can be different from the navigation action that satisfies the first navigation constraint. Rather, the action taken can be an action developed by an untrained system that can satisfy an augmented second navigation constraint (e.g., in response to detecting a particular condition in the host vehicle's environment, such as the presence of a navigation constraint augmentation factor).

[0333] There are many examples of navigation constraints that may be generated, supplemented, or augmented in response to detecting constraint augmentation factors within the host vehicle's environment. For example, if a default navigation constraint includes a buffer zone associated with a detected pedestrian, object, vehicle, etc., and at least a portion of the buffer zone extends a predetermined distance from the detected pedestrian / object / vehicle, an augmented navigation constraint (either newly developed, recalled from memory from a predetermined set, or generated as an augmented version of an existing constraint) may include a different or modified buffer zone. For example, the different or modified buffer zone may have a longer distance to the pedestrian / object / vehicle than the original or unmodified buffer zone to the detected pedestrian / object / vehicle. As a result, if an appropriate constraint augmentation factor is detected within or in relation to the host vehicle's environment, the host vehicle may be forced to take the augmented constraint into account and navigate further away from the detected pedestrian / object / vehicle.

[0334] The at least one augmented characteristic may also include other types of modifications in the navigation constraint characteristics. For example, the augmented characteristic may include a speed reduction associated with the at least one predefined navigation constraint. The augmented characteristic may also include a reduction in the maximum allowable deceleration / acceleration associated with the at least one predefined navigation constraint.

[0335] Navigation based on long-term planning

[0336] In some embodiments, the disclosed navigation system may not only respond to detected navigational states in the host vehicle's environment, but may also determine one or more navigational actions based on long-term planning. For example, the system may consider the potential impact on future navigational states of one or more navigational actions available as options for navigating with respect to the detected navigational state. Considering the effect of available actions on future states may allow the navigation system to determine a navigational action based not only on the navigational state currently detected, but also on long-term planning. Navigation using long-term planning techniques may be particularly applicable when one or more reward functions are used by the navigation system as a technique for selecting a navigational action from among available options. Potential rewards may be analyzed with respect to available navigational actions that can be performed in response to the detected current navigational state of the host vehicle. However, potential rewards may also be analyzed in relation to actions that can be performed in response to future navigational states predicted to result from the available actions for the current navigational state. As a result, in some cases, the disclosed navigation system may select a navigational action in response to the detected navigational state, even if the selected navigational action may not result in the highest reward among the available actions that can be performed in response to the current navigational state. This may be especially true when the system determines that the selected action may result in a future navigation state that triggers one or more potential navigation actions that provide higher rewards than either the selected action or, in some cases, any of the actions available for the current navigation state. This principle may be expressed more simply as taking a less advantageous action now in order to provide a higher reward option in the future. Thus, the disclosed navigation system, capable of long-term planning, may select a suboptimal short-term action when long-term predictions indicate that a short-term loss of reward may result in an increase in long-term reward.

[0337] In general, autonomous driving applications may include a set of planning problems where a navigation system may determine an immediate action to optimize a long-term objective. For example, when a vehicle faces a merging situation at a roundabout, the navigation system may determine an immediate acceleration or braking command to begin navigation into the roundabout. The immediate action for the navigation state detected at the roundabout may include an acceleration or braking command depending on the detected state, while the long-term objective is to successfully merge, and the long-term effect of the selected command is the success / failure of the merge. The planning problem can be addressed by decomposing the problem into two phases. First, supervised learning can be applied to predict the near future based on the present (assuming that the predictors are differentiable with respect to the current representation). Second, a recurrent neural network can be used to model the complete trajectory of the agent, with unexplained factors modeled as (additive) input nodes. This may allow the solution to the long-term planning problem to be found using supervised learning methods and direct optimization on the recurrent neural network. Such an approach may also allow the learning of robust policies by incorporating adversarial elements to the environment.

[0338] Two of the most fundamental elements of an autonomous driving system are sensing and planning. Sensing deals with finding a compact representation of the current state of the environment, while planning deals with deciding which action to take to optimize a future objective. Supervised machine learning methods are useful for solving the sensing problem. One can also use machine learning algorithm frameworks for the planning part, especially the Reinforcement Learning (RL) framework as described above.

[0339] RL can be executed by a series of successive rounds. In round t, the planner (aka agent or driving policy module 803) creates a state s representing the agents and the environment. tThen, the planner selects action a t After performing the action, the agent receives an immediate reward.

number

number

number

[0340] Supervised learning (SL) can be considered a special case of RL, where we choose s from some distribution over S. t is sampled, and the reward function is r t =-l(a t ,y t ), where l is the loss function, and the learning side can t is the (possibly noisy) value of the optimal action to take when admitting tObserve the values ​​of . There may be some differences between the model of general RL and the specific case of SL, and these differences may make the general RL problem more difficult.

[0341] In some SL situations, the actions (or predictions) made by the learner may not have any effect on the environment. t+1 and a t and are independent. This can have two important implications. First, in SL, the sample (s 1 ,y 1 ),...,(s m ,y m ) in advance, and only then can we begin to search for a policy (or predictor) that has good accuracy for the samples. In contrast, in RL, t+1 Typically, depends on the action taken (and on previous states), which in turn depends on the policy used to generate the action. This ties the data generation process to the policy learning process. Second, in SL, the effect of a on the performance of π is small, since actions do not affect the environment. t The contribution of the selection of is local. t only affects the value of the immediate reward. In contrast, in RL, actions taken in round t can have long-term effects on the reward value in future rounds.

[0342] In SL, the reward shape r t =-l(a t ,y t ) along with knowledge of the "correct" solution yt, t It can provide complete knowledge of the payoffs for all possible choices of tThis may allow for the computation of the derivative of the reward with respect to . In contrast, in RL, the "one-shot" value of the reward may be all that can be observed for a particular choice of action taken. This can be called "bandit" feedback. In an RL-based system, if only "bandit" feedback is available, the system may not always be able to know if the action taken was the best action to take, which is one of the most important reasons why "exploration" is necessary as part of long-term navigation planning.

[0343] Many RL algorithms rely, at least in part, on a mathematically explicit model of Markov decision processes (MDPs). The Markov assumption is that s t and a t Given s t+1 The distribution of π is fully determined. This yields a closed form for the cumulative reward of a given policy in terms of its stationary distribution over the states of the MDP. The stationary distribution of a policy can be expressed as a solution to a linear programming problem. This yields two families of algorithms: 1) optimization with respect to the primal problem, which can be called policy search, and 2) optimization with respect to the dual problem, whose variables are called the value function Vπ. The value function determines the expected cumulative reward if the MDP starts at an initial state s, from which actions are selected according to π. The quantity involved is the state-action value function Q π (s,a), which determines the cumulative reward given starting in state s, an immediately chosen action a, and an action chosen from there according to π. This Q-function can give rise to a characterization of the optimal policy (using the Bellman equation). Specifically, this Q-function can be shown to be a deterministic function from S to A (indeed, the optimal policy can be characterized as a "greedy" policy with respect to the optimal Q-function).

[0344] One potential advantage of MDP models is that they allow the future to be coupled to the present using a Q-function. For example, given that the host vehicle is currently in state s, Q πThe value of (s,a) may indicate the effect of performing action a on the future. Thus, the Q-function provides a local measure of the quality of action a, making RL problems more similar to SL scenarios.

[0345] Many RL algorithms approximate the V-function or Q-function in some way. Value iteration algorithms, e.g., Q-learning algorithms, can exploit the fact that the V-function and Q-function of an optimal policy can be fixed points of some operators obtained from the Bellman equations. Actor-critic policy iteration algorithms aim to learn a policy in an iterative way, where at iteration t, the "critic" is

number

[0346] Despite the mathematical simplicity of MDPs and the convenience of switching to a Q-function representation, this approach may have some limitations. For example, in some cases, an approximate notion of a Markovian behavior state may be all that can be found. Furthermore, state transitions may not only depend on the behavior of the agent, but also on the behavior of other players in the environment. For example, in the ACC example above, the dynamics of the autonomous vehicle may be Markovian, but the next state may depend on the behavior of drivers of other cars, which is not necessarily Markovian. One possible solution to this problem is to use partially observable MDPs, in which it is assumed that there are Markovian states, but what can be seen are observations that are distributed according to hidden states.

[0347] A more straightforward approach could consider a game-theoretic generalization of MDPs (e.g., in the framework of stochastic games). Indeed, algorithms for MDPs can be generalized to multi-agent games (e.g., minimax Q-learning or Nash Q-learning). Other approaches could include explicit modeling of other players and vanishing regret learning algorithms. Learning in a multi-agent setting can be more complex than learning in a single-agent setting.

[0348] A second limitation of the Q-function representation may arise from departing from the tabular setting. A tabular setting is when the number of states and actions is small, so that Q can be represented as a table with |S| rows and |A| columns. However, if the natural representation of S and A involves Euclidean space and the state and action spaces are discretized, the number of states / actions may be exponential in scale. In such cases, it may be impractical to adopt the tabular setting. Instead, the Q-function may be approximated by some function from a parametric hypothesis class (e.g., neural networks of a certain architecture). For example, the Deep Q-Network (DQN) learning algorithm may be used. In DQN, the state space may be continuous, but the action space may remain a small discrete set. Techniques for accommodating a continuous action space are possible, but they may rely on approximating the Q-function. In any case, the Q-function may be complex and sensitive to noise, and thus difficult to learn.

[0349] A different approach could be to use recurrent neural networks (RNNs) to address RL problems. In some cases, RNNs can be combined with concepts of multi-agent games and robustness to adversarial environments from game theory. Furthermore, this approach could be one that does not explicitly rely on the Markov assumption.

[0350] In the following, we describe in more detail an approach for navigation by prediction-based planning, in which the state space S is

number

number

[0351] As a first step in this approach, it can be observed that there are interesting problems where the bandit nature of the reward does not matter. For example, reward values ​​for ACC applications (discussed in more detail below) can be differentiable with respect to the current state and action. In fact, even if the reward is given in a "bandit" fashion,

number

number

number

number

number

[0352] Similar concepts can be used to address connections between the past and the future. For example,

number

number

number

number

number

number

number

[0353] As discussed above, the learning system can benefit from robustness to hostile environments, such as the host vehicle's environment, which may include multiple other drivers who may behave in unexpected ways. t In a model that does not impose probabilistic assumptions on v t We can consider an environment where μ is chosen in an adversarial way. t We can place restrictions on ||μ, which would otherwise make the planning problem difficult or even impossible for an adversary. One natural constraint is that ||μ t It may be necessary to require that || be bounded by a constraint.

[0354] Robustness to hostile environments can be useful in autonomous driving applications. tChoosing , can even speed up the learning process, since it can focus the learning system towards a robust optimal policy. A simple game can be used to illustrate this concept. The states are

number

number

[0355] Such an approach can be applied to virtually all possible navigation situations. In the following, we describe the approach applied to one example: Adaptive Cruise Control (ACC). In an ACC problem, a host vehicle may attempt to keep a sufficient distance to a target vehicle ahead (e.g., 1.5 seconds to the target vehicle). Another goal may be to drive as smoothly as possible while maintaining a desired gap. A model to represent this situation can be defined as follows. The state space is:

number

number

number

number

[0356] The complete dynamics of the system can be described by the following equation:

number

[0357] This can be written as the sum of two vectors.

number

[0358] The first vector is the predictable part and the second vector is the unpredictable part. The reward for round t is defined as:

number

[0359] The first term may introduce a penalty for non-zero acceleration, thus encouraging smooth driving. The second term may be used to calculate the acceleration of the target vehicle x t Distance to and desired distance

number

[0360] Implementing the techniques outlined above, the host vehicle's navigation system may select an action in response to observed conditions (e.g., by operation of the driving policy module 803 in the navigation system's processing unit 110). The action selected may be based not only on an analysis of rewards associated with available responsive actions with respect to the sensed navigation condition, but also on consideration and analysis of future conditions, potential actions in response to the future conditions, and rewards associated with the potential actions.

[0361] 16 illustrates an algorithmic approach to navigation based on detection and long-term planning. For example, in step 1601, at least one processing device 110 of a navigation system for a host vehicle may receive a plurality of images. These images may capture scenes representative of the host vehicle's environment and may be provided by any of the image capture devices (e.g., cameras, sensors, etc.) described above. Analyzing one or more of these images in step 1603 may enable the at least one processing device 110 to identify a current navigation state associated with the host vehicle (as described above).

[0362] In steps 1605, 1607, and 1609, various potential navigation actions may be determined depending on the sensed navigation conditions (e.g., to complete a merge, to smoothly follow a leading vehicle, to overtake a target vehicle, to avoid objects in the road, to slow down for a detected stop sign, to avoid an incoming target vehicle, or to complete any other navigation action that may further the navigation goals of the system). These potential navigation actions (e.g., from the first navigation action to the Nth available navigation action) may be determined based on the sensed conditions and the long-term goals of the navigation system.

[0363] For each of the determined potential navigation actions, the system may determine an expected reward. The expected reward may be determined according to any of the techniques described above and may include analysis of the particular potential action against one or more reward functions. Expected rewards 1606, 1608, and 1610 may be determined for each of the (e.g., first, second, and Nth) potential navigation actions determined in steps 1605, 1607, and 1609, respectively.

[0364] In some cases, the host vehicle's navigation system may make a selection among available potential actions based on values ​​associated with expected rewards 1606, 1608, and 1610 (or any other type of indicator of expected rewards). For example, in some situations, the action that results in the highest expected reward may be selected.

[0365] In other cases, particularly where the navigation system engages in long-term planning to determine a navigation action for the host vehicle, the system may not select the potential action that will result in the highest expected reward. Rather, the system may look to the future and analyze whether there may be an opportunity to realize a higher reward later if a low-reward action is selected depending on the current navigation state. For example, a future state may be determined for any or all of the potential actions determined in steps 1605, 1607, and 1609. Each future state determined in steps 1613, 1615, and 1617 may represent a future navigation state that is expected to occur based on the current navigation state as modified by the respective potential action (e.g., the potential actions determined in steps 1605, 1607, and 1609).

[0366] For each future state predicted in steps 1613, 1615, and 1617, one or more future actions (as navigation options available depending on the determined future state) may be determined and evaluated. In steps 1619, 1621, and 1623, for example, a value or any other type of indicator of an expected reward associated with one or more of the future actions may be developed (e.g., based on one or more reward functions). The expected reward associated with the one or more future actions may be evaluated by comparing values ​​of the reward functions associated with the respective future actions or by comparing any other indicator associated with the expected reward.

[0367] In step 1625, the navigation system for the host vehicle may select a navigation action for the host vehicle based not only on potential actions identified for the current navigation state (e.g., in steps 1605, 1607, and 1609), but also on expected rewards determined as a result of available future potential actions depending on predicted future states (e.g., determined in steps 1613, 1615, and 1617) and based on comparing expected rewards. The selection in step 1625 may be based on the analysis of options and rewards performed in steps 1619, 1621, and 1623.

[0368] The selection of a navigation action in step 1625 may be based solely on comparing expected rewards associated with alternative future actions. In this case, the navigation system may select an action for the current state based solely on comparing expected rewards resulting from actions for potential future navigation states. For example, the system may select the potential action identified in steps 1605, 1607, or 1609 that is associated with the highest future reward value as determined by the analysis in steps 1619, 1621, and 1623.

[0369] The selection of a navigation action in step 1625 may be based solely on comparing current action options (as discussed above). In this situation, the navigation system may select the potential action identified in steps 1605, 1607, or 1609 that is associated with the highest expected reward 1606, 1608, or 1610. This selection may be made with little or no consideration of future expected rewards for available navigation actions depending on the future navigation state or anticipated future navigation states.

[0370] On the other hand, in some cases, the selection of the navigation action in step 1625 may be based on comparing expected rewards associated with both future action options and current action options. This may in fact be one of the principles of navigation based on long-term planning. In order to realize potentially higher rewards for subsequent navigation actions expected to become available depending on future navigation states, the expected rewards for future actions may be analyzed to determine whether any may justify selecting an action with a lower reward depending on the current navigation state. As an example, the value or other indicator of expected reward 1606 may indicate the highest expected reward among rewards 1606, 1608, and 1610. On the other hand, expected reward 1608 may indicate the lowest expected reward among rewards 1606, 1608, and 1610. Rather than simply selecting the potential action determined in step 1605 (i.e., the action that results in the highest expected reward 1606), an analysis of the future states, potential future actions, and future rewards may be used in making the selection of the navigation action in step 1625. In one example, it may be determined that a reward identified in step 1621 (in response to at least one future action for the future state determined in step 1615 based on the second potential action determined in step 1607) is likely to be higher than the expected reward 1606. Based on this comparison, the second potential action determined in step 1607 may be selected over the first potential action determined in step 1605 even though the expected reward 1606 is higher than the expected reward 1608. In one example, the potential navigation action determined in step 1605 may include merging in front of the detected target vehicle, while the potential navigation action determined in step 1607 may include merging behind the target vehicle.Although the expected reward 1606 for merging in front of the target vehicle may be higher than the expected reward 1608 associated with merging behind the target vehicle, it may be determined that merging behind the target vehicle may result in a future state in which there may be an option for an action that provides a higher potential reward than the expected rewards 1606, 1608 or other rewards based on actions available depending on the current sensed navigation state.

[0371] The selection among the potential actions in step 1625 may be based on any suitable comparison of expected rewards (or any other metric or indicator of benefit associated with one potential action over another). In some cases, as described above, a second potential action may be selected in preference to a first potential action if the second potential action is predicted to result in at least one future action associated with a higher expected reward than the reward associated with the first potential action. In other cases, more complex comparisons may be used. For example, rewards associated with the action choices depending on the predicted future state may be compared to multiple expected rewards associated with the determined potential actions.

[0372] In some scenarios, the action and expected reward based on the predicted future state may influence the selection of a potential action for the current state if at least one of the future actions is expected to result in a reward higher than any of the expected rewards (e.g., expected rewards 1606, 1608, 1610, etc.) as a result of the potential action for the current state. In some cases, the future action option that results in the highest expected reward (e.g., among the expected rewards associated with the potential action for the sensed current state and the expected rewards associated with the potential future action options for the potential future navigation states) may be used as a guide for selecting a potential action for the current navigation state. That is, after identifying the future action options that result in the highest expected reward (or a reward above a predetermined threshold, etc.), the potential action that leads to the future state associated with the identified future action that results in the highest expected reward may be selected in step 1625.

[0373] In other cases, the selection of available actions may be based on the determined difference between the expected rewards. For example, the second potential action determined in step 1607 may be selected if the difference between the expected reward associated with the future action determined in step 1621 and the expected reward 1606 exceeds the difference between the expected reward 1608 and the expected reward 1606 (assuming a + sign difference). In another example, the second potential action determined in step 1607 may be selected if the difference between the expected reward associated with the future action determined in step 1621 and the expected reward associated with the future action determined in step 1619 exceeds the difference between the expected reward 1608 and the expected reward 1606.

[0374] Several examples have been described for selecting among potential actions for a current navigation state. However, any other suitable comparison technique or criteria can be used to select an available action by long-term planning based on an analysis of actions and rewards spanning predicted future states. In addition, while FIG. 16 shows two layers in the long-term planning analysis (e.g., a first layer considering rewards resulting from potential actions for the current state and a second layer considering rewards resulting from future action options depending on predicted future states), analysis based on more layers may be possible. For example, rather than basing the long-term planning analysis on one or two layers, three, four or even more layers of analysis can be used when selecting among potential actions available depending on the current navigation state.

[0375] After selecting among the potential actions responsive to the sensed navigation state, in step 1627, the at least one processor may cause adjustment of at least one navigation actuator of the host vehicle responsive to the selected potential navigation action. The navigation actuator may include any suitable device for controlling at least one aspect of the host vehicle. For example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0376] Navigation based on others' inferred aggression

[0377] The target vehicle can be monitored by analyzing the acquired image stream to determine indicators of driving aggression. In this specification, aggression is described as a qualitative or quantitative parameter, but other characteristics, i.e., detected attention level (potential driver deficiencies, distraction - cell phone, drowsiness, etc.) can be used. In some cases, the target vehicle can be considered to have a defensive attitude, and in some cases, the target vehicle can be considered to have a more aggressive attitude. Navigation actions can be selected or developed based on the indicators of aggression. For example, in some cases, the relative speed, relative acceleration, increase in relative acceleration, pursuit distance, etc. relative to the host vehicle can be tracked to determine whether the target vehicle is aggressive or defensive. If the target vehicle is determined to have a level of aggression above a threshold, for example, the host vehicle can be inclined to give way to the target vehicle. The level of aggression of the target vehicle can also be determined based on the determined behavior of the target vehicle with respect to one or more obstacles in the path or near the target vehicle (e.g., a preceding vehicle, an obstacle in the road, a traffic light, etc.).

[0378] As an introduction to this concept, an example experiment involving a host vehicle merging into a roundabout is described, where the navigation goal is to exit through the roundabout. The situation can begin with the host vehicle reaching the roundabout entrance and can end with the host vehicle reaching the roundabout exit (e.g., the second exit). Success can be measured based on whether the host vehicle always maintains a safe distance from all other vehicles, whether the host vehicle completes the route as quickly as possible, and whether the host vehicle follows a policy of smooth acceleration. In this discussion, N TTarget vehicles may be randomly placed on the roundabout. To model the mixture of hostile and typical behaviors, the target vehicles may be modeled with an "aggressive" driving policy with probability p, so that when the host vehicle attempts to merge in front of the target vehicle, the aggressive target vehicle accelerates. The target vehicles may be modeled with a "defensive" driving policy with probability 1-p, so that the target vehicle slows down and lets the host vehicle merge. In this experiment, p=0.5, and no information about the type of other drivers may be given to the host vehicle's navigation system. The type of other drivers may be randomly selected at the start of the episode.

[0379] The navigation state can be represented as the host vehicle (agent) speed and position, and the target vehicle's position, speed, and acceleration. To distinguish between aggressive and defensive drivers based on the current state, it may be important to keep track of the target's acceleration. All target vehicles may travel on a one-dimensional curve that outlines the roundabout's path. The host vehicle may travel on its own one-dimensional curve that intersects with the target vehicle's curve at the junction, which is the origin of both curves. To model reasonable driving, the absolute value of all vehicle accelerations can be capped by a constant. Since driving in the opposite direction is not allowed, the speed can also be passed through a ReLU. Note that by not allowing driving in the opposite direction, the agent cannot regret its past actions, which may require long-term planning.

[0380] As explained above, the next state s t+1 is the predictable part

number

number

number

[0381] FIG. 18 shows a flow chart illustrating an example of an algorithm for navigating a host vehicle based on the predicted aggressiveness of other vehicles. In the example of FIG. 18, a level of aggressiveness associated with at least one target vehicle can be inferred based on the observed behavior of the target vehicle with respect to objects in the target vehicle's environment. For example, in step 1801, at least one processing device (e.g., processing device 110) of the host vehicle's navigation system can receive a plurality of images representing the host vehicle's environment from a camera associated with the host vehicle. In step 1803, analyzing one or more of the received images can enable at least one processor to identify a target vehicle (e.g., vehicle 1703) in the host vehicle's environment 1701. In step 1805, analyzing one or more of the received images can enable at least one processing device to identify at least one obstacle for the target vehicle in the host vehicle's environment. The objects may include debris in the road, a stoplight / traffic light, a pedestrian, another vehicle (e.g., a vehicle moving in front of the target vehicle, a parked vehicle, etc.), a box in the road, a road barrier, a curve, or any other type of object that may be encountered in the host vehicle's environment. In step 1807, analyzing one or more of the received images may enable the at least one processing device to determine at least one navigation characteristic of the target vehicle relative to at least one identified obstacle for the target vehicle.

[0382] In order to develop an appropriate navigation response to the target vehicle, various navigation characteristics can be used to infer the level of aggressiveness of the detected target vehicle. For example, such navigation characteristics may include the relative acceleration between the target vehicle and at least one identified obstacle, the distance of the target vehicle from the obstacle (e.g., the trailing distance of a target vehicle behind another vehicle), and / or the relative speed between the target vehicle and the obstacle, etc.

[0383] In some embodiments, the navigation characteristics of the target vehicle may be determined based on output from sensors (e.g., radar, speed sensor, GPS, etc.) associated with the host vehicle. However, in some cases, the navigation characteristics of the target vehicle may be determined based in part or in full on analyzing images of the host vehicle's environment. For example, the image analysis techniques described, by way of example, above and in U.S. Pat. No. 9,168,868, incorporated herein by reference, may be used to recognize the target vehicle in the host vehicle's environment. Monitoring the position of the target vehicle in captured images over a period of time and / or monitoring the position in captured images of one or more features associated with the target vehicle (e.g., tail lights, headlights, bumpers, wheels, etc.) may enable the relative distance, speed, and / or acceleration between the target vehicle and the host vehicle or between the target vehicle and one or more other objects in the host vehicle's environment to be determined.

[0384] The level of aggressiveness of an identified target vehicle may be inferred from any suitable observed navigation characteristic or any combination of observed navigation characteristics of the target vehicle. For example, the determination of aggressiveness may be based on any observed characteristic and one or more predefined threshold levels or any other suitable qualitative or quantitative analysis. In some embodiments, a target vehicle may be considered aggressive if it is observed to follow a host vehicle or another vehicle at a distance less than a predefined aggressive distance threshold. On the other hand, a target vehicle may be considered defensive if it is observed to follow a host vehicle or another vehicle at a distance greater than a predefined aggressive distance threshold. The predefined aggressive distance threshold need not be the same as the predefined defensive distance threshold. In addition, either or both of the predefined aggressive distance threshold and the predefined defensive distance threshold may include ranges rather than clear boundaries. Furthermore, neither the predefined aggressive distance threshold nor the predefined defensive distance threshold need be fixed. Rather, these values ​​or ranges may shift over time, and different thresholds / threshold ranges may be applied based on the observed characteristics of the target vehicle. For example, the threshold applied may depend on one or more other characteristics of the target vehicle. A higher observed relative speed and / or acceleration may justify the application of a larger threshold / threshold range. Conversely, a lower relative speed and / or acceleration, including zero relative speed and / or acceleration, may justify the application of a smaller distance threshold / threshold range when making an offensive / defensive inference.

[0385] Aggressive / defensive inferences may also be based on relative speed and / or relative acceleration thresholds. A target vehicle may be considered aggressive if its observed relative speed and / or its relative acceleration with respect to another vehicle is above a predetermined level or range. A target vehicle may be considered defensive if its observed relative speed and / or its relative acceleration with respect to another vehicle is below a predetermined level or range.

[0386] The aggressive / defensive determination may be based solely on any observed navigation characteristic, but the determination may also depend on any combination of observed characteristics. For example, as noted above, in some cases, a target vehicle may be deemed aggressive based solely on being observed to follow another vehicle at a distance less than a certain threshold or range. However, in other cases, a target vehicle may be deemed aggressive if it follows another vehicle less than a predetermined amount (which may be the same or different than the threshold applied if the determination is based solely on distance) and has a relative speed and / or relative acceleration above a predetermined amount or range. Similarly, a target vehicle may be deemed defensive based solely on being observed to follow another vehicle at a distance greater than a certain threshold or range. However, in other cases, a target vehicle may be deemed defensive if it follows another vehicle more than a predetermined amount (which may be the same or different than the threshold applied if the determination is based solely on distance) and has a relative speed and / or relative acceleration below a predetermined amount or range. The system 100 may be aggressive / defensive, for example, if the vehicle exceeds 0.5G acceleration or deceleration (e.g., a jerk of 5 m / s3), if the vehicle has a lateral acceleration of 0.5G when changing lanes or on a curve, if the vehicle forces another vehicle to do any of the above, if the vehicle changes lanes and forces another vehicle to give way exceeding a deceleration of 0.3G or a jerk of 3 m / s3, and / or if the vehicle makes a two-lane change without stopping.

[0387] It should be understood that a reference to a quantity above a range may indicate that the quantity is above or within all values ​​associated with the range. Similarly, a reference to a quantity below a range may indicate that the quantity is below or within all values ​​associated with the range. In addition, while the described examples of making aggressive / defensive inferences have been described with respect to distance, relative acceleration, and relative velocity, any other suitable quantities may be used. For example, any indirect indicator of time to collision or distance, acceleration, and / or velocity of the target vehicle may be used for calculation. It should also be noted that while the above examples focus on the target vehicle relative to other vehicles, aggressive / defensive inferences may be made by observing the navigation characteristics of the target vehicle relative to any other type of obstacle (e.g., pedestrians, road barriers, traffic lights, debris, etc.).

[0388] Returning to the example shown in FIG. 17A and FIG. 17B, when the host vehicle 1701 approaches the roundabout, the navigation system, including its at least one processing device, may receive an image stream from a camera associated with the host vehicle. Based on one or more analyses of the received images, any of the target vehicles 1703, 1705, 1706, 1708, and 1710 may be identified. Furthermore, the navigation system may analyze one or more navigation characteristics of the identified target vehicles. The navigation system may recognize that a gap between the target vehicles 1703 and 1705 represents a first opportunity for a potential merge into the roundabout. The navigation system may analyze the target vehicle 1703 to determine an indicator of aggressiveness associated with the target vehicle 1703. If the target vehicle 1703 is deemed aggressive, the host vehicle's navigation system may decide to yield to the vehicle 1703 rather than merge in front of the vehicle 1703. On the other hand, if the target vehicle 1703 is deemed defensive, the host vehicle's navigation system may attempt to complete the merging maneuver in front of the vehicle 1703 .

[0389] When the host vehicle 1701 reaches the roundabout, at least one processing device of the navigation system can analyze the captured image to determine navigation characteristics associated with the target vehicle 1703. For example, based on the image, it can be determined that the vehicle 1703 is following the vehicle 1705 at a distance that provides sufficient clearance for the host vehicle 1701 to safely enter. Indeed, it may be determined that the vehicle 1703 is following the vehicle 1705 at a distance that exceeds an aggressive distance threshold, and therefore, based on this information, the host vehicle's navigation system may be inclined to identify the target vehicle 1703 as defensive. However, in some circumstances, multiple navigation characteristics of the target vehicle can be analyzed in making an aggressive / defensive decision as discussed above. Expanding the analysis, the host vehicle's navigation system can determine that the target vehicle 1703 is following behind the target vehicle 1705 at a non-aggressive distance, but that the vehicle 1703 has a relative speed and / or relative acceleration with respect to the vehicle 1705 that exceeds one or more thresholds associated with aggressive behavior. Indeed, the host vehicle 1701 may determine that the target vehicle 1703 is accelerating relative to the vehicle 1705, closing the gap between the vehicles 1703 and 1705. Based on further analysis of the relative speed, acceleration, and distance (as well as the rate at which the gap between the vehicles 1703 and 1705 is closing), the host vehicle 1701 may determine that the target vehicle 1703 is behaving aggressively. Thus, while there may be a sufficient gap for the host vehicle to navigate safely, the host vehicle 1701 may expect that merging in front of the target vehicle 1703 will result in a vehicle navigating aggressively directly behind the host vehicle. Furthermore, if the host vehicle 1701 merges in front of the vehicle 1703, based on behavior observed by image analysis or other sensor output, the target vehicle 1703 may be expected to continue accelerating toward the host vehicle 1701, or proceed toward the host vehicle 1701 at a non-zero relative speed. Such a situation may be undesirable from a safety standpoint and may also cause discomfort to the occupants of the host vehicle.For this reason, as shown in FIG. 17B, host vehicle 1701 may decide to yield to vehicle 1703 and merge into the roundabout behind vehicle 1703 and ahead of vehicle 1710, which is deemed defensive based on one or more analyses of its navigation characteristics.

[0390] Returning to FIG. 18, in step 1809, at least one processing device of the host vehicle's navigation system may determine a navigation action for the host vehicle (e.g., merge in front of vehicle 1710 and behind vehicle 1703) based on the at least one identified navigation characteristic of the target vehicle relative to the identified obstacle. To implement the navigation action (step 1811), the at least one processing device may cause adjustment of at least one navigation actuator of the host vehicle in response to the determined navigation action. For example, the brakes may be applied to give way to vehicle 1703 in FIG. 17A, and the accelerator may be applied along with steering the host vehicle's wheels to cause the host vehicle to enter the roundabout behind vehicle 1703 as shown in FIG. 17B.

[0391] As described in the examples above, the host vehicle's navigation can be based on the target vehicle's navigation characteristics relative to another vehicle or object. Additionally, the host vehicle's navigation can be based solely on the target vehicle's navigation characteristics without specific reference to another vehicle or object. For example, in step 1807 of FIG. 18, analyzing a plurality of images captured from the host vehicle's environment can enable determining at least one navigation characteristic of the identified target vehicle that is indicative of a level of aggressiveness associated with the target vehicle. The navigation characteristics can include speed, acceleration, etc. that do not need to be referenced to another object or target vehicle to make an aggressive / defensive determination. For example, observed acceleration and / or speed associated with the target vehicle that exceeds a predetermined threshold or falls within or exceeds a range of values ​​can indicate aggressive behavior. Conversely, observed acceleration and / or speed associated with the target vehicle that falls below a predetermined threshold or falls within or exceeds a range of values ​​can indicate defensive behavior.

[0392] Of course, in some examples, observed navigation characteristics (e.g., position, distance, acceleration, etc.) may be referenced to the host vehicle to make an aggressive / defensive determination. For example, observed navigation characteristics of a target vehicle indicative of a level of aggressiveness associated with the target vehicle may include an increase in relative acceleration between the target vehicle and the host vehicle, the trailing distance of the target vehicle behind the host vehicle, the relative speed between the target vehicle and the host vehicle, etc.

[0393] Navigation based on liability constraints

[0394] As described in the above section, planned navigation actions can be tested against predefined constraints to ensure compliance with certain regulations. In some embodiments, this concept can be extended to consideration of potential accident liability. As discussed below, a primary goal of autonomous navigation is safety. Since absolute safety may not be possible (e.g., because at least a particular host vehicle under autonomous control cannot control other vehicles around it, but can only control its own actions), using potential accident liability as a consideration in autonomous navigation and as a constraint on planned actions can help ensure that a particular autonomous vehicle does not perform any actions that are deemed dangerous (e.g., actions that may result in potential accident liability being attributable to the host vehicle). If the host vehicle only performs actions that are determined to be safe and not to cause accidents for which the host vehicle is attributable or liable, a desired level of accident avoidance (e.g., 10 per driving hour) can be achieved. -9 It is possible to achieve less than

[0395] Challenges posed by current approaches to autonomous driving are the lack of safety guarantees (or at least the inability to provide a desired level of safety), as well as the lack of scalability. Consider the problem of ensuring multi-agent safe driving. Since society is unlikely to tolerate road fatalities caused by machines, an acceptable level of safety is paramount to the acceptance of autonomous vehicles. The goal may be to have no accidents at all, but this may not be possible since one can envision situations in which multiple agents are typically involved in accidents and accidents occur only due to the negligence of other agents. For example, as shown in FIG. 19, a host vehicle 1901 is traveling on a multi-lane highway, and the host vehicle 1901 can control its own behavior relative to target vehicles 1903, 1905, 1907, and 1909, but cannot control the behavior of the target vehicles surrounding it. As a result, for example, if vehicle 1905 suddenly cuts into the host vehicle's lane on a collision course with the host vehicle, the host vehicle 1901 may not be able to avoid an accident with at least one of the target vehicles. To address this challenge, the typical response from autonomous vehicle experts is to use a data-driven approach, where safety validation becomes more rigorous as data across more miles is collected.

[0396] But to understand the problematic nature of a data-driven approach to safety, let’s say that the fatality rate caused by accidents per hour of (human) driving is 10 -6 For society to accept machines replacing humans in driving tasks, the fatality rate must be triple digits, or 10 per hour. -9 It is reasonable to assume that the probability of injury should be reduced to 10%. This estimate is similar to the assumed fatality rate for airbags and aviation standards. For example, -9 is the probability that a wing will spontaneously detach from an aircraft in mid-air. However, attempting to guarantee safety using a data-driven approach that provides more confidence in proportion to the accumulated miles traveled is not practical. -9 The amount of data required to guarantee a fatality rate of 109 The time required for a system to run is proportional to the number of hours of data, which is roughly 30 billion miles. Moreover, a multi-agent system interacts with its environment and is unlikely to be verifiable offline (unless a realistic simulator is available that emulates real human driving with all its richness and complexities such as reckless driving (but the problem of validating a simulator is even harder than making a safe autonomous vehicle agent)). Any change to the planning and control software would require the collection of new data on the same scale, which is obviously unwieldy and impractical. Furthermore, developing a system with data will always face a lack of interpretability and explainability of the actions taken, and if an autonomous vehicle (AV) has an accident that results in a fatal accident, we must know why. As a result, a model-based approach for safety is needed, but the existing "functional safety" and ASIL requirements in the automotive industry are not designed to deal with multi-agent environments.

[0397] The second main challenge in developing a safe driving model for autonomous vehicles is the need for scalability. The premise underlying AVs goes beyond "making the world better" and is instead based on the premise that driverless mobility can be sustained at a lower cost than with drivers. This premise is always tied to the concept of scalability in the sense of supporting mass production of AVs (in the millions) and more importantly the trivial incremental cost of being able to operate in new cities. Thus, the cost of computation and sensing is significant, and when AVs are mass produced, the cost of validation and the ability to operate "everywhere" instead of just a few selected cities are also necessary requirements to sustain the business.

[0398] The problem with the current approach is the "brute force" thinking along three axes: (i) the required "computational density," (ii) the way in which the high definition map is defined and created, and (iii) the required specifications of the sensors. The brute force approach defies scalability and shifts the burden to a future where infinite on-board computation is ubiquitous, the cost of building and maintaining HD maps is trivial and scalable, and novel, ultra-advanced sensors are developed and commercialized for vehicle grades at trivial cost. A future where any of the above is realized does indeed make sense, but gaining all of the above impacts is likely a low probability event. Thus, there is a need to provide a formal model that integrates safety and scalability within an AV program that is acceptable to society and scalable in the sense of supporting millions of cars operating anywhere in the developed world.

[0399] The disclosed embodiments represent a solution that can provide a target safety level (and even exceed the safety goal) and can also scale to systems containing millions (or more) of autonomous vehicles. On the safety side, a model called "Responsibility Sensitive Safety" (RSS) is introduced, which formalizes the notion of "accident at fault" to be interpretable and explainable, and incorporates a sense of "responsibility" into the behavior of robotic agents. The definition of RSS is agnostic by the way it is implemented, which is an important feature to further the goal of creating a convincing global safety model. RSS is motivated by the idea that agents play asymmetric roles in accidents (as seen in FIG. 19), in which typically only one of the agents is the cause of the accident and therefore bears responsibility for it. The RSS model also includes a formal treatment of "careful driving" under limited sensing conditions where not all agents are always visible (e.g., due to occlusion). One main goal of the RSS model is to ensure that agents never cause an accident for which they are "at fault" or responsible. A model can only be useful if it is accompanied by an efficient policy (e.g., a function that maps "detected states" to actions) that conforms to the RSS. For example, actions that seem harmless in the present may lead to catastrophe in the distant future (the "butterfly effect"). The RSS can help to construct a set of local constraints on the near-term future that can guarantee (or at least virtually guarantee) that no accidents will occur in the future as a result of the host vehicle's actions.

[0400] Another contribution revolves around the introduction of a "semantic" language made up of units, measurements and action spaces, and a specification of how these are incorporated into AV planning, sensing and actuation. To get an idea of ​​what semantics is like, consider how a human taking a driving course in this context might be instructed to think about a "driving policy". These instructions might be non-geometric ("drive 13.7 metres at current speed, then drive 0.8 m / s"). 2Formal semantic languages ​​do not take the form of "accelerate at a rate of 100%" ("follow the car in front" or "overtake the car on the left"). Rather, these instructions are semantic in nature ("follow the car in front" or "overtake the car on the left"). Typical languages ​​of human driving policies relate to longitudinal and lateral targets rather than to geometric units of acceleration vectors. A formal semantic language can be useful in several ways: it relates to the computational complexity of planning that does not grow exponentially with time and number of agents, it relates to the way safety and comfort interact, it relates to the way in which sensing computations are defined, and it relates to the specification of sensor modalities and how they interact within a fusion methodology. Fusion methodologies (based on semantic languages) can be implemented in a variety of ways, including within a 10 5 While the RSS model only performs offline validation over a data set of about 10 hours of driving data, -9 This could ensure that a mortality rate of 100% is achieved.

[0401] For example, in a reinforcement learning setting, the number of trajectories to be inspected at any given time is limited to 10 4 We consider a Q-function (e.g., a function that evaluates the long-term quality of performing an action a∈A when an agent is in state s∈S) on a semantic space constrained by π(s)=argmax aWe define a Q(s,a) that can be chosen to be the optimal Q(s,a). The signal-to-noise ratio in this space can be high, allowing effective machine learning methods to successfully model the Q-function. In the case of detection computations, semantics can make it possible to distinguish between mistakes that affect safety and mistakes that affect driving comfort. We define a PAC model (Probabilistically Approximately Correct (PAC), to borrow Valiant's PAC learning terminology) for detection that is tied to the Q-function, and show how to incorporate measurement mistakes into the plan in a way that is RSS-compliant but allows for optimization of driving comfort. Since other standard error measures, such as errors relative to a global coordinate system, may not be compliant with the PAC detection model, the semantic language can be important for the success of the characterization aspects of this model. In addition, the semantic language can be a key enabler for defining HD maps that can be built using low-bandwidth detection data and thus can be built by crowdsourcing and supports scalability.

[0402] In summary, the disclosed embodiments may include a formal model that spans the key elements of AVs: sensing, planning, and action. This model may help ensure that there are no incidents of AV responsibility from a planning perspective. Furthermore, due to the PAC sensing model, even with sensing errors, the described fusion methodology may only require a very reasonable amount of offline data set to comply with the described safety model. Furthermore, this model may combine safety and scalability through a semantic language, thereby providing a complete methodology for safe and scalable AVs. Finally, it should be noted that developing an approved safety model that is adopted by industry and regulatory agencies may be a necessary condition for the success of AVs.

[0403] The RSS model may generally follow a classical sense-plan-act robotics control method. The sense system may be responsible for understanding the current state of the host vehicle's environment. The plan part, which may be called a "driving policy" and may be implemented as a set of hard-coded instructions by a trained system (e.g., neural network) or a combination, may be responsible for determining what is the next best move in terms of available options to achieve the driving goal (e.g., how to move from the left lane to the right lane to exit the highway). The act part is responsible for implementing the plan (e.g., a system of actuators and one or more controllers to steer, accelerate, and / or brake the vehicle to perform the selected navigation action, etc.). The embodiments described below focus primarily on the sense and plan parts.

[0404] Accidents can originate from sensing errors or planning errors. Planning is a multi-agent endeavor, since there are other road users (human and machine) that react to the AV's actions. The described RSS model is designed to address safety, especially for the planning part. This can be called multi-agent safety. In statistical methods, the estimation of the probability of planning errors can be done "online". That is, every time a software is updated, billions of miles must be driven with the new version to yield an acceptable estimated level of the frequency of planning errors. This is clearly infeasible. As an alternative, the RSS model can give 100% guarantee (or virtually 100% guarantee) that the planning module will not make a mistake for which the AV is at fault (the concept of "fault" is formally defined). The RSS model can also provide an efficient means for its validation that does not rely on online testing.

[0405] Errors in detection systems may be easier to verify because detection can be independent of vehicle motion and therefore the probability of a significant detection error can be verified using "off-line" data. 9Even collecting offline data over time of driving is difficult. As part of the description of the disclosed detection system, we have described a fusion technique that can be validated using a significantly smaller amount of data.

[0406] The described RSS system can also be scalable to millions of vehicles, for example, the described semantic driving policies and applied safety constraints can be consistent with detection and mapping requirements that are scalable to millions of vehicles even with today's technology.

[0407] A fundamental building block of such a system is a safety definition, a minimum standard that an AV system may need to adhere to. The following technical lemma shows that statistical methods for verifying AV systems are infeasible, even for verifying simple assertions such as "the system will have N accidents per hour". This implies that only model-based safety definitions are feasible tools for verifying AV systems. Lemma 1 Let X be a probability space, A be such that Pr(A)=p 1 <0.1 is an event that occurs.

number

number

number

[0408] To get a perspective on typical values ​​of such probabilities, -9 If you want to reduce the accident probability by 10% for a particular AV system, -8 Assume that the system only provides probabilities of 10 8 Even after gaining operational time, there is a certain probability that the validation process will not show that the system is unsafe.

[0409] Finally, note that this problem concerns disabling a single, specific, dangerous AV system. The complete solution cannot be considered as a single system, since new versions, bug fixes and updates will be required. Each change, even a single line of code, creates a new system from the verifier's point of view. Therefore, a statistically verified solution must do so online, across new samples after every small modification or change, to accommodate shifts in the distribution of states observed and reached by the new system. Obtaining such a huge number of samples repeatedly and systematically (with a certain probability of still failing to verify the system) is infeasible.

[0410] Furthermore, any statistical claim must be formalized for measurement. Asserting statistical properties over the number of accidents a system has is significantly weaker than claiming that "the system operates in a safe manner." To say that, we need to formally define what safety is.

[0411] Absolute security is not possible

[0412] An action a performed by a car c can be considered absolutely safe if it is possible that no accidents will occur after that action at some point in the future. For example, by observing a simple driving scenario as shown in FIG. 19, it can be seen that absolute safety is impossible to achieve. From the viewpoint of the vehicle 1901, there is no action that can guarantee that the surrounding cars will not crash into it. It is also impossible to solve the problem by forbidding the autonomous vehicle from being in such a situation. Since all highways with more than two lanes will invite this problem at some point, forbidding this scenario would mean having to stay in the garage. These implications seem disappointing at first glance. Nothing is absolutely safe. However, as is evident from the fact that human drivers do not follow the requirements of absolute safety, such requirements for obtaining absolute safety as defined above may be too strict. Rather, humans act according to a concept of safety that depends on responsibility.

[0413] responsibility-sensitive safety

[0414] An important aspect missing from the concept of absolute safety is the asymmetric nature of most accidents, i.e. it is usually one of the drivers who is responsible for the collision and therefore at fault. In the example of Fig. 19, for example, the middle car 1901 is not at fault if the left car 1909 suddenly rear-ends it. To formalize the fact that the middle car 1901 is not at fault, the behavior of the AV 1901 that stays in its lane can be considered safe. To do so, we describe a formal concept of "accident fault" or accident liability that can serve as a premise for safe driver law.

[0415] As an example, two cars c are driving side by side at the same speed along a straight road. f ,c r Consider the simple case of the car in front, c f Assume that the car suddenly brakes due to an obstacle on the road and is able to avoid it. f Did not keep a sufficient distance from rI can't react in time. f The collision occurs at the rear of the r It is clear that the situation is different and it is the responsibility of the vehicle behind to maintain a safe distance from the vehicle in front a...

Claims

1. 1. A system for navigating a host vehicle, the system comprising: receiving at least one image from an image capture device representative of an environment of the host vehicle, the at least one image including a representation of a target vehicle within the environment of the host vehicle; determining a planned navigation operation for achieving a navigation goal for the host vehicle based on at least one driving policy; analyzing the at least one image to determine the target vehicle approaching the host vehicle; determining an expected distance between the host vehicle and the target vehicle that would occur after the planned navigation operation is performed if the planned navigation operation is performed; determining a deceleration during braking of the host vehicle, a maximum acceleration of the host vehicle, a response period of the host vehicle, and a current speed of the host vehicle; determining a stopping distance of the host vehicle based on the deceleration during braking of the host vehicle, the maximum acceleration of the host vehicle, the response period of the host vehicle, and the current speed of the host vehicle; determining a current speed of the target vehicle, a maximum acceleration of the target vehicle, a response period of the target vehicle, and a deceleration during braking of the target vehicle; determining a stopping distance for the target vehicle based on the deceleration during braking of the target vehicle, the maximum acceleration of the target vehicle, the response period of the target vehicle, and the current speed of the target vehicle; performing the planned navigation operation if the expected distance is greater than the sum of the stopping distance of the host vehicle and the stopping distance of the target vehicle; A system comprising at least one processing device programmed to:

2. The system of claim 1 , wherein the target vehicle and the host vehicle are traveling in a parking lot.

3. The system of claim 1 , wherein the target vehicle drives close to the host vehicle within a lane occupied by the host vehicle.

4. The system of claim 1 or 2, wherein the deceleration during braking of the host vehicle corresponds to a maximum braking capability of the host vehicle.

5. A system according to any one of claims 1 to 3, wherein the deceleration during braking of the host vehicle corresponds to a predetermined braking capacity which is lower than a maximum braking capacity of the host vehicle.

6. the deceleration during braking of the target vehicle corresponds to a maximum braking capacity of the target vehicle determined based on at least one characteristic of the target vehicle identified through analysis of the at least one image; The system of any one of claims 1 to 5, wherein the at least one characteristic of the target vehicle includes one or more of a vehicle type, a vehicle size, or a vehicle model.

7. A system according to any one of claims 1 to 5, wherein the deceleration during braking of the target vehicle corresponds to a predetermined braking capacity which is lower than a maximum braking capacity of the target vehicle.

8. A system according to any preceding claim, wherein the deceleration during braking of the host vehicle is less than the deceleration during braking of the target vehicle.

9. A system according to any preceding claim, wherein the current speed of the target vehicle is determined based on analysis of two or more received images representative of the environment of the host vehicle.

10. A system according to any preceding claim, wherein the current velocity of the target vehicle is determined based on the output of a radar or lidar unit associated with the host vehicle.

11. The system of any one of claims 1 to 10, wherein the stopping distance of the host vehicle includes a host vehicle acceleration distance corresponding to a distance the host vehicle can travel during the response period of the host vehicle at the maximum acceleration of the host vehicle starting from the determined current speed of the host vehicle.

12. 12. The system of claim 1, wherein the stopping distance of the target vehicle comprises a target vehicle acceleration distance corresponding to a distance the target vehicle can travel during the response period of the target vehicle at the maximum acceleration of the target vehicle starting from the determined current speed of the target vehicle.

13. A system according to any preceding claim, wherein at least one of the host vehicle's deceleration during braking or the target vehicle's deceleration during braking is determined based on a sensed condition of a road surface.

14. A system according to any preceding claim, wherein at least one of the host vehicle's deceleration during braking or the target vehicle's deceleration during braking is determined based on sensed weather conditions.

15. the maximum acceleration of the target vehicle is determined based on at least one characteristic of the target vehicle identified through analysis of the at least one image; The system of any one of claims 1 to 14, wherein the at least one characteristic comprises one or more of a vehicle type, a vehicle size, or a vehicle model.

16. The system of claim 15 , wherein the at least one characteristic of the target vehicle includes one or more of a vehicle type, a vehicle size, or a vehicle model.

17. 17. The system of claim 1, wherein the planned navigation operation includes at least one of a lane change operation, a merge operation, a through operation, a maintain forward operation, a parking lot navigation operation, or a maintain throttle operation.

18. 18. The system of claim 1, wherein the at least one processing device is configured to perform the planned navigation operation if the determined next state distance is greater than the sum of the stopping distance of the host vehicle and the stopping distance of the target vehicle by at least a predetermined minimum distance.

19. 20. The system of claim 18, wherein the predetermined minimum distance corresponds to a predetermined separation distance maintained between the host vehicle and other vehicles.

20. 20. The system of claim 18, wherein the predetermined separation distance is at least 1 meter.

21. A method for a system to navigate a host vehicle, the method comprising: receiving, by the system, at least one image representative of an environment of the host vehicle from an image capture device, the at least one image including a representation of a target vehicle within the environment of the host vehicle; determining a planned navigation operation for achieving a navigation goal for the host vehicle based on at least one driving policy; the system analyzing the at least one image to determine the target vehicle approaching the host vehicle; the system determining an expected distance between the host vehicle and the target vehicle that would occur after the planned navigation operation is performed if the planned navigation operation is performed; the system determining a deceleration during braking of the host vehicle, a maximum acceleration of the host vehicle, a response period of the host vehicle, and a current speed of the host vehicle; the system determining a stopping distance of the host vehicle based on the deceleration during braking of the host vehicle, the maximum acceleration of the host vehicle, the response period of the host vehicle, and the current speed of the host vehicle; the system determining a current speed of the target vehicle, a maximum acceleration of the target vehicle, a response period of the target vehicle, and a deceleration during braking of the target vehicle; the system determining a stopping distance for the target vehicle based on the deceleration during braking of the target vehicle, the maximum acceleration of the target vehicle, the response period of the target vehicle, and the current speed of the target vehicle; if the expected distance is greater than the sum of the stopping distance of the host vehicle and the stopping distance of the target vehicle, the system performs the planned navigation operation; A method comprising:

22. The method of claim 21 , wherein the deceleration during braking of the host vehicle corresponds to a maximum braking capability of the host vehicle.

23. 22. The method of claim 21, wherein the deceleration during braking of the host vehicle corresponds to a predetermined braking capacity that is less than a predetermined maximum braking capacity of the host vehicle.

24. the deceleration during braking of the target vehicle corresponds to a maximum braking capacity of the target vehicle determined based on at least one characteristic of the target vehicle identified through analysis of the at least one image; The method of any one of claims 21 to 23, wherein the at least one characteristic of the target vehicle includes one or more of a vehicle type, a vehicle size, or a vehicle model.

25. A method according to any one of claims 21 to 23, wherein the deceleration during braking of the target vehicle corresponds to a predetermined braking capacity which is lower than a predetermined maximum braking capacity of the target vehicle.

26. A method according to any one of claims 21 to 25, wherein the deceleration during braking of the host vehicle is less than the deceleration during braking of the target vehicle.

27. A method according to any one of claims 21 to 26, wherein the current speed of the target vehicle is determined based on analysis of two or more received images representative of the environment of the host vehicle.

28. A method according to any one of claims 21 to 26, wherein the current velocity of the target vehicle is determined based on the output of a radar or lidar unit associated with the host vehicle.

29. the maximum acceleration of the target vehicle is determined based on at least one characteristic of the target vehicle identified through analysis of the at least one image; The method of any one of claims 21 to 28, wherein the at least one characteristic comprises one or more of a vehicle type, a vehicle size, or a vehicle model.

30. 30. The method of claim 29, wherein the at least one characteristic of the target vehicle includes one or more of a vehicle type, a vehicle size, or a vehicle model.

31. 31. The method of any one of claims 21 to 30, wherein the planned navigation operation includes at least one of a lane change operation, a merge operation, a through operation, a maintain forward operation, a parking lot navigation operation, or a maintain throttle operation.

Images