Steering limiter for vehicle navigation

Abstract

Systems and methods for navigating a host vehicle are provided. In one implementation, a system may include at least one processor configured to receive at least one image captured by an image acquisition device, wherein the at least one image is representative of the host vehicle's environment; to analyze the at least one image to identify at least one property associated with the host vehicle's environment; to determine a navigation action for the host vehicle based on the at least one property associated with the host vehicle's environment and a steering limit corresponding to a maximum permissible lateral acceleration for the host vehicle; and to cause one or more actuators associated with the host vehicle to implement the determined navigation action.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 327,959, filed April 6, 2022. The aforementioned application is incorporated herein by reference in its entirety. BACKGROUND Field of technology

[0002] The present disclosure relates generally to vehicle navigation and, more particularly, to systems and methods for controlling steering operations to reduce shocks to a host vehicle and / or a passenger. Background information

[0003] As technology advances, the goal of a fully autonomous vehicle capable of navigating roadways is on the horizon. Autonomous vehicles may need to consider a variety of factors and make appropriate decisions based on those factors to reach an intended destination safely and accurately. For example, an autonomous vehicle may need to process and interpret visual information (e.g., information acquired by a camera) and may also use information from other sources (e.g., a GPS device, a speed sensor, an accelerometer, a suspension sensor, etc.). At the same time, to navigate to a destination, an autonomous vehicle may also need to know its location within a specific roadway (e.g.,a specific lane within a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, observe traffic signals and signs, proceed from one road to another at appropriate intersections or junctions, and respond to any other situation that occurs or develops during vehicle operation. In addition, the navigation system may be required to adhere to certain imposed restrictions. In some cases, these restrictions may relate to interactions between a host vehicle and one or more other objects, such as other vehicles, pedestrians, etc. In other cases, the restrictions may relate to liability rules that must be followed when implementing one or more navigation actions for a host vehicle.

[0004] In the field of autonomous driving, there are two key considerations for viable autonomous vehicle systems. The first is standardization of safety assurance, including the requirements that each self-driving car must meet to ensure safety and how those requirements can be verified. The second is scalability, as engineering solutions that result in escalating costs are not scalable to millions of cars and could prevent widespread, or even not-so-widespread, adoption of autonomous vehicles. Therefore, there is a need for an interpretable, mathematical model for safety assurance and a design of a system that adheres to safety assurance requirements while scaling to millions of cars. SUMMARY

[0005] Embodiments consistent with the present disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation functions. For example, in accordance with the disclosed embodiments, the disclosed systems may include one, two, or more cameras that monitor the surroundings of a vehicle. The disclosed systems may provide a navigation response based on, for example, an analysis of images captured by one or more cameras. The navigation response may also consider other data, including, for example, global positioning system (GPS) data, sensor data (e.g., from an accelerometer, a speed sensor, a suspension sensor, etc.), and / or other map data.

[0006] In one embodiment, a system for navigating a host vehicle may include at least one processor comprising circuitry and memory.The memory may include instructions executable by the circuitry to cause the at least one processor to receive at least one image captured by an image capture device, the at least one image representative of an environment of the host vehicle; analyze the at least one image to identify at least one characteristic associated with the environment of the host vehicle; determine a navigation action for the host vehicle based on: the at least one characteristic associated with the environment of the host vehicle and a steering limit corresponding to a maximum allowable lateral acceleration for the host vehicle; and cause one or more actuators associated with the host vehicle to implement the determined navigation action.

[0007] In one embodiment, a method for navigating an autonomous host vehicle may include receiving at least one image captured by an image capture device, the at least one image representative of an environment of the host vehicle; analyzing the at least one image to identify at least one characteristic associated with the environment of the host vehicle; determining a navigation action for the host vehicle based on: the at least one characteristic associated with the environment and a steering limit corresponding to a maximum allowable lateral acceleration for the host vehicle; and causing one or more actuators associated with the host vehicle to implement the determined navigation action.

[0008] In accordance with other disclosed embodiments, non-transitory computer-readable storage media may store program instructions executable by at least one processing device and performing any of the steps and / or methods described herein.

[0009] The foregoing general description and the following detailed description are exemplary and explanatory only and do not limit the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various disclosed embodiments. In the drawings: Fig. 1 is a schematic diagram of an exemplary system in accordance with the disclosed embodiments. Fig. 2A is a schematic side view illustration of an exemplary vehicle including a system, in accordance with the disclosed embodiments. Fig. Figure 2B is a schematic plan view of the Fig. 2A, in accordance with the disclosed embodiments. Fig. 2C is a schematic top view illustration of another embodiment of a vehicle including a system, in accordance with the disclosed embodiments. Fig. 2D is a schematic top view illustration of yet another embodiment of a vehicle including a system in accordance with the disclosed embodiments. Fig. 2E is a schematic top view illustration of yet another embodiment of a vehicle including a system in accordance with the disclosed embodiments. Fig. 2F is a schematic diagram of exemplary vehicle control systems, consistent with the disclosed embodiments. Fig. 3A is a schematic representation of an interior of a vehicle including a rearview mirror and a user interface for a vehicle imaging system, in accordance with the disclosed embodiments. Fig. 3B is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and against a vehicle windshield, in accordance with the disclosed embodiments. Fig. 3C is an illustration of the Fig. 3B from a different perspective, in accordance with the disclosed embodiments. Fig. 3D is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and against a vehicle windshield, in accordance with the disclosed embodiments. Fig. 4 is an exemplary block diagram of a memory configured to store instructions for performing one or more operations, in accordance with the disclosed embodiments. Fig. 5A is a flowchart illustrating an exemplary process for initiating one or more navigation responses based on monocular image analysis, in accordance with the disclosed embodiments. Fig. 5B is a flowchart illustrating an example process for detecting one or more vehicles and / or pedestrians in a set of images, in accordance with the disclosed embodiments. Fig. 5C is a flowchart illustrating an exemplary process for detecting road markings and / or lane geometry information in a set of images, in accordance with the disclosed embodiments. Fig. 5D is a flowchart illustrating an exemplary process for detecting traffic lights in a set of images, in accordance with the disclosed embodiments. Fig. 5E is a flowchart illustrating an example process for initiating one or more navigation responses based on a vehicle path, in accordance with the disclosed embodiments. Fig. 5F is a flowchart illustrating an exemplary process for determining whether a leading vehicle is changing lanes, in accordance with the disclosed embodiments. Fig. 6 is a flowchart illustrating an exemplary process for initiating one or more navigation responses based on stereo image analysis, in accordance with the disclosed embodiments. Fig. 7 is a flowchart illustrating an exemplary process for initiating one or more navigation responses based on an analysis of three sets of images, in accordance with the disclosed embodiments. Fig. 8 is a block diagram representation of modules that may be implemented by one or more specially programmed processing devices of an autonomous vehicle navigation system, in accordance with the disclosed embodiments. Fig. 9 is a navigation options graph, in accordance with the disclosed embodiments. Fig. 10 is a navigation options graph, in accordance with the disclosed embodiments. The Fig. 11A, Fig. 11B and Fig. 11C provide a schematic representation of the navigation options of a host vehicle in a merging zone, in accordance with the disclosed embodiments. Fig. 11D provides a schematic representation of a double-threading scenario, in accordance with the disclosed embodiments. Fig. 11E provides an options graph potentially useful in a double-threading scenario, in accordance with the disclosed embodiments. Fig. 12 provides a diagram of a representative image captured of a host vehicle's surroundings, along with potential navigation constraints, in accordance with the disclosed embodiments. Fig. 13 provides an algorithmic flowchart for navigating a vehicle in accordance with the disclosed embodiments. Fig. 14 provides an algorithmic flowchart for navigating a vehicle in accordance with the disclosed embodiments. Fig. 15 provides an algorithmic flowchart for navigating a vehicle in accordance with the disclosed embodiments. Fig. 16 provides an algorithmic flowchart for navigating a vehicle in accordance with the disclosed embodiments. The Fig. 17A and Fig. 17B provide a schematic illustration of a host vehicle navigating into a roundabout, in accordance with the disclosed embodiments. Fig. 18 provides an algorithmic flowchart for navigating a vehicle in accordance with the disclosed embodiments. Fig. 19 illustrates an example of a host vehicle traveling on a multi-lane highway in accordance with the disclosed embodiments. The Fig. 20A and Fig. 20B illustrate examples of a vehicle merging in front of another vehicle in accordance with the disclosed embodiments. Fig. 21 illustrates an example of a vehicle following another vehicle in accordance with the disclosed embodiments. Fig. 22 illustrates an example of a vehicle leaving a parking lot and merging onto a potentially busy roadway, in accordance with the disclosed embodiments. Fig. 23 illustrates a vehicle traveling on a road in accordance with the disclosed embodiments. The Fig. 24A-24D illustrate four example scenarios, consistent with the disclosed embodiments. Fig. 25 illustrates an example scenario, consistent with the disclosed embodiments. Fig. 26 illustrates an example scenario, consistent with the disclosed embodiments. Fig. 27 illustrates an example scenario, consistent with the disclosed embodiments. The Fig. 28A and Fig. 28B illustrate an example scenario in which a vehicle is following another vehicle, in accordance with the disclosed embodiments. The Fig. 29A and Fig. 29B illustrate an example of fault in cut-in scenarios, in accordance with the disclosed embodiments. The Fig. 30A and Fig. 30B illustrate an example of fault in cut-in scenarios, in accordance with the disclosed embodiments. The Fig. 31A-31D illustrate an example of fault determination in drift scenarios, in accordance with the disclosed embodiments. The Fig. 32A and Fig. 32B illustrate an example of fault determination in oncoming traffic scenarios, in accordance with the disclosed embodiments. The Fig. 33A and Fig. 33B illustrate an example of fault determination in oncoming traffic scenarios, in accordance with the disclosed embodiments. The Fig. 34A and Fig. 34B illustrate an example of fault determination in route priority scenarios, in accordance with the disclosed embodiments. The Fig. 35A and Fig. 35B illustrate an example of fault determination in route priority scenarios, in accordance with the disclosed embodiments. The Fig. 36A and Fig. 36B illustrate an example of fault determination in route priority scenarios, in accordance with the disclosed embodiments. The Fig. 37A and Fig. 37B illustrate an example of fault determination in route priority scenarios, in accordance with the disclosed embodiments. The Fig. 38A and Fig. 38B illustrate an example of fault determination in route priority scenarios, in accordance with the disclosed embodiments. The Fig. 39A and Fig. 39B illustrate an example of fault determination in route priority scenarios, in accordance with the disclosed embodiments. The Fig. 40A and Fig. 40B illustrate an example of the question of fault in traffic light scenarios, in accordance with the disclosed embodiments. The Fig. 41A and Fig. 41B illustrate an example of the question of fault in traffic light scenarios, in accordance with the disclosed embodiments. The Fig. 42A and Fig. 42B illustrate an example of the question of fault in traffic light scenarios, in accordance with the disclosed embodiments. The Fig. 43A-43C illustrate example scenarios involving vulnerable road users (VRUs), in accordance with the disclosed embodiments. The Fig. 44A-44C illustrate example scenarios involving vulnerable road users (VRUs), in accordance with the disclosed embodiments. The Fig. 45A-45C illustrate example scenarios involving vulnerable road users (VRUs), consistent with the disclosed embodiments. The Fig. 46A-46D illustrate example scenarios involving vulnerable road users (VRUs), in accordance with the disclosed embodiments. The Fig. 47A and Fig. 47B illustrate example scenarios in which a vehicle follows another vehicle, in accordance with the disclosed embodiments. Fig. 48 is a flowchart illustrating an exemplary process for navigating a host vehicle, in accordance with the disclosed embodiments. The Fig. 49A-49D illustrate example scenarios in which a vehicle follows another vehicle, in accordance with the disclosed embodiments. Fig. 50 is a flowchart illustrating an exemplary process for braking a host vehicle, in accordance with the disclosed embodiments. Fig. 51 is a flowchart illustrating an exemplary process for navigating a host vehicle, in accordance with the disclosed embodiments. The Fig. 52A-52D illustrate example proximity buffers for a host vehicle, in accordance with the disclosed embodiments. The Fig. 53A and Fig. 53B illustrate example scenarios including a proximity buffer, in accordance with the disclosed embodiments. The Fig. 54A and Fig. 54B illustrate example scenarios including a proximity buffer, in accordance with the disclosed embodiments. Fig. 55 provides a flowchart for selectively displacing human driver control of a host vehicle, in accordance with the disclosed embodiments. Fig. 56 is a flowchart illustrating an exemplary process for navigating a host vehicle, in accordance with the disclosed embodiments. The Fig. 57A-57C illustrate example scenarios consistent with the disclosed embodiments. Fig. 58 is a flowchart illustrating an exemplary process for navigating a host vehicle, in accordance with the disclosed embodiments. Fig. 59 illustrates an example scenario for navigating a vehicle using one or more control loops, in accordance with the disclosed embodiments. Fig. 60 is a flowchart illustrating an exemplary process for generating a steering command, in accordance with the disclosed embodiments. Fig. 61 is a schematic illustration of a steering movement of a vehicle, in accordance with the disclosed embodiments. Fig. 62 is a block diagram of an exemplary vehicle, in accordance with the disclosed embodiments. Fig. 63 is a flowchart illustrating an exemplary process for navigating a host vehicle, in accordance with the disclosed embodiments. Fig. 64 illustrates an example environment in which a vehicle may limit steering movement in accordance with the disclosed embodiments. Fig. 65 is a schematic illustration of a steering movement of a vehicle that may be limited by one or more steering limits, in accordance with the disclosed embodiments. Fig. 66 illustrates an example scenario in which a host vehicle may forego implementing a steering limit, in accordance with the disclosed embodiments. Fig. 67 is a flowchart illustrating an example process for navigating a host vehicle, in accordance with the disclosed embodiments. DETAILED DESCRIPTION

[0011] The following detailed description refers to the accompanying drawings. Where possible, the same reference numerals are used in the drawings and the following description to refer to the same or similar parts. While several illustrative embodiments are described herein, modifications, adaptations, and other implementations are possible. For example, substitutions, additions, or modifications may be made to the components illustrated in the drawings, and the illustrative methods described herein may be modified by replacing, rearranging, removing, or adding steps to the disclosed methods. Accordingly, the following detailed description is not limited to the disclosed embodiments and examples. Instead, the proper scope is defined by the appended claims. Overview of autonomous vehicles

[0012] As used throughout this disclosure, the term "autonomous vehicle" refers to a vehicle capable of implementing at least one navigation change without driver input. A "navigation change" refers to a change in one or more of the vehicle's steering, braking, or acceleration / deceleration. To be autonomous, a vehicle does not have to be fully automatic (e.g., operating entirely without a driver or without driver input). Instead, an autonomous vehicle includes those capable of operating under driver control during certain periods and without driver control during other periods. Autonomous vehicles may also include vehicles that control only some aspects of vehicle navigation, such as steering (e.g.,to maintain a vehicle's course between vehicle lane restrictions) or some steering operations under certain circumstances (but not all circumstances), but may leave other aspects to the driver (e.g., braking or deceleration under certain circumstances). In some cases, autonomous vehicles may handle some or all aspects of braking, speed control, and / or steering.

[0013] Because human drivers typically rely on visual cues and observations to steer a vehicle, traffic infrastructures are built accordingly, with lane markings, traffic signs, and traffic lights all designed to provide visual information to drivers. Given these design characteristics of traffic infrastructures, an autonomous vehicle may include a camera and a processing unit that analyzes visual information captured from the vehicle's surroundings. The visual information may include, for example, traffic infrastructure components (e.g., lane markings, traffic signs, traffic lights, etc.) observable by drivers and other obstacles (e.g., other vehicles, pedestrians, debris, etc.).Additionally, an autonomous vehicle may also use stored information, such as information that provides a model of the vehicle's surroundings when navigating. For example, the vehicle may use GPS data, sensor data (e.g., from an accelerometer, a speed sensor, a suspension sensor, etc.), and / or other mapping data to provide information related to its surroundings while the vehicle is traveling, and the vehicle (as well as other vehicles) may use the information to locate itself on the model. Some vehicles are also capable of communicating with each other, exchanging information, warning the oncoming vehicle of hazards or changes in the vehicles' surroundings, etc. System overview

[0014] Fig. 1 is a block diagram representation of a system 100, in accordance with the exemplary disclosed embodiments. The system 100 may include various components depending on the requirements of a particular implementation. In some embodiments, the system 100 may include a processing unit 110, an image capture unit 120, a position sensor 130, one or more storage units 140, 150, a map database 160, a user interface 170, and a wireless transceiver 172. The processing unit 110 may include one or more processing devices. In some embodiments, the processing unit 110 may include an application processor 180, an image processor 190, or other suitable processing device.Similarly, the image capture unit 120 may include any number of image capture devices and components depending on the requirements of a particular application. In some embodiments, the image capture unit 120 may include one or more image capture devices (e.g., cameras, CCDs, or any other type of image sensor), such as the image capture device 122, the image capture device 124, and the image capture device 126. The system 100 may also include a data interface 128 that communicatively connects the processing device 110 to the image capture device 120. For example, the data interface 128 may include one or more wired and / or wireless connections for transmitting the image data captured by the image capture device 120 to the processing unit 110.

[0015] The wireless transceiver 172 may include one or more devices configured to exchange transmissions over an air interface to one or more networks (e.g., cellular, Internet, etc.) using a radio frequency, an infrared frequency, a magnetic field, or an electric field. The wireless transceiver 172 may use any known standard for transmitting and / or receiving data (e.g., Wi-Fi, Bluetooth®, Bluetooth Smart, 802.15.4, ZigBee, etc.). Such transmissions may include communications from the host vehicle to one or more remotely located servers. Such transmissions may also include communications (one-way or two-way) between the host vehicle and one or more target vehicles in a vicinity of the host vehicle (e.g.,to facilitate the coordination of the host vehicle's navigation with respect to or together with target vehicles in the host vehicle's vicinity) or even involve broadcast transmission to unspecified receivers in an area surrounding the transmitting vehicle.

[0016] Both application processor 180 and image processor 190 may include various types of processing devices. For example, one or both of application processor 180 and image processor 190 may include a microprocessor, preprocessors (such as an image preprocessor), a graphics processor, a central processing unit (CPU), support circuitry, digital signal processors, integrated circuits, memory, or any other type of device suitable for executing applications and for image processing and analysis. In some embodiments, application processor 180 and / or image processor 190 may include any type of single- or multi-core processor, mobile device microcontroller, central processing unit, etc.Various processing devices may be used, including, for example, processors available from manufacturers such as Intel® AMD® etc., and may include different architectures (e.g., x86 processor, ARM® etc.).

[0017] In some embodiments, the application processor 180 and / or the image processor 190 may include any of the EyeQ series of processor chips available from Mobileye®. These processor configurations each include multiple processing units with local memory and instruction sets. Such processors may include video inputs for receiving image data from multiple image sensors and may also include video output capabilities. In one example, the EyeQ2® uses 90 nm micrometer technology operating at 332 MHz. The EyeQ2® architecture consists of two floating-point hyper-threaded 32-bit RISC CPUs (MIPS32® 34K® cores), five Vision Computing Engines (VCE), three Vector Microcode Processors (VMP® Denali 64-bit mobile DDR controller, 128-bit internal Sonics Interconnect, dual 16-bit video input and 18-bit video output controllers, 16-channel DMA, and multiple peripherals.The MIPS34K CPU manages the five VCEs, three VMP™, and the DMA, the second MIPS34K CPU and the multi-channel DMA, as well as the other peripherals. The five VCEs, three VMP®, and the MIPS34K CPU can perform intensive vision computations required by multifunction bundle applications. In another example, the EyeQ3®, which is a third-generation processor and six times more powerful than the EyeQ2®, can be used in the disclosed embodiments. In other examples, the EyeQ4® and / or the EyeQ5® can be used in the disclosed embodiments. Of course, any newer or future EyeQ processing devices can also be used with the disclosed embodiments.

[0018] Any of the processing devices disclosed herein may be configured to perform particular functions. Configuring a processing device, such as any of the described EyeQ processors or other controller or microprocessor, to perform particular functions may involve programming computer-executable instructions and providing those instructions to the processing device for execution during operation of the processing device. In some embodiments, configuring a processing device may involve programming the processing device directly with architectural instructions. In other embodiments, configuring a processing device may involve storing executable instructions in memory accessible to the processing device during operation.For example, the processing device may access the memory to obtain and execute the stored instructions during operation. In any event, the processing device configured to perform the sensor sensing, image analysis, and / or navigation functions disclosed herein represents a specialized hardware-based system that controls multiple hardware-based components of a host vehicle.

[0019] Although Fig. 1 depicts two separate processing devices included within processing unit 110, more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to accomplish the tasks of application processor 180 and image processor 190. In other embodiments, these tasks may be performed by more than two processing devices. Furthermore, in some embodiments, system 100 may include one or more processing units 110 without including other components, such as image acquisition unit 120.

[0020] The processing unit 110 may include various types of devices. For example, the processing unit 110 may include various devices such as a controller, an image preprocessor, a central processing unit (CPU), support circuits, digital signal processors, integrated circuits, memory, or any other type of device for image processing and analysis. The image preprocessor may include a video processor for acquiring, digitizing, and processing the image material from the image sensors. The CPU may include any number of microcontrollers or microprocessors. The support circuits may be any number of circuits generally well known in the art, including cache, power supply, clock, and input / output circuits. The memory may store software that, when executed by the processor, controls the operation of the system.The memory may include databases and image processing software. The memory may include any number of random-access memories, read-only memories, flash memories, disk drives, optical storage, tape storage, removable storage, and other types of storage. In one case, the memory may be separate from the processing unit 110. In another case, the memory may be integrated into the processing unit 110.

[0021] Each memory 140, 150 may include software instructions that, when executed by a processor (e.g., application processor 180 and / or image processor 190), may control the operation of various aspects of system 100. These storage units may include various databases and image processing software, as well as a trained system, such as a neural network or deep neural network. The storage units may include random-access memory, read-only memory, flash memory, disk drives, optical storage, tape storage, removable storage, and / or any other type of storage. In some embodiments, the storage units 140, 150 may be separate from the application processor 180 and / or image processor 190. In other embodiments, these storage units may be integrated into the application processor 180 and / or image processor 190.

[0022] Position sensor 130 may include any type of device suitable for determining a location associated with at least one component of system 100. In some embodiments, position sensor 130 may include a GPS receiver. Such receivers may determine user position and velocity by processing signals broadcast by Global Positioning System satellites. Position information from position sensor 130 may be provided to application processor 180 and / or image processor 190.

[0023] In some embodiments, system 100 may include components such as a speed sensor (e.g., a speedometer) for measuring a speed of vehicle 200. System 100 may also include one or more accelerometers (either single-axis or multi-axis) for measuring accelerations of vehicle 200 along one or more axes.

[0024] The storage units 140, 150 may include a database or otherwise organized data indicating the location of known landmarks. Sensory information (such as images, radar signals, depth information from lidar, or stereo processing of two or more images) of the environment may be processed along with position information, such as a GPS coordinate, the vehicle's own motion, etc., to determine the vehicle's current location relative to the known landmarks and to refine the vehicle's location. Certain aspects of this technology are included in a localization technology known as REM™, which is marketed by the assignee of the present application.

[0025] The user interface 170 may include any device suitable for providing information to or receiving input from one or more users of the system 100. In some embodiments, the user interface 170 may include user input devices, including, for example, a touchscreen, a microphone, a keyboard, pointing devices, scroll wheels, cameras, knobs, buttons, etc. With such input devices, a user may be able to provide information input or commands to the system 100 by typing instructions or information, providing voice commands, selecting menu options on a screen using buttons, pointers, or eye-tracking capabilities, or through any other suitable techniques for communicating information to the system 100.

[0026] The user interface 170 may be equipped with one or more processing devices configured to provide and receive information to or from a user and process that information for use by, for example, the application processor 180. In some embodiments, such processing devices may execute instructions for detecting and tracking eye movements, receiving and interpreting voice commands, detecting and interpreting touches and / or gestures on a touchscreen, responding to keyboard inputs or menu selections, etc. In some embodiments, the user interface 170 may include a display, a speaker, a tactile device, and / or any other device for providing output information to a user.

[0027] The map database 160 may include any type of database for storing map data useful to the system 100. In some embodiments, the map database 160 may include data relating to the position in a reference coordinate system of various elements, including roads, water features, geographical features, shops, points of interest, restaurants, gas stations, etc. The map database 160 may store not only the locations of such objects, but also descriptors relating to those elements, including, for example, names associated with any of the stored features. In some embodiments, the map database 160 may be physically located with other components of the system 100. Alternatively or additionally, the map database 160, or a portion thereof, may be remotely located with respect to other components of the system 100 (e.g., processing unit 110).In such embodiments, information may be downloaded from the map database 160 via a wired or wireless data connection to a network (e.g., via a cellular network and / or the Internet, etc.). In some cases, the map database 160 may store a sparse data model that includes polynomial representations of certain road features (e.g., lane markings) or target trajectories for the host vehicle. The map database 160 may also include stored representations of various detected landmarks that can be used to determine or update a known position of the host vehicle relative to a target trajectory. The landmark representations may include data fields such as the type of landmark and the location of the landmark, among other potential identifiers.

[0028] The image capture devices 122, 124, and 126 may include any type of device suitable for capturing at least one image from an environment. Furthermore, any number of image capture devices may be used to capture images for input to the image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. The image capture devices 122, 124, and 126 are described below with reference to the Fig. 2B-2E are described in more detail.

[0029] One or more cameras (e.g., image capture devices 122, 124, and 126) may be part of a sensor sensing block on a vehicle. Various other sensors may be included in the sensor sensing block, and any or all of the sensors may be used to develop a sensor-sensed navigation state of the vehicle. In addition to the cameras (forward, sideways, backward, etc.), other sensors, such as radar, lidar, and acoustic sensors, may also be included in the sensor sensing block. Additionally, the sensor sensing block may include one or more components configured to transmit / receive information related to the vehicle's surroundings. For example, such components may include wireless transceivers (RF, etc.).) that can receive sensor-based information or any other type of information regarding the host vehicle's surroundings from a source remote from the host vehicle. Such information may include sensor output information or related information received from vehicle systems other than the host vehicle. In some embodiments, such information may include information received from a remote computing device, a central server, etc. Furthermore, the cameras may take on many different configurations: single-camera units, multiple cameras, camera clusters, long field of view, short field of view, wide angle, fisheye, etc.

[0030] The system 100 or various components thereof may be integrated into various different platforms. In some embodiments, the system 100 may be included in a vehicle 200, as shown in Fig. 2A. For example, the vehicle 200 may be equipped with a processing unit 110 and any of the other components of the system 100 as described above with respect to Fig. 1. While in some embodiments, the vehicle 200 may be equipped with only a single image capture device (e.g., a camera), in other embodiments, such as those described in connection with the Fig. 2B-2E, multiple image capture devices may be used. For example, each of the image capture devices 122 and 124 of the vehicle 200, as shown in Fig. 2A, be part of an ADAS (Advanced Driver Assistance Systems) imaging set.

[0031] The image capture devices included in the vehicle 200 as part of the image capture unit 120 may be positioned at any suitable location. In some embodiments, as shown in the Fig. As shown in Figures 2A-2E and 3A-3C, the image capture device 122 may be located near the rearview mirror. This position may provide a line of sight similar to that of the driver of the vehicle 200, which may assist in determining what is and is not visible to the driver. The image capture device 122 may be positioned anywhere near the rearview mirror, but placing the image capture device 122 on the driver's side of the mirror may further assist in obtaining images representative of the driver's field of view and / or line of sight.

[0032] Other locations for the image capture devices of the image capture unit 120 may also be used. For example, the image capture device 124 may be located on or in a bumper of the vehicle 200. Such a location may be particularly suitable for image capture devices with a wide field of view. The line of sight of the bumper-located image capture devices may differ from that of the driver, and therefore the bumper image capture device and the driver may not always see the same objects. The image capture devices (e.g.,Image capture devices 122, 124, and 126) may also be located in other locations. For example, the image capture devices may be located on or in one or both of the side view mirrors of the vehicle 200, on the roof of the vehicle 200, on the hood of the vehicle 200, on the trunk of the vehicle 200, on the sides of the vehicle 200, mounted on, positioned behind, or positioned in front of one of the windows of the vehicle 200, and mounted in or near light figures on the front and / or rear of the vehicle 200, etc.

[0033] In addition to the image capture devices, the vehicle 200 may include various other components of the system 100. For example, the processing unit 110 may be included in the vehicle 200, either integrated with or separate from an engine control unit (ECU) of the vehicle. The vehicle 200 may also be equipped with a position sensor 130, such as a GPS receiver, and may also include a map database 160 and storage units 140 and 150.

[0034] As previously discussed, the wireless transceiver 172 may transmit and / or receive data over one or more networks (e.g., cellular networks, the Internet, etc.). For example, the wireless transceiver 172 may upload data collected by the system 100 to one or more servers and download data from the one or more servers. For example, the wireless transceiver 172 may receive periodic or on-demand updated data stored in the map database 160, the memory 140, and / or the storage 150 via the wireless transceiver 172. Similarly, the wireless transceiver 172 may upload any data (e.g., images captured by the image capture unit 120, data received from the position sensor 130 or other sensors, vehicle control systems, etc.) from the system 100 and / or any data processed by the processing unit 110 to the one or more servers.

[0035] System 100 may upload data to a server (e.g., to the cloud) based on a privacy level setting. For example, system 100 may implement privacy level settings to regulate or limit the types of data (including metadata) sent to the server that can uniquely identify a vehicle and / or a driver / owner of a vehicle. Such settings may be set by the user, for example, via wireless transceiver 172, initialized by factory default settings, or by data received by wireless transceiver 172.

[0036] In some embodiments, the system 100 may upload data according to a "high" privacy level, and, under a setting, the system 100 may transmit data (e.g., location information related to a route, captured images, etc.) without any details about the specific vehicle and / or driver / owner. For example, when uploading data according to a "high" privacy setting, the system 100 may not include a vehicle identification number (VIN) or a name of a driver or owner of the vehicle and may instead transmit data such as captured images and / or limited location information related to a route.

[0037] Other privacy levels are also contemplated. For example, system 100 may transmit data to a server according to a "medium" privacy level and include additional information not included under a "high" privacy level, such as a make and / or model of a vehicle and / or a vehicle type (e.g., a passenger car, an SUV, a truck, etc.). In some embodiments, system 100 may upload data according to a "low" privacy level. Under a "low" privacy level setting, system 100 may upload data and include information sufficient to uniquely identify a specific vehicle, a specific owner / driver, and / or a portion or the entirety of a route traveled by the vehicle.Such “low” privacy level data may include one or more of, for example, a VIN, a driver / owner name, a vehicle’s point of origin prior to departure, an intended destination of the vehicle, a make and / or model of the vehicle, a type of vehicle, etc.

[0038] Fig. 2A is a schematic side view illustration of an exemplary vehicle imaging system, in accordance with the disclosed embodiments. Fig. Figure 2B is a schematic top view illustration of the Fig. 2A. As shown in Fig. 2B, the disclosed embodiments may include a vehicle 200 including, within its body, a system 100 having a first image capture device 122 positioned near the rearview mirror and / or near the driver of the vehicle 200, a second image capture device 124 positioned on or in a bumper area (e.g., one of the bumper areas 210) of the vehicle 200, and a processing unit 110.

[0039] As in Fig. 2C, the image capture devices 122 and 124 may both be positioned near the rearview mirror and / or near the driver of the vehicle 200. In addition, although two image capture devices 122 and 124 may be Fig. 2B and Fig. 2C, it should be understood that other embodiments may include more than two image capture devices. For example, in the Fig. 2D and Fig. 2E, the first, second, and third image capture devices 122, 124, and 126 are included in the system 100 of the vehicle 200.

[0040] As in Fig. 2D, the image capture device 122 may be positioned near the rearview mirror and / or near the driver of the vehicle 200, and the image capture devices 124 and 126 may be positioned on or in a bumper area (e.g., one of the bumper areas 210) of the vehicle 200. And as in Fig. 2E, the image capture devices 122, 124, and 126 may be positioned near the rearview mirror and / or near the driver's seat of the vehicle 200. The disclosed embodiments are not limited to any particular number and configuration of the image capture devices, and the image capture devices may be positioned at any suitable location within and / or on the vehicle 200.

[0041] It is understood that the disclosed embodiments are not limited to vehicles and could be applied in other contexts. It is also understood that the disclosed embodiments are not limited to any particular vehicle type 200 and may apply to all vehicle types, including automobiles, trucks, trailers, and other vehicle types.

[0042] The first image capture device 122 may include any suitable type of image capture device. The image capture device 122 may include an optical axis. In one case, the image capture device 122 may include an Aptina M9V024 WGA sensor with a global shutter. In other embodiments, the image capture device 122 may provide a resolution of 1280x960 pixels and include a rolling shutter. The image capture device 122 may include various optical elements. In some embodiments, one or more lenses may be included, for example, to provide a desired focal length and field of view for the image capture device. In some embodiments, the image capture device 122 may be associated with a 6 mm lens or a 12 mm lens.In some embodiments, the image capture device 122 may be configured to capture images with a desired field of view (FOV) 202, as shown in FIG. Fig. 2D. For example, the image capture device 122 may be configured to have a regular FOV, such as within a range of 40 degrees to 56 degrees, including a 46-degree FOV, 50-degree FOV, 52-degree FOV, or greater. Alternatively, the image capture device 122 may be configured to have a narrow FOV in the range of 23 to 40 degrees, such as a 28-degree FOV or 36-degree FOV. Additionally, the image capture device 122 may be configured to have a wide FOV in the range of 100 to 180 degrees. In some embodiments, the image capture device 122 may include a wide-angle bumper camera or one with up to a 180-degree FOV. In some embodiments, the image capture device 122 may be a 7.2M pixel image capture device with an aspect ratio of about 2:1 (e.g., HxV = 3800x1900 pixels) with about 100 degrees horizontal FOV.Such an image capture device can be used instead of a configuration with three image capture devices. Due to significant lens distortion, the vertical FOV of such an image capture device can be significantly less than 50 degrees in implementations where the image capture device uses a radially symmetric lens. For example, such a lens may not be radially symmetric, which would allow a vertical FOV of more than 50 degrees with a horizontal FOV of 100 degrees.

[0043] The first image capture device 122 may capture a plurality of first images relative to a scene associated with the vehicle 200. Each of the plurality of first images may be acquired as a series of image scan lines, which may be captured using a rolling shutter. Each scan line may include a plurality of pixels.

[0044] The first image capture device 122 may have a sampling rate associated with the acquisition of each of the first series of image scan lines. The sampling rate may refer to a rate at which an image sensor can acquire image data associated with each pixel included in a particular scan line.

[0045] The image capture devices 122, 124, and 126 may include any suitable type and number of image sensors, including, for example, CCD sensors or CMOS sensors. In one embodiment, a CMOS image sensor may be employed in conjunction with a rolling shutter, such that each pixel in a row is read sequentially, and the rows are scanned on a row-by-row basis until an entire image frame has been captured. In some embodiments, the rows may be captured sequentially from top to bottom with respect to the frame.

[0046] In some embodiments, one or more of the image capture devices disclosed herein (e.g., image capture devices 122, 124, and 126) may form a high-resolution imager and may have a resolution of greater than 5 M pixels, 7 M pixels, 10 M pixels, or more.

[0047] The use of a rolling shutter can result in pixels in different rows being exposed and captured at different times, which can introduce distortion and other image artifacts into the captured frame. On the other hand, if the image capture device 122 is configured to operate with a global or synchronous shutter, all pixels may be exposed for the same time and during a common exposure period. As a result, the image data in a frame collected by a system using a global shutter represents a snapshot of the entire FOV (such as FOV 202) at a particular time. In contrast, in a rolling shutter application, each row in a frame is exposed, and data is captured at different times. Therefore, moving objects may appear distorted in an image capture device with a rolling shutter.This phenomenon is described in more detail below.

[0048] The second image capture device 124 and the third image capture device 126 may be any type of image capture device. Like the first image capture device 122, each of the image capture devices 124 and 126 may include an optical axis. In one embodiment, each of the image capture devices 124 and 126 may include an Aptina M9V024 WVGA sensor with a global shutter. Alternatively, each of the image capture devices 124 and 126 may include a rolling shutter. Like the image capture device 122, the image capture devices 124 and 126 may be configured to include various lenses and optical elements. In some embodiments, the lenses associated with image capture devices 124 and 126 may provide FOVs (such as FOVs 204 and 206) that are equal to or narrower than an FOV (such as FOV 202) associated with image capture device 122.For example, image capture devices 124 and 126 may have FOVs of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less.

[0049] Image capture devices 124 and 126 may capture a plurality of second and third images relative to a scene associated with vehicle 200. Each of the plurality of second and third images may be captured as a second and third series of image scan lines, which may be captured using a rolling shutter. Each scan line or row may include a plurality of pixels. Image capture devices 124 and 126 may have second and third sampling rates associated with the capture of each of the image scan lines included in the second and third series.

[0050] Each image capture device 122, 124, and 126 may be positioned at any suitable position and orientation relative to the vehicle 200. The relative positioning of the image capture devices 122, 124, and 126 may be selected to assist in merging the information captured by the image capture devices. For example, in some embodiments, an FOV (such as FOV 204) associated with the image capture device 124 may partially or completely overlap with an FOV (such as FOV 202) associated with the image capture device 122 and an FOV (such as FOV 206) associated with the image capture device 126.

[0051] The image capture devices 122, 124, and 126 may be located on the vehicle 200 at any suitable relative heights. In one case, there may be a height difference between the image capture devices 122, 124, and 126 that may provide sufficient parallax information to enable stereo analysis. For example, as shown in Fig. As shown in Figure 2A, the two image capture devices 122 and 124 are located at different heights. There may also be a lateral displacement difference between the image capture devices 122, 124, and 126, which, for example, provides additional parallax information for stereo analysis by the processing unit 110. The difference in lateral displacement can be determined by the x be referred to as in the Fig. 2C and Fig. 2D. In some embodiments, a forward or backward offset (e.g., distance offset) may exist between image capture devices 122, 124, and 126. For example, image capture device 122 may be located 0.5 to 2 meters or more behind image capture device 124 and / or image capture device 126. This type of offset may allow one of the image capture devices to cover potential blind spots of the other image capture device(s).

[0052] The image capture devices 122 may have any suitable resolution capability (e.g., number of pixels associated with the image sensor), and the resolution of the image sensor(s) associated with the image capture device 122 may be higher, lower, or equal to the resolution of the image sensor(s) associated with the image capture devices 124 and 126. In some embodiments, the image sensor(s) associated with the image capture device 122 and / or the image capture devices 124 and 126 may have a resolution of 640×480, 1024×768, 1280×960, or any other suitable resolution.

[0053] The frame rate (e.g., the rate at which an image capture device captures a set of pixel data of one image frame before moving to capture pixel data associated with the next image frame) may be controllable. The frame rate associated with image capture device 122 may be higher, lower, or equal to the frame rate associated with image capture devices 124 and 126. The frame rate associated with image capture devices 122, 124, and 126 may depend on a variety of factors that may affect the timing of the frame rate. For example, one or more of image capture devices 122, 124, and 126 may include a selectable pixel delay period imposed before or after the capture of image data associated with one or more pixels of an image sensor in image capture device 122, 124, and / or 126.In general, image data corresponding to each pixel may be captured according to a clock rate for the device (e.g., one pixel per clock cycle). Additionally, in embodiments including a rolling shutter, one or more of the image capture devices 122, 124, and 126 may include a selectable horizontal blanking period imposed before or after capturing image data associated with a row of pixels of an image sensor in the image capture device 122, 124, and / or 126. Further, one or more of the image capture devices 122, 124, and / or 126 may include a selectable vertical blanking period imposed before or after capturing image data associated with a frame of the image capture device 122, 124, and / or 126.

[0054] These timing controls can enable synchronization of frame rates associated with image capture devices 122, 124, and 126, even if the line scan rates are different. Additionally, as discussed in more detail below, these selectable timing controls, among other factors (e.g., image sensor resolution, maximum line scan rates, etc.), can enable synchronization of image capture from an area where the FOV of image capture device 122 overlaps with one or more FOVs of image capture devices 124 and 126, even if the field of view of image capture device 122 differs from the FOVs of image capture devices 124 and 126.

[0055] The frame rate timing in image capture devices 122, 124, and 126 may depend on the resolution of the associated image sensors. For example, if similar line scan rates are assumed for both devices, if one device includes an image sensor with a resolution of 640 × 480 and another device includes an image sensor with a resolution of 1280 × 960, then more time will be required to capture a frame of image data from the higher-resolution sensor.

[0056] Another factor that can affect the timing of image data acquisition in image capture devices 122, 124, and 126 is the maximum line scan rate. For example, acquiring a series of image data from an image sensor included in image capture devices 122, 124, and 126 requires a minimum amount of time. Assuming no pixel delay periods are added, this minimum amount of time for acquiring a series of image data is related to the maximum line scan rate for a particular device. Devices that offer higher maximum line scan rates have the potential to provide higher frame rates than devices with lower maximum line scan rates. In some embodiments, one or more of image capture devices 124 and 126 may have a maximum line scan rate that is higher than a maximum line scan rate associated with image capture device 122.In some embodiments, the maximum line scan rate of image capture device 124 and / or 126 may be 1.25, 1.5, 1.75, or 2 times or more a maximum line scan rate of image capture device 122.

[0057] In another embodiment, image capture devices 122, 124, and 126 may have the same maximum line scan rate, but image capture device 122 may operate at a scan rate less than or equal to its maximum scan rate. The system may be configured such that one or more of image capture devices 124 and 126 operate at a line scan rate equal to the line scan rate of image capture device 122. In other cases, the system may be configured such that the line scan rate of image capture device 124 and / or image capture device 126 may be 1.25, 1.5, 1.75, or 2 times or more the line scan rate of image capture device 122.

[0058] In some embodiments, the image capture devices 122, 124, and 126 may be asymmetric. That is, they may include cameras with different fields of view (FOV) and focal lengths. The fields of view of the image capture devices 122, 124, and 126 may, for example, include any desired area relative to an environment of the vehicle 200. In some embodiments, one or more of the image capture devices 122, 124, and 126 may be configured to capture image data from an environment in front of the vehicle 200, behind the vehicle 200, on the sides of the vehicle 200, or combinations thereof.

[0059] Furthermore, the focal length associated with each image capture device 122, 124, and / or 126 may be selectable (e.g., by incorporating appropriate lenses, etc.) such that each device captures images of objects within a desired range of distances relative to the vehicle 200. For example, in some embodiments, the image capture devices 122, 124, and 126 may capture images of close-up objects within a few meters of the vehicle. The image capture devices 122, 124, and 126 may also be configured to capture images of objects at distances farther from the vehicle (e.g., 25 m, 50 m, 100 m, 150 m, or more). Furthermore, the focal lengths of the image capture devices 122, 124 and 126 may be selected such that an image capture device (e.g., image capture device 122) can acquire images of objects relatively close to the vehicle (e.g.,within 10 m or within 20 m), while the other image capture devices (e.g., image capture devices 124 and 126) can capture images of objects further away (e.g., greater than 20 m, 50 m, 100 m, 150 m, etc.) from the vehicle 200.

[0060] According to some embodiments, the FOV of one or more image capture devices 122, 124, and 126 may have a wide angle. For example, it may be advantageous to have an FOV of 140 degrees, particularly for image capture devices 122, 124, and 126 that may be used to capture images of the area proximate to vehicle 200. For example, image capture device 122 may be used to capture images of the area to the right or left of vehicle 200, and in such embodiments, it may be desirable for image capture device 122 to have a wide FOV (e.g., at least 140 degrees).

[0061] The field of view associated with each of the image capture devices 122, 124, and 126 may depend on the respective focal lengths. For example, as the focal length increases, the corresponding field of view decreases.

[0062] The image capture devices 122, 124, and 126 may be configured to have any suitable fields of view. In one particular example, the image capture device 122 may have a horizontal FOV of 46 degrees, the image capture device 124 may have a horizontal FOV of 23 degrees, and the image capture device 126 may have a horizontal FOV between 23 and 46 degrees. In another case, the image capture device 122 may have a horizontal FOV of 52 degrees, the image capture device 124 may have a horizontal FOV of 26 degrees, and the image capture device 126 may have a horizontal FOV between 26 and 52 degrees. In some embodiments, a ratio of the FOV of the image capture device 122 to the FOVs of the image capture device 124 and / or the image capture device 126 may vary from 1.5 to 2.0. In other embodiments, this ratio may vary between 1.25 and 2.25.

[0063] The system 100 may be configured such that a field of view of the image capture device 122 at least partially or completely overlaps with a field of view of the image capture device 124 and / or the image capture device 126. In some embodiments, the system 100 may be configured such that the fields of view of the image capture devices 124 and 126, for example, fall within (e.g., are narrower than) the field of view of the image capture device 122 and share a common center with it. In other embodiments, the image capture devices 122, 124, and 126 may capture adjacent FOVs or have a partial overlap in their FOVs. In some embodiments, the fields of view of the image capture devices 122, 124, and 126 may be aligned such that a center of the narrower FOV image capture devices 124 and / or 126 may be located in a lower half of the field of view of the wider FOV device 122.

[0064] Fig. Figure 2F is a schematic representation of exemplary vehicle control systems in accordance with the disclosed embodiments. As shown in Fig. 2F, the vehicle 200 may include a throttle system 220, a braking system 230, and a steering system 240. The system 100 may provide inputs (e.g., control signals) to one or more of a throttle system 220, a braking system 230, and a steering system 240 via one or more data connections (e.g., any wired and / or wireless connection or connections for transmitting data). For example, based on an analysis of images captured by the image capture devices 122, 124, and / or 126, the system 100 may provide control signals to one or more of a throttle system 220, a braking system 230, and a steering system 240 to navigate the vehicle 200 (e.g., by initiating acceleration, a turn, a lane shift, etc.).Further, system 100 may receive inputs from one or more of a throttle system 220, a braking system 230, and a steering system 24 indicative of operating conditions of vehicle 200 (e.g., speed, whether vehicle 200 is braking and / or turning, etc.). Further details are provided in connection with the following. Fig. 4-7 provided.

[0065] As in Fig. 3A, the vehicle 200 may also include a user interface 170 for interacting with a driver or a passenger of the vehicle 200. For example, in a vehicle application, the user interface 170 may include a touchscreen 320, knobs 330, buttons 340, and a microphone 350. A driver or passenger of the vehicle 200 may also use handles (e.g., located on or near the steering column of the vehicle 200, including, for example, turn signal handles), buttons (e.g., located on the steering wheel of the vehicle 200), and the like to interact with the system 100. In some embodiments, the microphone 350 may be positioned adjacent to a rearview mirror 310. Similarly, in some embodiments, the image capture device 122 may be located near the rearview mirror 310. In some embodiments, the user interface 170 may also include one or more speakers 360 (e.g.,speakers of a vehicle audio system). For example, the system 100 may provide various notifications (e.g., alerts) through the speakers 360.

[0066] The Fig. 3B-3D are illustrations of an exemplary camera mount 370 configured to be positioned behind a rearview mirror (e.g., rearview mirror 310) and against a vehicle windshield, in accordance with the disclosed embodiments. As shown in Fig. 3B, the camera mount 370 may include image capture devices 122, 124, and 126. The image capture devices 124 and 126 may be positioned behind an anti-glare shield 380, which may be flush with the vehicle windshield and may include a composition of film and / or anti-reflective materials. For example, the anti-glare shield 380 may be positioned such that the shield aligns with a vehicle windshield having a matching slope. In some embodiments, each of the image capture devices 122, 124, and 126 may be positioned behind the anti-glare shield 380, such as in Fig. 3D. The disclosed embodiments are not limited to any particular configuration of the image capture devices 122, 124, and 126, the camera mount 370, and the glare shield 380. Fig. 3C is an illustration of the camera mount 370 shown in Fig. 3B is shown from a front perspective.

[0067] As will be apparent to one of ordinary skill in the art having the benefit of this disclosure, numerous variations and / or modifications may be made to the above-disclosed embodiments. For example, not all components are essential to the operation of system 100. Further, any component may be located in any suitable part of system 100, and the components may be rearranged into a variety of configurations while providing the functionality of the disclosed embodiments. Therefore, the above configurations are examples, and regardless of the configurations discussed above, system 100 may provide a wide range of functionality to analyze the surroundings of vehicle 200 and navigate vehicle 200 in response to the analysis.

[0068] As discussed in more detail below, and in accordance with various disclosed embodiments, system 100 may provide a variety of features related to autonomous driving and / or driver assistance technology. For example, system 100 may analyze image data, position data (e.g., GPS location information), map data, speed data, and / or data from sensors included in vehicle 200. System 100 may collect the data for analysis, for example, from image capture unit 120, position sensor 130, and other sensors. Further, system 100 may analyze the collected data to determine whether or not vehicle 200 should perform a particular action and then automatically perform the particular action without human intervention.For example, when the vehicle 200 is navigating without human intervention, the system 100 may automatically control the braking, acceleration, and / or steering of the vehicle 200 (e.g., by sending control signals to one or more of a throttle system 220, a braking system 230, and a steering system 240). Further, the system 100 may analyze the collected data and issue warnings and / or alarms to vehicle occupants based on the analysis of the collected data. Additional details regarding the various embodiments provided by the system 100 are provided below. Forward-facing multi-imaging system

[0069] As discussed above, the system 100 may provide driver assistance functionality using a multi-camera system. The multi-camera system may use one or more cameras directed in the forward direction of a vehicle. In other embodiments, the multi-camera system may include one or more cameras facing the side of a vehicle or the rear of the vehicle. For example, in one embodiment, the system 100 may use a two-camera imaging system, where a first camera and a second camera (e.g., image capture devices 122 and 124) may be positioned on the front and / or sides of a vehicle (e.g., vehicle 200). Other camera configurations are consistent with the disclosed embodiments, and the configurations described herein are examples. For example, the system 100 may include a configuration with any number of cameras (e.g.,one, two, three, four, five, six, seven, eight, etc.). Furthermore, system 100 may include "clusters" of cameras. For example, a cluster of cameras (including any suitable number of cameras, e.g., one, four, eight, etc.) may be facing forward relative to a vehicle or may be facing in any other direction (e.g., reward-facing, side-facing, at an angle, etc.). Accordingly, system 100 may include multiple clusters of cameras, with each cluster facing a particular direction to capture images from a particular area of ​​the vehicle's surroundings.

[0070] The first camera may have a field of view that is larger than, smaller than, or partially overlapping with the field of view of the second camera. Additionally, the first camera may be connected to a first image processor to perform monocular image analysis of images provided by the first camera, and the second camera may be connected to a second image processor to perform monocular image analysis of images provided by the second camera. The outputs (e.g., processed information) of the first and second image processors may be combined. In some embodiments, the second image processor may receive images from both the first camera and the second camera to perform stereo analysis. In another embodiment, system 100 may use a three-camera imaging system, with each of the cameras having a different field of view.Such a system can therefore make decisions based on information derived from objects located at varying distances both to the front and to the sides of the vehicle. References to monocular image analysis may refer to cases where image analysis is performed based on images acquired from a single viewpoint (e.g., from a single camera). Stereo image analysis may refer to cases where image analysis is performed based on two or more images acquired with one or more variations of an image acquisition parameter. For example, acquired images suitable for performing stereo image analysis may include images acquired from two or more different positions, from different fields of view, using different focal lengths, along with parallax information, etc.

[0071] For example, in one embodiment, system 100 may implement a three-camera configuration using image capture devices 122-126. In such a configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees or other values ​​selected from a range of about 20 to 45 degrees, etc.), image capture device 124 may provide a wide field of view (e.g., 150 degrees or other values ​​selected from a range of about 100 to about 180 degrees), and image capture device 126 may provide an intermediate field of view (e.g., 46 degrees or other values ​​selected from a range of about 35 to about 60 degrees). In some embodiments, image capture device 126 may function as the main or primary camera.The image capture devices 122-126 may be positioned behind the rearview mirror 310 and positioned substantially side by side (e.g., 6 cm apart). Furthermore, in some embodiments, as discussed above, one or more of the image capture devices 122-126 may be mounted behind the glare shield 380, which is flush with the windshield of the vehicle 200. Such shielding may serve to minimize the impact of reflections from inside the vehicle on the image capture devices 122-126.

[0072] In another embodiment, as described above in connection with the Fig. 3B and Fig. 3C, the wide field of view camera (e.g., image capture device 124 in the above example) may be mounted lower than the narrow field of view camera and main field of view camera (e.g., image capture devices 122 and 126 in the above example). This configuration may provide a clear line of sight from the wide field of view camera. To reduce reflections, the cameras may be mounted near the windshield of the vehicle 200 and include polarizers on the cameras to attenuate reflected light.

[0073] A three-camera system may provide certain performance characteristics. For example, some embodiments may include the ability to validate the detection of objects by one camera based on detection results from another camera. In the three-camera configuration discussed above, for example, processing unit 110 may include three processing devices (e.g., three EyeQ series of processor chips, as discussed above), with each processing device dedicated to processing images captured by one or more of image capture devices 122-126.

[0074] In a three-camera system, a first processing device may receive images from both the main camera and the narrow FOV camera and perform image processing of the narrow FOV camera to detect, for example, other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Furthermore, the first processing device may calculate a pixel disparity between the images from the main camera and the narrow FOV camera and create a 3D reconstruction of the surroundings of the vehicle 200. The first processing device may then combine the 3D reconstruction with 3D map data or with 3D information calculated based on information from another camera.

[0075] The second processing device can receive images from the main camera and perform image processing to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the second processing device can calculate a camera displacement and, based on the displacement, calculate a disparity of pixels between consecutive images and create a 3D reconstruction of the scene (e.g., a structure from motion). The second processing device can send the structure from a motion-based 3D reconstruction to the first processing device to be combined with the stereo 3D images.

[0076] The third processing device may receive images from the wide FOV camera and process the images to detect vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. The third processing device may further execute additional processing instructions to analyze images to identify moving objects in the image, such as vehicles changing lanes, pedestrians, etc.

[0077] In some embodiments, streams of image-based information that are independently captured and processed may provide a way to provide redundancy in the system. Such redundancy may, for example, involve using a first image capture device and the images processed by that device to validate and / or supplement information obtained by capturing and processing image information from at least one second image capture device.

[0078] In some embodiments, system 100 may use two image capture devices (e.g., image capture devices 122 and 124) to provide navigation assistance for vehicle 200 and a third image capture device (e.g., image capture device 126) to provide redundancy and validate the analysis of data received from the other two image capture devices. For example, in such a configuration, image capture devices 122 and 124 may provide images for stereo analysis by system 100 for navigating vehicle 200, while image capture device 126 may provide images for monocular analysis by system 100 to provide redundancy and validation of information obtained based on images acquired by image capture device 122 and / or image capture device 124.That is, the image capture device 126 (and a corresponding processing device) may be considered a redundant subsystem for providing verification of the analysis derived from the image capture devices 122 and 124 (e.g., to provide an automatic emergency braking (AEB) system). Furthermore, in some embodiments, the redundancy and validation of the received data may be supplemented based on information received from one or more sensors (e.g., radar, lidar, acoustic sensors, information received from one or more transceivers external to a vehicle, etc.).

[0079] One skilled in the art will recognize that the above camera configurations, camera placements, number of cameras, camera locations, etc., are merely examples. These components and others described with respect to the overall system may be assembled and used in a variety of different configurations without departing from the scope of the disclosed embodiments. Further details regarding the use of a multi-camera system to provide driver assistance and / or autonomous vehicle functionality follow below.

[0080] Fig. Figure 4 is an exemplary functional block diagram of memory 140 and / or 150, which may be stored / programmed with instructions for performing one or more operations, in accordance with the disclosed embodiments. Although the following refers to memory 140, one of ordinary skill in the art will recognize that instructions may be stored in memory 140 and / or 150.

[0081] As in Fig. 4, memory 140 may store a monocular image analysis module 402, a stereo image analysis module 404, a velocity and acceleration module 406, and a navigation response module 408. The disclosed embodiments are not limited to any particular configuration of memory 140. Further, application processor 180 and / or image processor 190 may execute the instructions stored in any of modules 402-408 included in memory 140. One of ordinary skill in the art will understand that references in the following discussions to processing unit 110 may refer individually or collectively to application processor 180 and image processor 190. Accordingly, steps of any of the following processes may be performed by one or more processing devices.

[0082] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform a monocular image analysis of a set of images acquired by one of the image acquisition devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from a set of images with additional sensory information (e.g., information from radar) to perform the monocular image analysis. As described in connection with the following Fig. 5A-5D, the monocular image analysis module 402 may include instructions for detecting a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other feature associated with an environment of a vehicle. Based on the analysis, the system 100 (e.g., via the processing unit 110) may initiate one or more navigational responses within the vehicle 200, such as a turn, a lane shift, a change in acceleration, and the like, as discussed below in connection with the navigational response module 408.

[0083] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform a monocular image analysis of a set of images acquired by one of the image acquisition devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from a set of images with additional sensory information (e.g., information from radar, lidar, etc.) to perform the monocular image analysis. As described in connection with the following Fig. 5A-5D, the monocular image analysis module 402 may include instructions for detecting a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other feature associated with an environment of a vehicle. Based on the analysis, the system 100 (e.g., via the processing unit 110) may initiate one or more navigational responses in the vehicle 200, such as a turn, a lane shift, a change in acceleration, and the like, as discussed below in connection with determining a navigational response.

[0084] In one embodiment, the stereo image analysis module 404 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform a stereo image analysis of a first and second set of images acquired by a combination of image acquisition devices selected from any of the image acquisition devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the first and second sets of images with additional sensory information (e.g., information from radar) to perform the stereo image analysis.For example, the stereo image analysis module 404 may include instructions for performing a stereo image analysis based on a first set of images acquired by the image capture device 124 and a second set of images acquired by the image capture device 126. As described in connection with the following. Fig. 6, the stereo image analysis module 404 may include instructions for detecting a set of features within the first and second sets of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and the like. Based on the analysis, the processing unit 110 may initiate one or more navigational responses within the vehicle 200, such as a turn, a lane shift, a change in acceleration, and the like, as discussed below in connection with the navigational response module 408. Furthermore, in some embodiments, the stereo image analysis module 404 may implement techniques associated with a trained system (such as a neural network or a deep neural network) or an untrained system.

[0085] In one embodiment, the speed and acceleration module 406 may store software configured to analyze data received from one or more computing devices and electromechanical devices within the vehicle 200 that are configured to cause a change in the speed and / or acceleration of the vehicle 200. For example, the processing unit 110 may execute instructions associated with the speed and acceleration module 406 to calculate a target speed for the vehicle 200 based on data derived from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404.Such data may include, for example, a target position, speed, and / or acceleration, the position and / or speed of the vehicle 200 relative to a nearby vehicle, pedestrian, or road object, position information for the vehicle 200 relative to lane markings of the road, and the like. Additionally, the processing unit 110 may calculate a target speed for the vehicle 200 based on sensory inputs (e.g., information from radar) and inputs from other systems of the vehicle 200, such as the throttle system 220, the braking system 230, and / or the steering system 240 of the vehicle 200.Based on the calculated target speed, the processing unit 110 may transmit electronic signals to the throttle system 220, the braking system 230, and / or the steering system 240 of the vehicle 200 to trigger a change in speed and / or acceleration, for example, by physically depressing the brake or releasing the accelerator of the vehicle 200.

[0086] In one embodiment, the navigation response module 408 may store software executable by the processing unit 110 to determine a desired navigation response based on data derived from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include position and velocity information associated with nearby vehicles, pedestrians, and road objects, target position information for the vehicle 200, and the like.Additionally, in some embodiments, the navigation response may be based (partially or entirely) on map data, a predetermined position of the vehicle 200, and / or a relative velocity or acceleration between the vehicle 200 and one or more objects detected by the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. The navigation response module 408 may also determine a desired navigation response based on sensory inputs (e.g., information from radar) and inputs from other systems of the vehicle 200, such as the throttle system 220, the braking system 230, and the steering system 240 of the vehicle 200.Based on the desired navigation response, processing unit 110 may transmit electronic signals to the throttle system 220, the braking system 230, and the steering system 240 of the vehicle 200 to trigger a desired navigation response, for example, by turning the steering wheel of the vehicle 200 to achieve a rotation of a predetermined angle. In some embodiments, processing unit 110 may use the output of the navigation response module 408 (e.g., the desired navigation response) as an input to execute the speed and acceleration module 406 to calculate a change in speed of the vehicle 200.

[0087] Furthermore, each of the modules disclosed herein (e.g., modules 402, 404, and 406) may implement techniques associated with a trained system (such as a neural network or a deep neural network) or an untrained system.

[0088] Fig. 5A is a flowchart illustrating an exemplary process 500A for initiating one or more navigation responses based on monocular image analysis, in accordance with the disclosed embodiments. At step 510, the processing unit 110 may receive a plurality of images via the data interface 128 between the processing unit 110 and the image capture unit 120. For example, a camera included in the image capture unit 120 (such as the image capture device 122 with the field of view 202) may capture a plurality of images of an area in front of the vehicle 200 (or, for example, to the sides or rear of a vehicle) and transmit them to the processing unit 110 via a data connection (e.g., digital, wired, USB, wireless, Bluetooth, etc.).The processing unit 110 may execute the monocular image analysis module 402 to analyze the plurality of images at step 520, as described below in connection with FIGS. Fig. 5B-5D. By performing the analysis, the processing unit 110 can recognize a set of features within the set of images, such as lane markings, vehicles, pedestrians, street signs, highway exit ramps, traffic lights, and the like.

[0089] The processing unit 110 may also execute the monocular image analysis module 402 to detect various road hazards at step 520, such as, for example, pieces of a truck tire, fallen road signs, loose cargo, small animals, and the like. Road hazards can vary in texture, shape, size, and color, which can make detection of such hazards more difficult. In some embodiments, the processing unit 110 may execute the monocular image analysis module 402 to perform multi-frame analysis on the plurality of images to detect road hazards. For example, the processing unit 110 may estimate camera motion between consecutive frames and calculate the disparities in pixels between the frames to create a 3D map of the road.The processing unit 110 can then use the 3D map to detect the road surface as well as the hazards existing above the road surface.

[0090] At step 530, the processing unit 110 may execute the navigation response module 408 to determine one or more navigation responses in the vehicle 200 based on the analysis performed at step 520 and the techniques described above in connection with Fig. 4. Navigation responses may include, for example, a turn, a lane shift, a change in acceleration, and the like. In some embodiments, processing unit 110 may use data derived from the execution of speed and acceleration module 406 to prompt the one or more navigation responses. Additionally, multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof. For example, processing unit 110 may cause vehicle 200 to change lanes and then accelerate, for example, by sequentially transmitting control signals to steering system 240 and throttle system 220 of vehicle 200.Alternatively, the processing unit 110 may cause the vehicle 200 to brake while simultaneously changing lanes, for example, by simultaneously transmitting control signals to the braking system 230 and the steering system 240 of the vehicle 200.

[0091] Fig. 5B is a flowchart illustrating an exemplary process 500B for detecting one or more vehicles and / or pedestrians in a set of images, in accordance with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500B. At step 540, processing unit 110 may determine a set of candidate objects representing possible vehicles and / or pedestrians. For example, processing unit 110 may scan one or more images, compare the images to one or more predetermined patterns, and identify possible locations within each image that may contain objects of interest (e.g., vehicles, pedestrians, or portions thereof). The predetermined patterns may be configured to achieve a high false hit rate and a low false miss rate.For example, processing unit 110 may use a low threshold of similarity to predetermined patterns to identify candidate objects as possible vehicles or pedestrians. This may allow processing unit 110 to reduce the likelihood that a candidate object representing a vehicle or pedestrian is missing (e.g., not identified).

[0092] At step 542, the processing unit 110 may filter the set of candidate objects to exclude certain candidates (e.g., irrelevant or less relevant objects) based on classification criteria. Such criteria may be derived from various properties associated with object types stored in a database (e.g., a database stored in memory 140). Properties may include object shape, dimensions, texture, position (e.g., relative to the vehicle 200), and the like. Thus, the processing unit 110 may use one or more sets of criteria to reject false candidates from the set of candidate objects.

[0093] At step 544, processing unit 110 may analyze multiple frames of images to determine whether objects in the set of candidate objects represent vehicles and / or pedestrians. For example, processing unit 110 may track a detected candidate object across consecutive frames and accumulate frame data associated with the detected object (e.g., size, position relative to vehicle 200, etc.). Additionally, processing unit 110 may estimate parameters for the detected object and compare the frame position data of the object to a predicted position.

[0094] At step 546, processing unit 110 may create a set of measurements for the detected objects. Such measurements may include, for example, position, velocity, and acceleration values ​​(relative to vehicle 200) associated with the detected objects. In some embodiments, processing unit 110 may create the measurements based on estimation techniques using a set of time-based observations, such as Kalman filters or linear quadratic estimation (LQE), and / or based on available modeling data for different object types (e.g., cars, trucks, pedestrians, bicycles, road signs, etc.). The Kalman filters may be based on a measurement of the scale of an object, where the scale measurement is proportional to a time to collision (e.g., the amount of time it takes vehicle 200 to reach the object).Thus, by performing steps 540-546, processing unit 110 may identify vehicles and pedestrians that appear within the set of captured images and derive information (e.g., position, speed, size) associated with the vehicles and pedestrians. Based on the identification and the derived information, processing unit 110 may initiate one or more navigational responses in vehicle 200, as described above in connection with [ ]. Fig. 5A.

[0095] At step 548, the processing unit 110 may perform optical flow analysis of one or more images to reduce the likelihood of detecting a "false hit" and missing a candidate object representing a vehicle or pedestrian. Optical flow analysis may refer, for example, to analyzing motion patterns relative to the vehicle 200 in one or more images that are associated with other vehicles and pedestrians and that are distinct from road surface motion. The processing unit 110 may calculate the motion of candidate objects by observing the various positions of the objects across multiple frames captured at different times. The processing unit 110 may use the position and time values ​​as inputs to mathematical models for calculating the motion of the candidate objects.Thus, optical flow analysis may provide another method for detecting vehicles and pedestrians near vehicle 200. Processing unit 110 may perform optical flow analysis in combination with steps 540-546 to provide redundancy for detecting vehicles and pedestrians and increase the reliability of system 100.

[0096] Fig. 5C is a flowchart illustrating an exemplary process 500C for detecting road markings and / or lane geometry information in a set of images, in accordance with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500C. At step 550, processing unit 110 may detect a set of objects by scanning one or more images. To detect segments of lane markings, lane geometry information, and other relevant road markings, processing unit 110 may filter the set of objects to exclude those determined to be irrelevant (e.g., small potholes, small rocks, etc.). At step 552, processing unit 110 may group together the segments detected in step 550 that belong to the same road marking or lane marking.Based on the grouping, the processing unit 110 may develop a model to represent the detected segments, such as a mathematical model.

[0097] At step 554, processing unit 110 may construct a set of measurements associated with the detected segments. In some embodiments, processing unit 110 may create a projection of the detected segments from the image plane to the real plane. The projection may be characterized using a third-order polynomial having coefficients corresponding to physical properties, such as the position, slope, curvature, and curvature derivative of the detected road. In generating the projection, processing unit 110 may consider changes in the road surface, as well as pitch and roll rates associated with vehicle 200. Additionally, processing unit 110 may model the road elevation by analyzing position and motion cues present on the road surface.Further, the processing unit 110 may estimate the pitch and roll rates associated with the vehicle 200 by tracking a set of feature points in the one or more images.

[0098] At step 556, the processing unit 110 may perform a multi-frame analysis, for example, by tracking the detected segments across successive frames and accumulating frame data associated with detected segments. As the processing unit 110 performs a multi-frame analysis, the set of measurements created at step 554 may become more reliable and associated with an increasingly higher level of confidence. Thus, by performing steps 550-556, the processing unit 110 may identify road markings that appear within the set of acquired images and derive lane geometry information. Based on the identification and the derived information, the processing unit 110 may initiate one or more navigation responses in the vehicle 200, as described above in connection with Fig. 5A.

[0099] At step 558, processing unit 110 may consider additional information sources to further develop a safety model for vehicle 200 in the context of its environment. Processing unit 110 may use the safety model to define a context in which system 100 can safely perform autonomous control of vehicle 200. To develop the safety model, in some embodiments, processing unit 110 may consider the position and movement of other vehicles, detected road edges and obstacles, and / or general road shape descriptions extracted from map data (such as data from map database 160). By considering additional information sources, processing unit 110 may provide redundancy for detecting road markings and lane geometry and increase the reliability of system 100.

[0100] Fig. 5D is a flowchart illustrating an exemplary process 500D for detecting traffic lights in a set of images, in accordance with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500D. At step 560, processing unit 110 may scan the set of images and identify objects that appear at locations in the images that are likely to contain traffic lights. For example, processing unit 110 may filter the identified objects to create a set of candidate objects, eliminating those objects from being likely to correspond to traffic lights. The filtering may be performed based on various properties associated with traffic lights, such as shape, dimensions, texture, position (e.g., relative to vehicle 200), and the like.Such properties may be based on multiple examples of traffic lights and traffic control signals and stored in a database. In some embodiments, processing unit 110 may perform a multi-frame analysis on the set of candidate objects that reflect possible traffic lights. For example, processing unit 110 may track the candidate objects over consecutive frames, estimate the real-world position of the candidate objects, and filter out those objects that are moving (which are unlikely to be traffic lights). In some embodiments, processing unit 110 may perform a color analysis on the candidate objects and identify the relative position of the detected colors that appear within possible traffic lights.

[0101] At step 562, processing unit 110 may analyze the geometry of an intersection. The analysis may be based on any combination of: (i) the number of lanes detected on either side of vehicle 200, (ii) markers (such as arrow markers) detected on the road, and (iii) descriptions of the intersection extracted from map data (such as data from map database 160). Processing unit 110 may perform the analysis using information derived from the execution of monocular analysis module 402. Additionally, processing unit 110 may determine a correspondence between the traffic lights detected at step 560 and the lanes appearing near vehicle 200.

[0102] As the vehicle 200 approaches the intersection, the processing unit 110 may update the confidence level associated with the analyzed intersection geometry and the detected traffic lights at step 564. For example, the number of traffic lights estimated to appear at the intersection compared to the number actually appearing at the intersection may impact the confidence level. Thus, based on the confidence level, the processing unit 110 may delegate control to the driver of the vehicle 200 to improve safety conditions. By performing steps 560-564, the processing unit 110 may identify traffic lights that appear within the set of captured images and analyze intersection geometry information. Based on the identification and analysis, the processing unit 110 may initiate one or more navigation responses in the vehicle 200, as described above in connection with Fig. 5A.

[0103] Fig. 5E is a flowchart illustrating an exemplary process 500E for initiating one or more navigation responses in vehicle 200 based on a vehicle path, in accordance with the disclosed embodiments. At step 570, processing unit 110 may construct an initial vehicle path associated with vehicle 200. The vehicle path may be represented using a set of points expressed in coordinates (x, z) and the distance d ibetween two points in the set of points may range from 1 to 5 meters. In one embodiment, the processing unit 110 may construct the initial vehicle path using two polynomials, such as left and right road polynomials. The processing unit 110 may calculate the geometric midpoint between the two polynomials and offset each point included in the resulting vehicle path by a predetermined offset (e.g., a smart lane offset), if present (a zero offset may correspond to traveling in the center of a lane). The offset may be in a direction perpendicular to a segment between any two points in the vehicle path. In another embodiment, the processing unit 110 may use a polynomial and an estimated lane width to offset each point of the vehicle path by half the estimated lane width plus a predetermined offset (e.g.,an offset of an intelligent lane).

[0104] At step 572, the processing unit 110 may update the vehicle path constructed at step 570. The processing unit 110 may reconstruct the vehicle path constructed at step 570 using a higher resolution so that the distance d k between two points in the set of points representing the vehicle path is less than the distance d described above i For example, the distance d k fall within the range of 0.1 to 0.3 meters. The processing unit 110 may reconstruct the vehicle path using a parabolic spline algorithm, which may provide a cumulative distance vector S corresponding to the total length of the vehicle path (i.e., based on the set of points representing the vehicle path).

[0105] At step 574, the processing unit 110 may determine a lookahead point (expressed in coordinates as (x l , e.g. l)) based on the updated vehicle path constructed at step 572. The processing unit 110 may extract the look-ahead point from the cumulative range vector S, and the look-ahead point may be associated with a look-ahead distance and look-ahead time. The look-ahead distance, which may have a lower bound in the range of 10 to 20 meters, may be calculated as the product of the speed of the vehicle 200 and the look-ahead time. For example, as the speed of the vehicle 200 decreases, the look-ahead distance may also decrease (e.g., until it reaches the lower bound). The look-ahead time, which may range from 0.5 to 1.5 seconds, may be inversely proportional to the gain of one or more control loops associated with causing a navigation response in the vehicle 200, such as the heading error tracking control loop.For example, the gain of the heading error tracking control loop may depend on the bandwidth of a yaw rate control loop, a steering actuator control loop, the lateral dynamics of the vehicle, and so on. Thus, the higher the gain of the heading error tracking control loop, the shorter the look-ahead time.

[0106] At step 576, the processing unit 110 may determine a heading error and a yaw rate command based on the look-ahead point determined at step 574. The processing unit 110 may determine the heading error by calculating the arctangent of the look-ahead point, e.g., arctan (x l / z l). Processing unit 110 may determine the yaw rate command as the product of the heading error and a high control gain. The high control gain may be equal to: (2 / look-ahead time) if the look-ahead distance is not at the lower limit. Otherwise, the high control gain may be equal to: (2 * vehicle speed 200 / look-ahead distance).

[0107] Fig. 5F is a flowchart illustrating an exemplary process 500F for determining whether a leading vehicle is changing lanes, in accordance with the disclosed embodiments. At step 580, the processing unit 110 may determine navigation information associated with a leading vehicle (e.g., a vehicle traveling in front of the vehicle 200). For example, the processing unit 110 may determine the position, speed (e.g., direction and speed), and / or acceleration of the leading vehicle using the methods described above in connection with the Fig. 5A and Fig. 5B. The processing unit 110 may also determine one or more road polynomials, a look-ahead point (associated with the vehicle 200), and / or a snail trail (e.g., a set of points describing a path taken by the preceding vehicle) using the techniques described above in connection with Fig. 5E described techniques.

[0108] At step 582, processing unit 110 may analyze the navigation information determined at step 580. In one embodiment, processing unit 110 may calculate the distance between a snail trail and a road polynomial (e.g., along the trail). If the variance of this distance along the trail exceeds a predetermined threshold (e.g., 0.1 to 0.2 meters on a straight road, 0.3 to 0.4 meters on a moderately curved road, and 0.5 to 0.6 meters on a road with sharp curves), processing unit 110 may determine that the leading vehicle is likely changing lanes. In the event that multiple vehicles are detected traveling in front of vehicle 200, processing unit 110 may compare the snail trails associated with each vehicle.Based on the comparison, processing unit 110 may determine that a vehicle whose snail trail does not match the snail trails of other vehicles is likely to change lanes. Processing unit 110 may additionally compare the curvature of the snail trail (associated with the leading vehicle) with the expected curvature of the road segment in which the leading vehicle is traveling. The expected curvature may be extracted from map data (e.g., data from map database 160), road polynomials, snail trails of other vehicles, prior knowledge of the road, and the like. If the difference in the curvature of the snail trail and the expected curvature of the road segment exceeds a predetermined threshold, processing unit 110 may determine that the leading vehicle is likely to change lanes.

[0109] In another embodiment, processing unit 110 may compare the current position of the preceding vehicle with the look-ahead point (associated with vehicle 200) over a specific period of time (e.g., 0.5 to 1.5 seconds). If the distance between the current position of the preceding vehicle and the look-ahead point varies during the specific period of time and the cumulative sum of the variation exceeds a predetermined threshold (e.g., 0.3 to 0.4 meters on a straight road, 0.7 to 0.8 meters on a moderately curved road, and 1.3 to 1.7 meters on a road with sharp curves), processing unit 110 may determine that the preceding vehicle is likely changing lanes.In another embodiment, the processing unit 110 may analyze the geometry of the snail trail by comparing the lateral distance traveled along the trail with the expected curvature of the snail trail. The expected radius of curvature may be determined according to the calculation: (δ z 2 + δ x 2 ) / 2 / (δ x ), where δ x represents the lateral distance traveled and δ zrepresents the traveled longitudinal distance. If the difference between the traveled lateral distance and the expected curvature exceeds a predetermined threshold (e.g., 500 to 700 meters), the processing unit 110 may determine that the leading vehicle is likely to change lanes. In another embodiment, the processing unit 110 may analyze the position of the leading vehicle. If the position of the leading vehicle obscures a road polynomial (e.g., the leading vehicle is overlaid on the road polynomial), then the processing unit 110 may determine that the leading vehicle is likely to change lanes.In case the position of the preceding vehicle is such that another vehicle is detected in front of the preceding vehicle and the snail trails of the two vehicles are not parallel, the processing unit 110 may determine that the (nearer) preceding vehicle is likely to change lanes.

[0110] At step 584, processing unit 110 may determine whether or not the leading vehicle 200 is changing lanes based on the analysis performed at step 582. For example, processing unit 110 may make the determination based on a weighted average of the individual analyses performed at step 582. Under such a scheme, for example, a decision by processing unit 110 that the leading vehicle is likely to change lanes may be assigned a value of "1" (and "0" to represent a determination that the leading vehicle is not likely to change lanes) based on a particular type of analysis. Different analyses performed at step 582 may be assigned different weights, and the disclosed embodiments are not limited to any particular combination of analyses and weights.Furthermore, in some embodiments, the analysis may use a trained system (e.g., a machine learning or deep learning system) that may, for example, estimate a future path ahead of a current location of a vehicle based on an image captured at the current location.

[0111] Fig. 6 is a flowchart illustrating an exemplary process 600 for initiating one or more navigation responses based on stereo image analysis, in accordance with the disclosed embodiments. At step 610, the processing unit 110 may receive a first and second plurality of images via the data interface 128. For example, cameras included in the image capture unit 120 (such as image capture devices 122 and 124 with fields of view 202 and 204) may capture a first and second plurality of images of an area ahead of the vehicle 200 and transmit them to the processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, the processing unit 110 may receive the first and second plurality of images via two or more data interfaces.The disclosed embodiments are not limited to any particular data interface configurations or protocols.

[0112] At step 620, the processing unit 110 may execute the stereo image analysis module 404 to perform a stereo image analysis of the first and second plurality of images to create a 3D map of the road ahead of the vehicle and to detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, and the like. The stereo image analysis may be performed in a manner consistent with the methods described above in connection with the Fig. 5A-5D. For example, the processing unit 110 may execute the stereo image analysis module 404 to detect candidate objects (e.g., vehicles, pedestrians, road markings, traffic lights, road hazards, etc.) within the first and second plurality of images, filter out a subset of the candidate objects based on various criteria, and perform a multi-frame analysis, taking measurements, and determining a confidence level for the remaining candidate objects.In performing the above steps, the processing unit 110 may consider information from both the first and second pluralities of images, as well as information from one set of images alone. For example, the processing unit 110 may analyze the differences in pixel-level data (or other data subsets from the two streams of acquired images) for a candidate object that appears in both the first and second pluralities of images. As another example, the processing unit 110 may estimate a position and / or velocity of a candidate object (e.g., relative to the vehicle 200) by observing that the object appears in one of the plurality of images but not the other, or relative to other differences that may exist relative to objects that appear when the two image streams are combined.For example, position, velocity, and / or acceleration relative to the vehicle 200 may be determined based on trajectories, positions, motion characteristics, etc. of features associated with an object appearing in one or both of the image streams.

[0113] At step 630, the processing unit 110 may execute the navigation response module 408 to determine one or more navigation responses in the vehicle 200 based on the analysis performed at step 620 and the techniques as described above in connection with Fig. 4. Navigational responses may include, for example, a turn, a lane shift, a change in acceleration, a change in speed, braking, and the like. In some embodiments, processing unit 110 may use data derived from the execution of speed and acceleration module 406 to prompt the one or more navigational responses. Additionally, multiple navigational responses may occur simultaneously, sequentially, or in any combination thereof.

[0114] Fig. 7 is a flowchart illustrating an exemplary process 700 for initiating one or more navigation responses based on an analysis of three sets of images, in accordance with disclosed embodiments. At step 710, the processing unit 110 may receive a first, second, and third plurality of images via the data interface 128. For example, cameras included in the image capture unit 120 (such as image capture devices 122, 124, and 126 with fields of view 202, 204, and 206) may capture a first, second, and third plurality of images of an area in front of and / or to the side of the vehicle 200 and transmit them to the processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, the processing unit 110 may receive the first, second, and third plurality of images via three or more data interfaces.For example, each of the image capture devices 122, 124, 126 may have an associated data interface for communicating data to the processing unit 110. The disclosed embodiments are not limited to any particular data interface configurations or protocols.

[0115] At step 720, the processing unit 110 may analyze the first, second, and third plurality of images to identify features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, and the like. The analysis may be performed in a manner consistent with the methods described above in connection with the Fig. 5A-5D and 6. For example, the processing unit 110 may perform monocular image analysis (e.g., via execution of the monocular image analysis module 402 and based on the steps described above in connection with the Fig. 5A-5D) on each of the first, second, and third plurality of images. Alternatively, the processing unit 110 may perform a stereo image analysis (e.g., via execution of the stereo image analysis module 404 and based on the steps described above in connection with Fig. 6) on the first and second plurality of images, the second and third plurality of images, and / or the first and third plurality of images. The processed information corresponding to the analysis of the first, second, and / or third plurality of images may be combined. In some embodiments, the processing unit 110 may perform a combination of monocular and stereo image analyses. For example, the processing unit 110 may perform a monocular image analysis (e.g., via execution of the monocular image analysis module 402) on the first plurality of images and a stereo image analysis (e.g., via execution of the stereo image analysis module 404) on the second and third plurality of images.The configuration of the image capture devices 122, 124, and 126—including their respective locations and fields of view 202, 204, and 206—may influence the types of analyses performed on the first, second, and third pluralities of images. The disclosed embodiments are not limited to any particular configuration of the image capture devices 122, 124, and 126 or the types of analyses performed on the first, second, and third pluralities of images.

[0116] In some embodiments, processing unit 110 may perform testing on system 100 based on the images captured and analyzed at steps 710 and 720. Such testing may provide an indicator of the overall performance of system 100 for particular configurations of image capture devices 122, 124, and 126. For example, processing unit 110 may determine the proportion of "false hits" (e.g., cases where system 100 incorrectly determined the presence of a vehicle or pedestrian) and "misses."

[0117] At step 730, the processing unit 110 may initiate one or more navigation responses in the vehicle 200 based on information derived from two of the first, second, and third plurality of images. The selection of two of the first, second, and third plurality of images may depend on various factors, such as the number, types, and sizes of objects detected in each of the plurality of images. The processing unit 110 may also make the selection based on image quality and resolution, the effective field of view reflected in the images, the number of captured frames, the extent to which one or more objects of interest actually appear in the frames (e.g., the percentage of frames in which an object appears, the proportion of the object that appears in each of those frames, etc.), and the like.

[0118] In some embodiments, processing unit 110 may select information derived from two of the first, second, and third pluralities of images by determining the extent to which information derived from one image source is consistent with information derived from other image sources. For example, processing unit 110 may combine the processed information derived from each of image capture devices 122, 124, and 126 (whether through monocular analysis, stereo analysis, or any combination of the two) and determine visual indicators (e.g., lane markings, a detected vehicle and its location and / or path, a detected traffic light, etc.) that are consistent across the images captured by each of image capture devices 122, 124, and 126. Processing unit 110 may also exclude information that is inconsistent across the captured images (e.g.,a vehicle changing lanes, a lane model indicating a vehicle that is too close to the vehicle 200, etc.). Thus, the processing unit 110 may select information derived from two of the first, second, and third pluralities of images based on the determinations of consistent and inconsistent information.

[0119] Navigational responses may include, for example, a turn, a lane shift, a change in acceleration, and the like. The processing unit 110 may determine the one or more navigational responses based on the analysis performed at step 720 and the techniques described above in connection with Fig. 4. The processing unit 110 may also use data derived from the execution of the speed and acceleration module 406 to initiate the one or more navigation responses. In some embodiments, the processing unit 110 may initiate the one or more navigation responses based on a relative position, relative speed, and / or relative acceleration between the vehicle 200 and an object detected within any of the first, second, and third pluralities of images. Multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof. Reinforcement learning and trained navigation systems

[0120] The following sections discuss autonomous driving along with systems and methods for implementing autonomous control of a vehicle, whether that control is fully autonomous (a self-driving vehicle) or partially autonomous (e.g., one or more driver assistance systems or functions). As described in Fig. 8, the autonomous driving task can be divided into three main modules, including a sensor sensing module 801, a driving policy module 803, and a control module 805. In some embodiments, modules 801, 803, and 805 may be stored in the storage unit 140 and / or storage unit 150 of the system 100, or modules 801, 803, and 805 (or portions thereof) may be stored remotely from the system 100 (e.g., stored on a server accessible to the system 100, for example, via the wireless transceiver 172). Furthermore, each of the modules disclosed herein (e.g., modules 801, 803, and 805) may implement techniques associated with a trained system (such as a neural network or a deep neural network) or an untrained system.

[0121] The sensor sensing module 801, which may be implemented using the processing unit 110, may handle various tasks related to the sensor sensing of a navigation state in the environment of a host vehicle. Such tasks may be based on inputs from various sensors and sensor sensing systems associated with the host vehicle. These inputs may include images or image streams from one or more onboard cameras, GPS position information, accelerometer outputs, user feedback, or user inputs to one or more user interfaces, radar, lidar, etc. The sensor sensing, which may include data from cameras and / or any other available sensors along with map information, may be collected, analyzed, and formulated into a "sensor-sensed state" that describes information extracted from a scene in the environment of the host vehicle.The sensor-sensed state may include information related to target vehicles, lane markings, pedestrians, traffic lights, road geometry, lane shape, obstacles, distances to other objects / vehicles, relative speeds, and relative accelerations, among other potential sensor-sensed information. Supervised machine learning may be implemented to generate a sensor-sensed state output based on the sensor-sensed data provided to the sensor sensing module 801. The sensor sensing module output may represent a sensor-sensed navigation "state" of the host vehicle, which may be forwarded to the driving policy module 803.

[0122] While a sensor-detected state may be developed based on image data received from one or more cameras or image sensors associated with a host vehicle, a sensor-detected state for use in navigation may be developed using any suitable sensor or combination of sensors. In some embodiments, the sensor-detected state may be developed without reliance on sensed image data. Indeed, any of the navigation principles described herein may apply to sensor-detected states developed based on sensed image data as well as to sensor-detected states developed using other, non-image-based sensors. The sensor-detected state may also be determined via sources external to the host vehicle.For example, a sensor-sensed state may be developed in whole or in part based on information received from sources remote from the host vehicle (e.g., based on sensor information, processed state information, etc., shared by other vehicles, shared by a central server, or any other information source relevant to a navigation state of the host vehicle).

[0123] The driving policy module 803, which is discussed in more detail below and may be implemented using the processing unit 110, may implement a desired driving policy to decide on one or more navigation actions of the host vehicle in response to the sensor-detected navigation state. If no other actors (e.g., target vehicles or pedestrians) are present in the vicinity of the host vehicle, the input of the sensor-detected state to the driving policy module 803 may be handled in a relatively straightforward manner. The task becomes more complex if the sensor-detected state requires negotiations with one or more other actors. The technology used to generate the output of the driving policy module 803 may include reinforcement learning (discussed in more detail below).The output of the driving policy module 803 may include at least one navigation action for the host vehicle and may include a desired acceleration (which may translate into an updated speed for the host vehicle), a desired yaw rate for the host vehicle, and a desired trajectory, among other potential desired navigation actions.

[0124] Based on the output of the driving policy module 803, the control module 805, which may also be implemented using the processing unit 110, may develop control instructions for one or more actuators or controlled devices associated with the host vehicle. Such actuators and devices may include an accelerator, one or more steering controls, a brake, a signal generator, a display, or any other actuator or device that may be controlled as part of a navigation operation associated with a host vehicle. Aspects of control theory may be used to generate the output of the control module 805. The control module 805 may be responsible for developing and issuing instructions to controllable components of the host vehicle to implement the desired navigation goals or requirements of the driving policy module 803.

[0125] Referring again to the driving policy module 803, in some embodiments, a reinforcement learning-trained system may be used to implement the driving policy module 803. In other embodiments, the driving policy module 803 may be implemented without a machine learning approach, using specified algorithms to "manually" address the various scenarios that may arise during autonomous navigation. While practical, such an approach may result in an overly simplistic driving policy and lack the flexibility of a machine learning-based trained system.For example, a trained system can better handle complex navigation states and determine whether a taxi is parked or stopped to pick up or drop off a passenger; determine whether a pedestrian intends to cross the street in front of the host vehicle; compensate for unexpected behavior of other drivers by adopting a defensive posture; negotiate with target vehicles and / or pedestrians in heavy traffic; decide when to suspend certain navigation rules or expand other rules; anticipate non-sensor-detected but expected conditions (e.g., whether a pedestrian will step out from behind a car or an obstacle), etc. A trained system based on reinforcement learning may also be better equipped to address a continuous and high-dimensional state space along with a continuous action space.

[0126] Training the system using reinforcement learning may involve learning a driving policy to map sensor-detected states to navigation actions. A driving policy is a function π: S → A , where S is a set of states and A ⊂ ℝ 2 the action space (e.g., desired speed, acceleration, yaw commands, etc.). The state space is S = S s × S p , where S s the sensor detection state is and S p is additional information about the state stored by the policy. When working in discrete time intervals, the current state S at time t t ∈ S can be observed, and the policy can be applied to perform a desired action, a t = π(S t ), to obtain.

[0127] The system can be trained by exposing it to different navigation states, where the system applies the policy and provides a reward (based on a reward function designed to reward the desired navigation behavior). Based on the reward feedback, the system can "learn" the policy and is trained to produce desired navigation actions. For example, the learning system can retain the current state S t ∈ S and decide on an action a t ∈ A based on a policy π:S→D(A) Based on the decided action (and the implementation of the action), the environment moves to the next state S t+1 ∈ S is passed on to observation by the learning system. For each action developed in response to the observed state, the feedback to the learning system is a reward signal r1, r2, ....

[0128] The goal of reinforcement learning (RL) is to find a policy π. It is usually assumed that at time t there is a reward function r t which determines the momentary quality of being by the state of t and taking action a t measures. Taking action a tat a time t, however, influences the environment and therefore influences the value of future states. Consequently, when deciding on the action to take, not only the current reward should be considered, but future rewards should also be considered. In some cases, the system should take a certain action even if it is associated with a lower reward than another available option if the system determines that a greater reward can be realized in the future if the option with the lower reward is taken now. To formalize this, note that a policy π and an initial state s have a distribution over ℝ T where the probability of a vector (r1, ..., r T ) the probability of observing the rewards r1, ...,r Tis when the agent starts in state s0 = s and from there follows the policy π. The value of the initial state s can be defined as follows: Vπ(s)=E[∑t=1T〚rt|s0〛=s,∀t≥1,at=π(st)]

[0129] Instead of restricting the time horizon to T, the future rewards can be reduced to define the following for a fixed γ ∈ (0,1): Vπ(s)=E[∑t=1∈γtrt|s0=s,∀t≥1,at=π(st)]

[0130] In any case, the optimal policy is to solve argmaxπE[Vπ(s)] where the expectation over the initial state is s.

[0131] There are several possible methods for training the driving policy system. For example, an imitation approach (e.g., behavior cloning) can be used, where the system learns from state / action pairs, where the actions are those that a good agent (e.g., a human) would choose in response to a given observed state. Suppose a human driver is observed. Through this observation, many examples of the form (s t , a t ), where s t the condition and a t the action of the human driver, can be obtained, observed, and used as a basis for training the driving policy system. For example, supervised learning can be used to learn a policy π such that π(s t ≈ a t). There are many potential advantages of this approach. First, there is no requirement to define a reward function. Second, the learning is supervised and occurs offline (there is no need to use the agent in the learning process). A disadvantage of this method is that different human drivers, and even the same human drivers, are not deterministic in their policy choices. Therefore, learning a function for which ||π(s t ) - a t || is very small, often impractical. And even small errors can accumulate over time to result in large errors.

[0132] Another technique that can be used is policy-based learning. Here, the policy can be expressed in parametric form and optimized directly using a suitable optimization technique (e.g., stochastic gradient descent). The approach consists in argmaxπE[Vπ(s)] to solve a given problem directly. There are, of course, many ways to solve the problem. One advantage of this approach is that it addresses the problem directly and therefore often leads to good practical results. A potential disadvantage is that it often requires "on-policy" training, i.e., learning π is an iterative process where at iteration j we learn an imperfect policy π j and to find the next guideline π j we need to interact with the environment while we are based on π j act.

[0133] The system can also be trained using value-based learning (learning Q or V functions). Assume that a good approximation to the optimal value function V* can be learned. An optimal policy can be constructed (e.g., by invoking the Bellman equation). Some versions of value-based learning can be implemented offline (called off-policy training). Some disadvantages of the value-based approach can arise from its heavy reliance on Markovian assumptions and the required approximation of a complicated function (it may be more difficult to approximate the value function than to approximate the policy directly).

[0134] Another technique can involve model-based learning and planning (learning the probability of state transitions and solving the optimization problem to find the optimal V). Combinations of these techniques can also be used to train the learning system. In this approach, the dynamics of the process can be learned, namely the function that (s t , a t ) and a distribution over the next state s t+1 Once this function is learned, the optimization problem can be solved to find the policy π whose value is optimal. This is called “scheduling.” An advantage of this approach could be that the learning part is supervised and can be performed offline by observing triplets (s t , a t , s t+1). A disadvantage of this approach, similar to the "imitation" approach, could be that small errors can accumulate in the learning process and result in insufficiently powerful policies.

[0135] Another approach to training the driving policy module 803 may involve decomposing the driving policy function into semantically meaningful components. This allows for the manual implementation of parts of the policy, which can ensure the safety of the policy, and the implementation of other parts of the policy using reinforcement learning techniques, which can enable adaptability to many scenarios, a human-like balance between defensive / aggressive behavior, and human-like negotiation with other drivers. From a technical perspective, a reinforcement learning approach can combine multiple methods and provide a manageable training procedure where most of the training can be performed using either recorded data or a custom-built simulator.

[0136] In some embodiments, training of the driving policy module 803 may rely on an "options" mechanism. To illustrate, consider a simple driving policy scenario for a two-lane highway. In a direct RL approach, a policy π represents the state in A ⊂ ℝ 2 where the first component of π (s) is the desired acceleration command and the second component of π (s) is the yaw rate. Using a modified approach, the following guidelines can be established:

[0137] Guideline for automatic distance control (Automatic Cruise Control- ACC), O ACC : S → A: this policy always outputs a yaw rate of 0 and only changes the speed to implement smooth and accident-free driving.

[0138] ACC+Links Policy, O L:S → A: The longitudinal command of this policy is the same as the ACC command. The yaw rate is a straightforward implementation of centering the vehicle toward the center of the left lane while ensuring safe lateral movement (e.g., not moving to the left if there is a car on the left).

[0139] ACC+Legal Directive, O R : S → A: Like O L , but the vehicle can be centered towards the center of the right lane.

[0140] These policies can be called “options.” Based on these “options,” a policy can be learned that contains options π o :S → O, where O is the set of available options. In one case, O = {o ACC , o L , o R}. The option selection policy π o defines an actual policy π:S → A by defining for each s π(s) = o πo(s) (s) is determined.

[0141] In practice, the policy function can be decomposed into an option graph 901, as in Fig. 9. Another example options graph 1000 is shown in Fig. 10. The options graph can represent a hierarchical set of decisions organized as a Directed Acyclic Graph (DAG). There is a special node called the root node 903 of the graph. This node has no incoming nodes. The decision process traverses the graph, starting from the root node, until it reaches a "leaf" node, which points to a node that has no outgoing decision lines. As shown in Fig. 9, the leaf nodes may include, for example, nodes 905, 907, and 909. Upon encountering a leaf node, the driving policy module 803 may output the acceleration and steering commands associated with a desired navigation action associated with the leaf node.

[0142] Internal nodes, such as nodes 911, 913, and 915, can lead to the implementation of a policy that selects a child from the available options. The set of available children of an internal node includes all nodes that are associated with a particular internal node via decision lines. For example, the internal node 913, which is shown in Fig. 9 is marked as “Threading”, three child nodes 909, 915 and 917 (“Stay”, “Overtake on the right” and “Overtake on the left”), each of which is connected to node 913 by a decision line.

[0143] The flexibility of the decision-making system can be increased by allowing nodes to adjust their position in the hierarchy of the options graph. For example, any node is allowed to declare itself as "critical." Each node can implement an "is critical" function that returns "true" if the node is in a critical section of its policy implementation. For example, a node responsible for an overtaking maneuver can declare itself as critical while in the middle of a maneuver. This can impose restrictions on the set of available children of a node u, which can include all nodes v that are children of node u and for which there is a path from v to a leaf node that passes through all nodes marked as critical.Such an approach can, on the one hand, allow the declaration of the desired path in the graph at each time step, while, on the other hand, maintaining the stability of a policy, especially while critical parts of the policy are being implemented.

[0144] By defining an option graph, the problem of learning the driving policy π:S → A can be decomposed into a problem of defining a policy for each node of the graph, where the policy should select from available child nodes at internal nodes. For some of the nodes, the respective policy can be implemented manually (e.g., through if-then algorithms that specify a set of actions in response to an observed state), while for others, the policies can be implemented using a trained system built through reinforcement learning. The choice between manual or trained / learned approaches may depend on the safety aspects associated with the task and their relative simplicity. The option graphs can be constructed in such a way that some of the nodes are straightforward to implement, while others can rely on trained models.Such an approach can ensure the safe operation of the system.

[0145] The following discussion provides further details regarding the role of the options graph of Fig. 9 within the driving policy module 803. As discussed above, the input to the driving policy module is a "sensor-detected state" that summarizes the environment map, such as that obtained from the available sensors. The output of the driving policy module 803 is a set of desires (optionally along with a set of hard constraints) that define a trajectory as a solution to an optimization problem.

[0146] As described above, the options graph represents a hierarchical set of decisions organized as a DAG. There is a special node called the "root" of the graph. The root node is the only node that has no incoming edges (e.g., decision lines). The decision process traverses the graph, starting from the root node until it reaches a "leaf" node, namely a node that has no outgoing edges. Each internal node should implement a policy that selects a child from among its available children. Each leaf node should implement a policy that defines a set of preferences (e.g., a set of navigation goals for the host vehicle) based on the entire path from the root to the leaf.The set of desires, together with a set of hard constraints defined directly based on the sensor-detected state, forms an optimization problem, the solution to which is the trajectory for the vehicle. The hard constraints can be used to further increase the safety of the system, and the desires can be used to provide ride comfort and human-like driving behavior of the system. The trajectory provided as a solution to the optimization problem, in turn, defines the commands that should be provided to the steering, braking, and / or motor actuators to achieve the trajectory.

[0147] With further reference to Fig. 9, the options graph 901 represents an options graph for a two-lane highway, including merge lanes (meaning that at some points a third lane merges into the right or left lane of the highway). The root node 903 first decides whether the host vehicle is in a single-road scenario or approaching a merge scenario. This is an example of a decision that can be implemented based on the sensor detection state. The single-road node 911 includes three child nodes: stay node 909, pass-left node 917, and pass-right node 915. Stay refers to a situation where the host vehicle wishes to continue traveling in the same lane. The stay node is a leaf node (no outgoing edges / lines). Therefore, the stay node defines a set of desires.The first desire it defines may include the desired lateral position—e.g., as close as possible to the center of the current lane. It may also desire to navigate smoothly (e.g., within predefined or permissible acceleration limits). The Stay node may also define how the host vehicle should react to other vehicles. For example, the Stay node may review the sensor-detected target vehicles and assign each a semantic meaning that can be translated into components of the trajectory.

[0148] Different semantic meanings can be assigned to target vehicles in a host vehicle's environment. For example, in some embodiments, the semantic meaning may include one of the following labels: 1) not relevant: indicates that the sensor-detected vehicle is not currently relevant in the scene; 2) nearest lane: indicates that the sensor-detected vehicle is in an adjacent lane and a reasonable offset to that vehicle should be maintained (the exact offset can be calculated in the optimization problem that constructs the trajectory given the desires and hard constraints, and may be vehicle-dependent—the stay leaf of the options graph sets the semantic type of the target vehicle, which defines the desire regarding the target vehicle);3) Yield: The host vehicle will attempt to yield the right-of-way to the sensor-detected target vehicle, for example, by reducing speed (especially if the host vehicle determines that the target vehicle is likely to merge into the host vehicle's lane); 4) Yield: The host vehicle will attempt to yield the right-of-way, for example, by increasing speed; 5) Follow: The host vehicle wishes to maintain smooth driving by following this target vehicle; 6) Left / Right Overtake: This means that the host vehicle wishes to initiate a lane change to the left or right lane. The Left Overtake node 917 and the Right Overtake node 915 are internal nodes that do not yet define any desires.

[0149] The next node in options graph 901 is the select-gap node 919. This node may be responsible for selecting a gap between two target vehicles in a particular target lane into which the host vehicle wishes to enter. By selecting a node of the form IDj for some value of j, the host vehicle arrives at a leaf indicating a desire for the trajectory optimization problem—e.g., the host vehicle wishes to perform a maneuver to reach the selected gap. Such a maneuver may involve first accelerating / decelerating in the current lane and then approaching the target lane at an appropriate time to enter the selected gap. If the select-gap node 919 does not find a suitable gap, it transitions to the abort node 921, which defines the desire to return to the center of the current lane and cancel the overtaking.

[0150] Referring again to the merge node 913, when approaching a merge, the host vehicle has several options that may depend on a particular situation. As in Fig. For example, as shown in Figure 11A, host vehicle 1105 is traveling along a two-lane road without detecting any other target vehicles, either in the main lanes of the two-lane road or in the merge lane 1111. In this situation, driving policy module 803 may select stay node 909 upon reaching merge node 913. That is, remaining within its current lane may be desired if no target vehicles are sensor-detected as merging into the roadway.

[0151] In Fig. 11B, the situation is slightly different. Here, the host vehicle 1105 senses one or more target vehicles 1107 entering the main lane 1112 from the merge lane 1111. In this situation, the driving policy module 803 may initiate a left-passing maneuver upon encountering the merge node 913 to avoid the merge situation.

[0152] In Fig. 11C, the host vehicle 1105 encounters one or more target vehicles 1107 entering the main lane 1112 from the merge lane 1111. The host vehicle 1105 also detects target vehicles 1109 traveling in a lane adjacent to the host vehicle's lane. The host vehicle also detects one or more target vehicles 1110 traveling in the same lane as the host vehicle 1105. In this situation, the driving policy module 803 may decide to adjust the speed of the host vehicle 1105 to yield to the target vehicle 1107 and proceed ahead of the target vehicle 1115. This can be achieved, for example, by proceeding to the Select Gap node 919, which in turn selects a gap between ID0 (vehicle 1107) and ID1 (vehicle 1115) as a suitable merging gap. In such a case, the suitable gap of the merging situation defines the objective for an optimization problem of the trajectory planner.

[0153] As discussed above, nodes of the options graph can declare themselves as "critical," which can ensure that the selected option traverses the critical nodes. Formally, each node can implement an IsCritical function. After performing a forward traversal on the options graph, from the root to a leaf, and solving the trajectory planner's optimization problem, a backward traversal from the leaf back to the root can be performed. Along this backward traversal, the IsCritical function of all nodes in the traversal can be called, and a list of all critical nodes can be stored. In the forward path corresponding to the next time frame, the driving policy module 803 may be required to choose a path from the root node to a leaf that traverses all critical nodes.

[0154] The Fig. 11A-11C can be used to demonstrate a potential benefit of this approach. For example, in a situation where an overtaking maneuver is initiated and the driving policy module 803 arrives at the leaf corresponding to IDk, it would not be desirable to select, for example, the stay node 909 when the host vehicle is in the middle of the overtaking maneuver. To avoid such erratic behavior, the IDj node may classify itself as critical. During the maneuver, the success of the trajectory planner may be monitored, and the IsCritical function returns the value "True" if the overtaking maneuver is progressing as intended. This approach can ensure that in the next timeframe, the overtaking maneuver continues (rather than jumping to another, possibly inconsistent maneuver before the initially selected maneuver is completed).On the other hand, if the maneuver monitoring indicates that the selected maneuver is not progressing as intended, or if the maneuver has become unnecessary or impossible, the IsCritical function can return a "False" value. This can allow the Select Gap node to select a different gap in the next time frame or abort the overtaking maneuver altogether. This approach can, on the one hand, enable the declaration of the desired path in the options graph at each time step and, on the other hand, help promote policy stability during critical parts of execution.

[0155] Hard constraints, discussed in more detail below, can be differentiated from navigation desires. For example, hard constraints can ensure safe driving by applying an additional layer of filtering to a planned navigation action. The implied hard constraints, which can be manually programmed and defined rather than using a trained system based on reinforcement learning, can be determined from the sensor-sensed state. However, in some embodiments, the trained system can learn the applicable hard constraints to apply and follow.Such an approach may encourage the driving policy module 803 to arrive at a selected action that is already consistent with the applicable hard constraints, which may reduce or eliminate selected actions that may later require modification to comply with applicable hard constraints. Nevertheless, as a redundant safety measure, hard constraints may be applied to the output of the driving policy module 803, even if the driving policy module 803 has been trained to consider predetermined hard constraints.

[0156] There are many examples of potential hard constraints. For example, a hard constraint can be defined in connection with a guardrail at the edge of a road. Under no circumstances is the host vehicle allowed to cross the guardrail. Such a rule induces a hard lateral constraint on the host vehicle's trajectory. Another example of a hard constraint can be a road hump (e.g., a speed limit hump), which can induce a hard restriction on the vehicle speed before and during crossing the hump. Hard constraints can be considered safety-critical and can therefore be defined manually rather than relying solely on a trained system that learns the constraints during training.

[0157] In contrast to hard constraints, the goal of desires may be to enable or achieve comfortable driving. As discussed above, an example of a desire may be the goal to position the host vehicle in a lateral position within a lane that corresponds to the center of the host vehicle's lane. Another desire may be the ID of a gap into which to fit. Note that there is no requirement that the host vehicle be exactly in the center of the lane, but instead, a desire to be as close as possible may ensure that the host vehicle tends to migrate toward the center of the lane even in the event of deviations from the center of the lane. Desires may not be safety-critical. In some embodiments, desires may require negotiation with other drivers and pedestrians.An approach to creating the desires may rely on the options graph, and the policy implemented in at least some nodes of the graph may be based on reinforcement learning.

[0158] For the nodes of the option graph 901 or 1000 implemented as nodes trained based on learning, the training process may involve decomposing the problem into a supervised learning phase and a reinforcement learning phase. In the supervised learning phase, a differentiable mapping of (s t , a t ) on ŝ t+1 be learned so that ŝ t+1 ≈ s t+1 This can be similar to “model-based” reinforcement learning. In the forward loop of the network, ŝ t+1 however, by the actual value of s t+1 be replaced, thus eliminating the problem of error accumulation. The role of predicting ŝ t+1consists of propagating messages from the future back to past actions. In this sense, the algorithm can be a combination of "model-based" reinforcement learning and "policy-based learning."

[0159] An important element that can be provided in some scenarios is a differentiable path from future losses / rewards back to decisions about actions. In the options graph structure, the implementation of options that involve security constraints is usually not differentiable. To overcome this problem, the choice of a child in a learned policy node can be stochastic. That is, a node can output a probability vector p that assigns the probabilities used in selecting each of the children of the given node. Suppose a node has k children, and let a (1) , ..., a (k)the actions of the path from each child to a leaf. The resulting predicted action is thereforeâ = ∑i=1kpia(i), which can lead to a differentiable path from the action to p. In practice, an action a can be defined as a (i) for i~p, and the difference between a and â can be called additive noise.

[0160] For the training of ŝ t+1 for given s t , a tSupervised learning can be used together with real data. For training, the policy of the node simulators can be used. Later, the fine-tuning of a policy can be achieved using real data. Two concepts can make the simulation more realistic. First, using imitation, an initial policy can be created using the "behavioral cloning" paradigm using large real datasets. In some cases, the resulting agents may be suitable. In other cases, the resulting agents at least form very good initial policies for the other agents on the roads. Second, using self-play, our own policy can be used to augment the training.For example, a policy can be trained based on a simulator for an initial implementation of the other agents (cars / pedestrians) that might be experienced. Some of the other agents can be replaced with the new policy, and the process can be repeated. As a result, the strategy can continue to improve, as it should respond to a wider variety of other agents with different levels of complexity.

[0161] Furthermore, in some embodiments, the system may implement a multi-agent approach. For example, the system may consider data from different sources and / or images captured from multiple angles. Furthermore, some disclosed embodiments may provide energy savings because the anticipation of an event that does not directly affect the host vehicle but may have an impact on the host vehicle may be considered, or even the anticipation of an event that may lead to unforeseeable circumstances involving other vehicles may be considered (e.g., the radar may "see through" the preceding vehicle and anticipate an unavoidable or even highly probable event that impacts the host vehicle). Trained system with imposed navigation constraints

[0162] In the context of autonomous driving, a key concern is how to ensure that a learned policy of a trained navigation network is safe. In some embodiments, the driving policy system may be trained using constraints so that the actions selected by the trained system can take into account already existing safety constraints. Additionally, in some embodiments, an additional layer of safety may be provided by guiding the selected actions of the trained system through one or more hard constraints implied by a particular sensor-detected scene in the host vehicle's environment. Such an approach can ensure that the actions taken by the host vehicle have been limited to those that have been confirmed as meeting the applicable safety constraints.

[0163] At its core, the navigation system may include a learning algorithm based on a policy function that maps an observed state to one or more desired actions. In some implementations, the learning algorithm is a deep learning algorithm. The desired actions may include at least one action that is expected to maximize an expected reward for a vehicle. While in some cases the actual action taken by the vehicle may correspond to one of the desired actions, in other cases the actual action taken may be determined based on the observed state, one or more desired actions, and unlearned, hard constraints (e.g., safety constraints) imposed on the learning navigation system. These constraints may include no-driving zones around various types of detected objects (e.g.,Target vehicles, pedestrians, stationary objects at the edge of a road or in a carriageway, moving objects at the edge of a road or in a carriageway, guardrails, etc.). In some cases, the size of the zone may vary based on a detected object's movement (e.g., speed and / or direction). Further restrictions may include a maximum travel speed when passing within a pedestrian's influence zone, a maximum deceleration (to account for a target vehicle distance behind the host vehicle), a mandatory stop at a sensor-detected crosswalk or railroad crossing, etc.

[0164] Hard constraints used in conjunction with a machine learning-trained system can provide a level of confidence in autonomous driving that may exceed one based solely on the output of the trained system. For example, the machine learning system can be trained using a desired set of constraints as training guidelines, and thus, in response to a sensor-detected navigation state, the trained system can select an action that considers and adheres to the constraints of the applicable navigation constraints. Nevertheless, the trained system exhibits some flexibility in selecting navigation actions, and thus at least some situations may exist in which an action selected by the trained system does not strictly adhere to the relevant navigation constraints.Therefore, to require that a selected action strictly adheres to the relevant navigation constraints, the output of the trained system can be combined, compared, filtered, adjusted, modified, etc., using a non-machine learning component outside the learning / training framework that guarantees the strict application of the relevant navigation constraints.

[0165] The following discussion provides additional details regarding the trained system and the potential benefits (particularly from a security perspective) that can be gained by combining a trained system with an algorithmic component outside the training / learning framework. As discussed, the objective of reinforcement learning can be optimized by policies through stochastic gradient descent. The objective (e.g., the expected reward) can be defined as Es¯PsR(s¯) be defined.

[0166] Goals that include an expected value can be used in machine learning scenarios. However, such a goal, which is not bound by navigation constraints, cannot return actions that are strictly bound by those constraints. For example, considering a reward function such that R(s) = -r for trajectories that represent a rare, avoidable "corner" event (e.g., a crash), and R(s) ∈ [-1, 1] for the rest of the trajectories, one goal for the learning system might be to learn how to perform an overtaking maneuver. Normally, for an accident-free trajectory, R(s) would reward successful, smooth overtakes and penalize remaining in a lane without completing the overtake—hence the range [-1, 1]. If a sequence s represents an accident, the reward -r should provide a sufficiently high punishment to deter such an incident.The question is how high the value of r should be to ensure accident-free driving.

[0167] Note that the impact of an accident on E[R(s¯)] the additive term is -pr, where p is the probability mass of the trajectories with an accident event. If this term is negligible, i.e. p<<1r, then the learning system may prefer a policy that performs an accident (or generally adopts a reckless driving policy) to successfully complete the overtaking maneuver more often than a policy that would be more defensive, at the expense of some overtaking maneuvers that are not successfully completed. In other words, if the probability of accidents is to be at most p, then r must be set so that r>>1p. It may be desirable to make p extremely small (e.g., on the order of p = 10 -9). Therefore, r should be large. In a policy gradient, the gradient of E[R(s¯)] The following lemma shows that the variance of the random variable R(s) is 2 grows, which is larger than r for r>>1p Therefore, estimating the target can be difficult, and estimating its gradient can be even more difficult.

[0168] Lemma: Let π o a guideline and let p and r be scalars such that with probability p R(s) = -r and with probability 1 - p R(s) ∈ [-1,1]. Then Var[R(s¯)]≥pr2−(pr+(1=p))2=(p−p2)r2−2p(1−p)r−(1−p)2≈pr2 where the last approximation for the case r≥1p applies.

[0169] This discussion shows that a goal of the form E[R(s¯)] cannot ensure functional safety without causing a variance problem. The baseline subtraction method for variance reduction may not provide a sufficient remedy for the problem, as the problem would shift from a high variance of R(s) to an equally high variance of the baseline constant, the estimation of which would also suffer from numerical instabilities. Furthermore, if the probability of an accident is p, then on average at least 1p ≈ sequences are sampled before an accident event is obtained. This implies a lower limit of 1p Sampling sequences for a learning algorithm that aims to E[R(s¯)] The solution to this problem can be found in the architectural design described here instead of numerical conditioning techniques. The approach here is based on the idea that hard constraints should be introduced outside the learning framework. In other words, the policy function can be decomposed into a learnable part and a non-learnable part. Formally, the policy function can be written as π0=π(T)∘πθ(D) be structured, whereby πθ(D) maps the (agnostic) state space to a set of desires (e.g. desired navigation destinations, etc.), while π(T) maps the wishes to a trajectory (which can determine how the vehicle should move in a short area). The function πθ(D) is responsible for driving comfort and for making strategic decisions, such as which other vehicles should be overtaken or which should be given priority and what the desired position of the host vehicle is within its lane, etc. The assignment of the sensor-detected navigation state to wishes is a guideline πθ(D), which can be learned from experience by maximizing an expected reward. πθ(D) The desires generated can be translated into a cost function over the travel trajectories. The function π (T) , which is not a learned function, can be implemented by finding a trajectory that minimizes the costs subject to hard functional safety constraints. This decomposition can ensure functional safety while providing comfortable driving.

[0170] A navigation situation of double threading, as in Fig. Figure 11D provides an example that further illustrates these concepts. In a double merge, vehicles approach the merge area 1130 from both the left and right sides. And from either side, a vehicle, such as vehicle 1133 or vehicle 1135, can decide whether to merge into the lanes on the other side of the merge area 1130. Successfully executing a double merge in heavy traffic can require considerable negotiation and experience and can be difficult to perform using a heuristic or brute-force approach by enumerating all possible trajectories that could be taken by all agents in the scene. In this example of double merge, a set of desires, D, which are suitable for the double merging maneuver. D can be the Cartesian product of the following quantities: D=[0,vmax]×L×{g,t,o}n, where [0, v max] is the desired target speed of the host vehicle, L = {1, 1.5, 2, 2.5, 3, 3.5, 4} is the desired lateral position in lane units, where integers denote a lane center and fractional numbers denote lane boundaries, and {g, t, o} are the classification flags assigned to each of the n other vehicles. The other vehicles can be assigned "g" if the host vehicle should yield to the other vehicle, "f" if the host vehicle should yield relative to the other vehicle, or "o" if the host vehicle should maintain an offset distance relative to the other vehicle.

[0171] Below you will find a description of how a set of wishes, (v,l,c1,…,cn)∈D, can be translated into a cost function over travel trajectories. A travel trajectory can be represented by (x1, y1), ..., (x k , y k ), where (x i , y i ) the (lateral, longitudinal) location of the host vehicle (in egocentric units) at the time T⋅i In some experiments, T = 0.1 sec and k = 10. Of course, other values ​​can be chosen. The cost assigned to a trajectory can include a weighted sum of the individual costs assigned to the desired speed, the lateral position, and the label assigned to each of the other n vehicles.

[0172] For a desired speed v ∈ [0, v max] are the costs associated with the speed of a trajectory ∑i=2k(v−‖xi,yi−(xi−1,yi−1)‖T)2.

[0173] At the desired lateral position, l ∈ L, the costs associated with the desired lateral position are ∑i=1kdist(xi,yi,l) where dist(x i , y i , l) is the distance from point (x, y) to lane position l. Regarding the costs due to other vehicles, (x1',y1'),…,(xk',yk1) for any other vehicle, represent the other vehicle in egocentric units of the host vehicle, and i can be the earliest point for which j exists, so that the distance between (x i , y i ) and (xj',yj') is small. If there is no such point, i can be set as i = -. If another vehicle is classified as “give way”, it may be desirable that Ti>Tj+0.5, which means that the host vehicle arrives at the intersection point of the trajectory at least 0.5 seconds later than the other vehicle arrives at the same point. A possible formula for translating the above constraint into costs is [T(j−i)+0.5]+.

[0174] Similarly, if another vehicle is classified as "taking priority," it may be desirable for Tj > Ti + 0.5, which translates into a cost of [T(i - j) + 0.5] + . If another vehicle is classified as "offset," it may be desirable for i = ∞, meaning that the host vehicle's trajectory and the offset vehicle's trajectory do not overlap. This condition can translate into a cost by penalizing the distance between the trajectories.

[0175] Assigning a weight to each of these costs can provide a single objective function for the trajectory planner, π(T) Costs that encourage smooth driving can be added to the goal. And to ensure the functional safety of the trajectory, hard constraints can be added to the goal. For example, (x i ,y i ) be prohibited from leaving the roadway, and (x i ,y i ) may be prohibited from coming near (xj',yj') for any trajectory point (xj',yj') of any other vehicle if |i - j| is small.

[0176] In summary, the directive π θinto a mapping from the agnostic state to a set of desires and a mapping from desires to an actual trajectory. The latter mapping is not based on learning and can be implemented by solving an optimization problem whose cost depends on the desires and whose hard constraints can guarantee the functional safety of the policy.

[0177] The following discussion describes the mapping from an agnostic state to a set of desires. As described above, a system relying solely on reinforcement learning may suffer from high and unmanageable variance in reward R(s) to be consistent with functional safety. This outcome can be avoided by decomposing the problem into a mapping from an (agnostic) state space to a set of desires using policy gradient iterations, followed by a mapping to an actual trajectory, which does not involve a system trained based on machine learning.

[0178] For various reasons, the decision-making process can be further decomposed into semantically meaningful components. For example, the size of D large and even continuous. In the double threading scenario described above with reference to Fig. 11D applies D=[0,vmax]×L×{g,t,o}n In addition, the gradient estimator can use the term ∑t=1T∇θπθ(at|st) . With such an expression, the variance may increase with the time horizon T. In some cases, the value of T may be roughly 250, which may be high enough to create a significant variance. Assuming the sampling rate is in the range of 10 Hz and the merging area 1130 is 100 meters, then merging preparation may begin approximately 300 meters before the merging area. If the host vehicle is traveling at 16 meters per second (about 60 km per hour), then the value of T for an episode may be roughly 250.

[0179] Referring again to the concept of an options graph, Fig. 11E shows an options graph that is representative of the Fig. 11D. As discussed above, an options graph may represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There may be a special node in the graph called the "root" node 1140, which may be the only node that has no incoming edges (e.g., decision lines). The decision process may traverse the graph, starting from the root node, until it reaches a "leaf" node, namely, a node that has no outgoing edges. Each internal node may implement a policy function that chooses a child from its available children. There may be a predefined mapping from the set of traversals across the options graph to the set of desires, D. In other words, a traversal on the options graph can be D be automatically translated into a wish. For a node v in the graph, a parameter vector θ v specify the policy of selecting a child of v. If θ is the concatenation of all θ v is, then πθ(D) be defined by traversing from the root of the graph to a leaf, while at each node v the function defined by θ v defined policy is used to select a child node.

[0180] In the double threading options graph 1139 of Fig. 11E, the root node 1140 may first decide whether the host vehicle is within the merging area (e.g., area 1130 of Fig. 11D) or whether the host vehicle is instead approaching the merging area and needs to prepare for a possible merge. In both cases, the host vehicle may need to decide whether to change lanes (e.g., to the left or right) or remain in the current lane. Once the host vehicle has decided to change lanes, the host vehicle may need to decide whether conditions are suitable to proceed and perform the lane change maneuver (e.g., at "Drive" node 1142). If it is not possible to change lanes, the host vehicle may attempt to "push" towards the desired lane (e.g., at node 1144 as part of a negotiation with vehicles in the desired lane) by aiming to be on the lane marking. Alternatively, the host vehicle may choose to “stay” in the same lane (e.g., at junction 1146).Such a process can naturally determine the lateral position of the host vehicle.

[0181] This can allow the desired lateral position to be determined naturally. For example, if the host vehicle changes lanes from lane 2 to lane 3, the "drive" node can set the desired lateral position to 3, the "stay" node can set the desired lateral position to 2, and the "push" node can set the desired lateral position to 2.5. Next, the host vehicle can decide whether to maintain the "same" speed (node ​​1148), "accelerate" (node ​​1150), or "decelerate" (node ​​1152). Next, the host vehicle can enter a "chain-like" structure 1154 that traverses the other vehicles and sets their semantic meaning to a value in the set {g, t, o}. This process can establish the desires relative to the other vehicles. The parameters of all nodes in this chain can be shared (similar to recurrent neural networks).

[0182] A potential advantage of the options is the interpretability of the results. Another potential advantage is that one can rely on the decomposable structure of the set D and thus the policy can choose from a small number of possibilities at each node. Additionally, the structure can enable a reduction in the variance of the policy gradient estimator.

[0183] As discussed previously, the length of an episode in the double-merging scenario can be roughly T = 250 steps. Such a value (or any other suitable value depending on a specific navigation scenario) can provide sufficient time to see the consequences of the host vehicle's actions (e.g., if the host vehicle decides to change lanes in preparation for merging, the host vehicle will only see the benefit after the merge is successfully completed). On the other hand, due to vehicle dynamics, the host vehicle must make decisions at a sufficiently fast frequency (e.g., 10 Hz in the case described above).

[0184] The options graph can allow for a reduction of the effective value of T in at least two ways. First, higher-level decisions can be rewarded for lower-level decisions, taking shorter episodes into account. For example, if the host vehicle has already chosen a "lane change" and the "drive" node, a policy for assigning semantic meaning to vehicles can be learned by considering episodes of 2-3 seconds (meaning that T becomes 20-30 instead of 250). Second, for high-level decisions (such as whether to change lanes or stay in the same lane), the host vehicle may not need to make decisions every 0.1 seconds. Instead, the host vehicle may be able to make decisions at a lower frequency (e.g.,Each second) or implementing an "option termination" function, and then the gradient can only be computed after each option termination. In both cases, the effective value of T may be an order of magnitude smaller than its original value. All in all, the estimator at each node may depend on a value of T that is an order of magnitude smaller than the original 250 steps, which may immediately translate to a smaller variance.

[0185] As discussed above, hard constraints can promote safer driving, and there can be several different types of constraints. For example, static hard constraints can be defined directly from the sensor detection state. These can include speed bumps, speed limits, road curvatures, intersections, etc. within the host vehicle's environment, which can imply one or more restrictions on vehicle speed, direction of travel, acceleration, braking (deceleration), etc. Static hard constraints can also include a semantic clearance space, where the host vehicle is, for example, prohibited from leaving the clearance space and navigating too close to physical barriers. Static hard constraints can also restrict maneuvers (e.g.,prohibit) that do not correspond to various aspects of a vehicle's kinematic motion; for example, a static hard constraint may be used to prohibit maneuvers that could cause the host vehicle to roll over, slide, or otherwise lose control.

[0186] Hard constraints can also be associated with vehicles. For example, a constraint can be employed that requires a vehicle to maintain a longitudinal separation of at least one meter from other vehicles and a lateral separation of at least 0.5 meters from other vehicles. Constraints can also be applied so that the host vehicle avoids maintaining a collision course with one or more other vehicles. For example, a time τ can be a time measure based on a particular scene. The predicted trajectories of the host vehicle and one or more other vehicles can be considered from a current time to a time τ. Where the two trajectories intersect, (tia,til) represent the arrival time and departure time of vehicle i at the intersection point. That is, each car arrives at the point when a first part of the car passes the intersection point, and a certain amount of time is needed until the last part of the car passes the intersection point. This time separates the arrival time from the departure time. Assuming that t1a <t2a (i.e., that the arrival time of vehicle 1 is less than the arrival time of vehicle 2), we will want to ensure that vehicle 1 has left the intersection before vehicle 2 arrives. Otherwise, a collision would result. Therefore, a hard constraint can be implemented such that t1a <t2a. To ensure that vehicle 1 and vehicle 2 do not miss each other by a minimal amount, an additional safety margin can be obtained by including a buffer time in the constraint (e.g., 0.5 seconds or another suitable value). A hard constraint related to the predicted intersection trajectories of two vehicles can be defined as t1l <t2a+0,5 be expressed.

[0187] The time period π over which the trajectories of the host vehicle and one or more other vehicles are tracked can vary. However, in intersection scenarios where speeds may be lower, τ can be longer, and τ can be defined such that a host vehicle enters and exits the intersection in less than τ seconds.

[0188] Applying hard constraints to vehicle trajectories naturally requires that the trajectories of these vehicles can be predicted. For the host vehicle, trajectory prediction can be relatively straightforward, as the host vehicle generally already understands an intended trajectory and is actually planning it at any given time. Compared to other vehicles, predicting their trajectories can be less straightforward. For other vehicles, the baseline calculation for determining the predicted trajectories can be based on the current speed and heading of the other vehicles, as determined, for example, based on the analysis of an image stream acquired by one or more cameras and / or other sensors (radar, lidar, acoustics, etc.) onboard the host vehicle.

[0189] However, there may be some exceptions that can simplify the problem or at least provide additional confidence in a trajectory predicted for another vehicle. For example, with respect to structured roads where there is an indication of the lanes and where right-of-way rules may exist, the trajectories of other vehicles may be based at least in part on the position of the other vehicles relative to the lanes and on the applicable right-of-way rules. Therefore, in some situations, when lane structures are detected, it can be assumed that vehicles in the adjacent lane will respect the lane boundaries. That is, the host vehicle may assume that a vehicle in the adjacent lane will stay in its lane unless there are detected signs (e.g.a signal light, strong lateral movement, movement across a lane boundary) indicating that the vehicle in the adjacent lane is about to merge into the host vehicle's lane.

[0190] Other situations may also provide clues regarding the expected trajectories of other vehicles. For example, at stop signs, traffic lights, roundabouts, etc., where the host vehicle may have the right-of-way, it can be assumed that other vehicles will respect that right-of-way. Therefore, other vehicles can be assumed to proceed along a trajectory that respects the host vehicle's right-of-way, unless there are detected signs of a rule violation.

[0191] Hard restrictions may also be applied with respect to pedestrians in the vicinity of the host vehicle. For example, a buffer distance with respect to pedestrians may be established, such that the host vehicle is prohibited from navigating closer than the prescribed buffer distance relative to a detected pedestrian. The pedestrian buffer distance may be any suitable distance. In some embodiments, the buffer distance may be at least one meter relative to a detected pedestrian.

[0192] Similar to the situation with vehicles, hard constraints can also be applied regarding the relative movement between pedestrians and the host vehicle. For example, the trajectory of a pedestrian (based on a direction of travel and speed) can be monitored relative to the projected trajectory of the host vehicle. For a given pedestrian trajectory, t(p), with any point p on the trajectory, can represent the time it takes the pedestrian to reach point p. To maintain the required buffer distance of at least 1 meter from the pedestrian, either t(p) must be greater than the time it takes the host vehicle to reach point p (with a sufficient time difference so that the host vehicle passes in front of the pedestrian at a distance of at least 1 meter) or t(p) must be less than the time it takes the host vehicle to reach point p (e.g.when the host vehicle brakes to yield to the pedestrian). Nevertheless, the hard constraint in the latter example may require that the host vehicle will reach point p sufficiently later than the pedestrian so that the host vehicle can pass behind the pedestrian and maintain the required buffer distance of at least one meter. Of course, there may be exceptions to the hard constraint for the pedestrian. For example, where the host vehicle has the right-of-way, or where speeds are very low and there are no observed indications that the pedestrian will refuse to yield to the host vehicle or otherwise navigate towards the host vehicle, the hard constraint for pedestrians may be relaxed (e.g., to a smaller buffer of at least 0.75 meters or 0.50 meters).

[0193] In some examples, constraints may be relaxed if it is determined that not all can be met. For example, in situations where a road is too narrow to maintain the desired distance (e.g., 0.5 meters) to both curbs, or to a curb and a parked vehicle, one or more of the constraints may be relaxed if there are extenuating circumstances. For example, if there are no pedestrians (or other objects) on the sidewalk, one may proceed slowly at 0.1 meters from the curb. In some embodiments, the constraints may be relaxed if doing so improves the user experience. For example, to avoid a pothole, the constraints may be relaxed to allow a vehicle to navigate closer to the edges of the lane, a curb, or a pedestrian than would normally be allowed.Additionally, when determining which restrictions should be relaxed, in some embodiments, the one or more restrictions deemed to have the least negative impact on safety are chosen. For example, a restriction regarding how close the vehicle may travel to the curb or a concrete barrier may be relaxed before one addressing proximity to other vehicles. In some embodiments, pedestrian restrictions may be relaxed last, or in some situations, may never be relaxed.

[0194] Fig. 12 shows an example of a scene that may be captured and analyzed during navigation of a host vehicle. For example, a host vehicle may include a navigation system (e.g., system 100), as described above, that may receive a plurality of images representative of an environment of the host vehicle from a camera associated with the host vehicle (e.g., at least one of image capture device 122, image capture device 124, and image capture device 126). Fig. The scene shown in Figure 12 is an example of one of the images that may be captured at time t of an environment of a host vehicle traveling in lane 1210 along a predicted trajectory 1212. The navigation system may include at least one processing device (e.g., including any of the EyeQ processors or other devices described above) specifically programmed to receive the plurality of images and analyze the images to determine an action in response to the scene. In particular, the at least one processing device may implement the sensor detection module 801, the driving policy module 803, and the control module 805, as shown in Fig. 8. The sensor detection module 801 may be responsible for collecting and outputting the image information collected by the cameras and providing this information in the form of an identified navigation state to the driving policy module 803, which may represent a trained navigation system trained by machine learning techniques such as supervised learning, reinforcement learning, etc. Based on the information about the navigation state provided to the driving policy module 803 by the sensor detection module 801, the driving policy module 803 may generate (e.g., by implementing the options graph approach described above) a desired navigation action for execution by the host vehicle in response to the identified navigation state.

[0195] In some embodiments, the at least one processing device may translate the desired navigation action directly into navigation commands using, for example, control module 805. However, in other embodiments, hard constraints may be applied such that the desired navigation action provided by driving policy module 803 is tested against various predetermined navigation constraints that may be implied by the scene and the desired navigation action. For example, where driving policy module 803 outputs a desired navigation action that would cause the host vehicle to follow trajectory 1212, that navigation action may be tested relative to one or more hard constraints associated with various aspects of the host vehicle's environment.For example, a captured image 1201 may depict a curb 1213, a pedestrian 1215, a target vehicle 1217, and a stationary object present in the scene (e.g., an overturned box). Each of these may be associated with one or more hard constraints. For example, the curb 1213 may be associated with a static constraint that prohibits the host vehicle from navigating into the curb or over the curb and onto a sidewalk 1214. The curb 1213 may also be associated with a road barrier that defines a distance (e.g., a buffer zone) extending away from the curb (e.g., by 0.1 meters, 0.25 meters, 0.5 meters, 1 meter, etc.) and along the curb, defining a no-navigation zone for the host vehicle. Of course, static constraints may also be associated with other types of roadside boundaries (e.g.,Guardrails, concrete pillars, traffic cones, poles or any other type of roadside barrier).

[0196] It should be noted that distances and ranging may be determined by any suitable methods. For example, in some embodiments, distance information may be provided by on-board radar and / or lidar systems. Alternatively or additionally, distance information may be derived from the analysis of one or more images acquired from the host vehicle's surroundings. For example, the number of pixels of a detected object represented in an image may be determined and compared to known field of view and focal length geometries of the image acquisition devices to determine scale and distances. Velocities and accelerations may be determined, for example, by detecting changes in scale between objects from image to image over known time intervals.This analysis can indicate the direction of movement toward or away from the host vehicle, along with how fast the object is passing or approaching the host vehicle. Crossing speed can be determined by analyzing the change in an object's X-coordinate position from one frame to another over known time periods.

[0197] The pedestrian 1215 may be associated with a pedestrian envelope that defines a buffer zone 1216. In some cases, an imposed hard restriction may prevent the host vehicle from moving within 1 meter of the pedestrian 1215 (in any direction relative to the pedestrian). The pedestrian 1215 may also define the location of a pedestrian influence zone 1220. Such an influence zone may be associated with a restriction that limits the speed of the host vehicle within the influence zone. The influence zone may extend 5 meters, 10 meters, 20 meters, etc., from the pedestrian 1215. Each gradation of the influence zone may be associated with a different speed limit. For example, the host vehicle may be limited to a first speed (e.g., 10 mph, 20 mph, etc.) within a zone from 1 meter to 5 meters from the pedestrian 1215.), which may be less than the speed limit in a pedestrian influence zone extending from 5 meters to 10 meters. Any gradations for the different levels of the influence zone may be used. In some embodiments, the first level may be narrower than 1 meter to 5 meters and may only extend from 1 meter to 2 meters. In other embodiments, the first level of the influence zone may extend from 1 meter (the boundary of the non-navigation zone around a pedestrian) to a distance of at least 10 meters. A second level may, in turn, extend from 10 meters to at least about 20 meters. The second level may be associated with a maximum travel speed for the host vehicle that is greater than the maximum travel speed associated with the first level of the pedestrian influence zone.

[0198] One or more stationary object constraints may also be implied by the detected scene in the environment of the host vehicle. For example, in image 1201, the at least one processing device may detect a stationary object, such as the box 1219 present on the roadway. Detected stationary objects may include various objects, such as at least one tree, a post, a road sign, or an object on a roadway. One or more predefined navigation constraints may be associated with the detected stationary object. For example, such constraints may include a shell of the stationary object, where the shell of the stationary object defines a buffer zone around the object within which navigation of the host vehicle may be prohibited. At least a portion of the buffer zone may extend a predetermined distance from an edge of the detected stationary object.For example, in the scene represented by image 1201, a buffer zone of at least 0.1 m, 0.25 m, 0.5 m or more may be associated with the box 1219 so that the host vehicle passes the box by at least a certain distance (e.g., the buffer zone distance) to the right or left to avoid a collision with the detected stationary object.

[0199] The predefined hard constraints may also include one or more target vehicle constraints. For example, a target vehicle 1217 may be detected in image 1201. To ensure that the host vehicle does not collide with the target vehicle 1217, one or more hard constraints may be employed. In some cases, a target vehicle envelope may be associated with a single buffer zone distance. For example, the buffer zone may be defined by a distance of 1 meter surrounding the target vehicle in all directions. The buffer zone may define an area extending at least one meter from the target vehicle in which the host vehicle is prohibited from navigating.

[0200] However, the envelope surrounding the target vehicle 1217 does not need to be defined by a fixed buffer distance. In some cases, the predefined hard constraints associated with target vehicles (or any other moving objects detected in the vicinity of the host vehicle) may depend on the orientation of the host vehicle relative to the detected target vehicle. For example, in some cases, a longitudinal buffer zone distance (e.g., one extending from the target vehicle toward the front or rear of the host vehicle—such as in the case where the host vehicle is traveling toward the target vehicle) may be at least one meter. A lateral buffer zone distance (e.g.,one extending from the target vehicle to both sides of the host vehicle - such as when the host vehicle is traveling in the same or opposite direction as the target vehicle, so that one side of the host vehicle passes alongside one side of the target vehicle) may be at least 0.5 meters.

[0201] As described above, other constraints may also be implied by the detection of a target vehicle or a pedestrian in the vicinity of the host vehicle. For example, the predicted trajectories of the host vehicle and the target vehicle 1217 may be considered, and where the two trajectories intersect (e.g., at intersection point 1230), a hard constraint t1l <t2a or t1l <t2a+0,5 be required, or where the host vehicle is vehicle 1 and the target vehicle 1217 is vehicle 2. Similarly, the trajectory of the pedestrian 1215 (based on a direction of travel and speed) may be monitored relative to the projected trajectory of the host vehicle. For a particular pedestrian trajectory, t(p), with each point p on the trajectory, represents the time required for the pedestrian to reach point p (i.e., point 1231 in Fig. 12). To maintain the required buffer distance of at least 1 meter from the pedestrian, either t(p) must be greater than the time required for the host vehicle to reach point p (with a sufficient time difference so that the host vehicle passes in front of the pedestrian at a distance of at least 1 meter) or t(p) must be less than the time required for the host vehicle to reach point p (e.g., if the host vehicle brakes to yield the right-of-way to the pedestrian). Nevertheless, the hard constraint in the latter example requires that the host vehicle will reach point p at a sufficient time later than the pedestrian so that the host vehicle can pass behind the pedestrian and maintain the required buffer distance of at least 1 meter.

[0202] Other hard constraints may also be employed. For example, a maximum deceleration rate of the host vehicle may be employed in at least some cases. Such a maximum deceleration rate may be determined based on a detected distance to a target vehicle following the host vehicle (e.g., using images collected by a rear-facing camera). The hard constraints may include a mandatory stop at a sensor-detected crosswalk or railroad crossing, or other applicable constraints.

[0203] Where analysis of a scene in the host vehicle's environment indicates that one or more predefined navigation constraints may be implied, these constraints may be imposed relative to one or more planned navigation actions for the host vehicle. For example, where analysis of a scene results in the driving policy module 803 returning a desired navigation action, this desired navigation action may be tested against one or more implied constraints. If it is determined that the desired navigation action violates any aspect of the implied constraints (e.g.,If the desired navigation action would bring the host vehicle within 0.7 meters of pedestrian 1215, where a predefined hard constraint requires the host vehicle to remain at least 1.0 meter away from pedestrian 1215, then at least one change may be made to the desired navigation action based on one or more predefined navigation constraints. Adjusting the desired navigation action in this manner may provide an actual navigation action for the host vehicle consistent with the constraints implied by a particular scene in the host vehicle's environment.

[0204] After determining the actual navigation action for the host vehicle, this navigation action can be implemented by initiating at least one adjustment of a navigation actuator of the host vehicle in response to the determined actual navigation action for the host vehicle. Such a navigation actuator can include at least one of a steering mechanism, a brake, or an accelerator of the host vehicle. Prioritized restrictions

[0205] As described above, a navigation system can employ various hard constraints to ensure the safe operation of a host vehicle. These constraints may include, among other things, a minimum safe driving distance from a pedestrian, a target vehicle, a guardrail, or a detected object; a maximum driving speed when passing within a zone of influence of a detected pedestrian; or a maximum deceleration rate for the host vehicle. These constraints can be imposed using a trained system based on machine learning (supervised, reinforcement, or a combination), but they can also be useful for untrained systems (e.g., those that employ algorithms to directly address expected situations occurring in scenes from a host vehicle's environment).

[0206] In both cases, there may be a hierarchy of constraints. In other words, some navigation constraints may take priority over other constraints. Therefore, if a situation arises where no navigation action is available that would result in all implied constraints being satisfied, the navigation system may determine the available navigation action that achieves the highest priority constraints first. For example, the system may cause the vehicle to yield to a pedestrian first, even if navigating to yield to the pedestrian would result in a collision with another vehicle or an object detected on the road. In another example, the system may cause the vehicle to drive onto a curb to yield to a pedestrian.

[0207] Fig. 13 provides a flowchart illustrating an algorithm for implementing a hierarchy of implied constraints based on the analysis of a scene in a host vehicle environment. For example, at step 1301, at least one processing device associated with the navigation system (e.g., an EyeQ processor, etc.) may receive a plurality of images representative of an environment of the host vehicle from a camera mounted on the host vehicle. By analyzing an image or images representative of the scene of the host vehicle environment at step 1303, a navigation state associated with the host vehicle may be identified. For example, a navigation state may indicate that the host vehicle is traveling along a two-lane road 1210, as in Fig. 12, that a target vehicle 1217 is moving through an intersection in front of the host vehicle, that a pedestrian 1215 is waiting to cross the street on which the host vehicle is traveling, that an object 1219 is present in front of the host vehicle's lane, among various other attributes of the scene.

[0208] At step 1305, one or more navigation constraints implied by the navigation state of the host vehicle may be determined. For example, after analyzing a scene in the environment of the host vehicle represented by one or more captured images, the at least one processing device may determine one or more navigation constraints implied by objects, vehicles, pedestrians, etc., detected by image analysis of the captured images. In some embodiments, the at least one processing device may determine at least a first predefined navigation constraint and a second predefined navigation constraint implied by the navigation state, and the first predefined navigation constraint may be different from the second predefined navigation constraint.For example, the first navigation constraint may relate to one or more target vehicles detected in the vicinity of the host vehicle, and the second navigation constraint may relate to a pedestrian detected in the vicinity of the host vehicle.

[0209] At step 1307, the at least one processing device may determine a priority associated with the constraints identified in step 1305. In the described example, the second predefined navigation constraint related to pedestrians may have a higher priority than the first predefined navigation constraint related to target vehicles. While priorities associated with navigation constraints may be determined or assigned based on various factors, in some embodiments, the priority of a navigation constraint may relate to its relative importance from a safety perspective. For example, while it may be important that all implemented navigation constraints are followed or met in as many situations as possible, some constraints may be associated with greater safety risks than others and may therefore be assigned higher priorities.For example, a navigation restriction requiring the host vehicle to maintain a distance of at least one meter from a pedestrian may have a higher priority than one requiring the host vehicle to maintain a distance of at least one meter from a target vehicle. This may be because a collision with a pedestrian may have more severe consequences than a collision with another vehicle. Similarly, maintaining a distance between the host vehicle and a target vehicle may have a higher priority than a restriction requiring the host vehicle to avoid a box in the road, travel at a lower speed over a speed bump, or not subject the host vehicle's occupants to more than a maximum level of acceleration.

[0210] While the driving policy module 803 is configured to maximize safety by satisfying navigation constraints implied by a particular scene or navigation state, in some situations it may be physically impossible to satisfy every implied constraint. In such situations, the priority of each implied constraint may be used to determine which of the implied constraints should be satisfied first, as shown at step 1309. Continuing the above example, in a situation where it is not possible to satisfy both the pedestrian gap constraint and the target vehicle gap constraint, but only one of the constraints can be satisfied, the higher priority of the pedestrian gap constraint may result in that constraint being satisfied before attempting to maintain a gap to the target vehicle.Therefore, in normal situations, based on the identified navigation state of the host vehicle, the at least one processing device may determine a first navigation action for the host vehicle that satisfies both the first predefined navigation constraint and the second predefined navigation constraint, where both the first predefined navigation constraint and the second predefined navigation constraint may be satisfied, as shown at step 1311. However, in other situations where not all implied constraints can be satisfied, the at least one processing device may determine a second navigation action for the host vehicle that satisfies the second predefined navigation constraint based on the identified navigation state (i.e.the higher priority constraint) but does not satisfy the first predefined navigation constraint (which has a lower priority than the second navigation constraint), where the first predefined navigation constraint and the second predefined navigation constraint cannot both be satisfied, as shown at step 1313.

[0211] Next, at step 1315, to implement the determined navigation actions for the host vehicle, the at least one processing device may cause at least one adjustment of a navigation actuator of the host vehicle in response to the determined first navigation action or the determined second navigation action for the host vehicle. As in the previous example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator. Relaxation of restrictions

[0212] As discussed above, navigation constraints may be imposed for safety purposes. The constraints may include, among other things, a minimum safe driving distance from a pedestrian, a target vehicle, a guardrail, or a detected object; a maximum driving speed when passing within a zone of influence of a detected pedestrian; or a maximum deceleration rate for the host vehicle. These constraints may be imposed in a learning or non-learning navigation system. In certain situations, these constraints may be relaxed. For example, if the host vehicle slows down near a pedestrian or stops and then continues driving slowly to convey the intention to pass the pedestrian, a pedestrian response may be detected from the captured images.If the pedestrian's response is to stand still or stop moving (and / or if eye contact with the pedestrian is detected by the sensor), the pedestrian can be interpreted as detecting an intention of the navigation system to pass the pedestrian. In such situations, the system can relax one or more predefined constraints and implement a less stringent restriction (e.g., allowing the vehicle to navigate within 0.5 meters of a pedestrian instead of within a stricter 1-meter limit).

[0213] Fig. 14 provides a flowchart for implementing control of the host vehicle based on the relaxation of one or more navigation constraints. At step 1401, the at least one processing device may receive a plurality of images representative of an environment of the host vehicle from a camera associated with the host vehicle. Analysis of the images at step 1403 may enable identification of a navigation state associated with the host vehicle. At step 1405, the at least one processor may determine navigation constraints associated with the navigation state of the host vehicle. The navigation constraints may include a first predefined navigation constraint implied by at least one aspect of the navigation state. At step 1407, analysis of the plurality of images may reveal the presence of at least one relaxation factor for the navigation constraint.

[0214] A navigation restriction relaxation factor may include any suitable indicator that one or more navigation restrictions may be suspended, modified, or otherwise relaxed in at least one aspect. In some embodiments, the at least one navigation restriction relaxation factor may include a determination (based on image analysis) that a pedestrian's eyes are looking toward the host vehicle. In such cases, it may be more safely assumed that the pedestrian is aware of the host vehicle. As a result, a higher confidence level may be obtained that the pedestrian will not take unexpected actions that cause the pedestrian to move into the path of the host vehicle. Other restriction relaxation factors may also be used.For example, the at least one navigation restriction relaxation factor may include a pedestrian determined not to be moving (e.g., one presumed to be less likely to enter a path of the host vehicle); or a pedestrian determined to be slowing down. The navigation restriction relaxation factor may also include more complicated actions, such as a pedestrian determined not to be moving after the host vehicle has stopped and then resumed movement. In such a situation, the pedestrian may be assumed to understand that the host vehicle has the right-of-way, and the pedestrian's stopping may indicate the pedestrian's intent to yield the right-of-way to the host vehicle.Other situations that may cause one or more restrictions to be relaxed include the type of curb (e.g., a low curb or one with a gradual slope might allow a relaxed distance restriction), the absence of pedestrians or other objects on the sidewalk, a vehicle with its engine not running might have a relaxed distance, or a situation where a pedestrian is looking away and / or moving away from the area toward which the host vehicle is traveling.

[0215] Where the presence of a relaxation factor for the navigation constraint is identified (e.g., at step 1407), a second navigation constraint may be determined or developed in response to the detection of the relaxation factor for the constraint. This second navigation constraint may be different from the first navigation constraint and may include at least one property that is relaxed compared to the first navigation constraint. The second navigation constraint may include a newly generated constraint based on the first constraint, wherein the newly generated constraint includes at least one modification that relaxes the first constraint in at least one respect. Alternatively, the second constraint may represent a predetermined constraint that is less stringent than the first navigation constraint in at least one respect.In some embodiments, such second constraints may be reserved for use only in situations where a constraint relaxation factor is identified in an environment of the host vehicle. Whether the second constraint is newly generated or selected from a set of fully or partially available predetermined constraints, the application of a second navigation constraint in place of a more stringent first navigation constraint (which may be applied in the absence of detection of relevant navigation constraint relaxation factors) may be referred to as constraint relaxation and may be accomplished in step 1409.

[0216] If at least one constraint relaxation factor is detected at step 1407 and at least one constraint has been relaxed at step 1409, a navigation action for the host vehicle may be determined at step 1411. The navigation action for the host vehicle may be based on the identified navigation state and satisfy the second navigation constraint. The navigation action may be implemented at step 1413 by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action.

[0217] As discussed above, the use of navigation constraints and relaxed navigation constraints can be employed with navigation systems that are trained (e.g., through machine learning) or untrained (e.g., systems programmed to respond with predetermined actions in response to specific navigation conditions). Where trained navigation systems are used, the availability of relaxed navigation constraints for specific navigation situations can represent a mode switch from a trained system response to an untrained system response. For example, a trained navigation network can determine an initial navigation action for the host vehicle based on the initial navigation constraint. However, the action taken by the vehicle may be different from the navigation action that satisfies the initial navigation constraint.Instead, the action taken may satisfy the relaxed second navigation constraint and be an action developed by an untrained system (e.g., in response to the detection of a particular condition in the host vehicle's environment, such as the presence of a relaxation factor for the navigation constraint).

[0218] There are many examples of navigation constraints that may be relaxed in response to the detection of a constraint relaxation factor in the host vehicle's environment. For example, where a predefined navigation constraint includes a buffer zone associated with a detected pedestrian, and at least a portion of the buffer zone extends at a distance from the detected pedestrian, a relaxed navigation constraint (either newly developed, retrieved from memory from a predetermined set, or generated as a relaxed version of a pre-existing constraint) may include a different or modified buffer zone. For example, the different or modified buffer zone may have a distance relative to the pedestrian that is less than the distance of the original or unmodified buffer zone relative to the detected pedestrian.As a result, the host vehicle may be allowed to navigate closer to a detected pedestrian in light of the relaxed constraint if an appropriate constraint relaxation factor is detected in the host vehicle's environment.

[0219] A relaxed characteristic of a navigation restriction may include a reduced width in a buffer zone associated with at least one pedestrian, as noted above. However, the relaxed characteristic may also include a reduced width in a buffer zone associated with a target vehicle, a detected object, a roadside guardrail, or another object detected in the vicinity of the host vehicle.

[0220] The at least one relaxed property may also include other types of modifications to the navigation restriction properties. For example, the relaxed property may include an increase in the speed associated with at least one predefined navigation restriction. The relaxed property may also include an increase in the maximum allowable deceleration / acceleration associated with at least one predefined navigation restriction.

[0221] While restrictions may be relaxed in certain situations, as described above, navigation restrictions may be expanded in others. For example, in some situations, a navigation system may determine that conditions warrant expanding a normal set of navigation restrictions. Such expansion may involve adding new restrictions to a predefined set of restrictions or adjusting one or more aspects of a predefined restriction. Adding or adjusting may result in more conservative navigation relative to the predefined set of restrictions that apply under normal driving conditions. Conditions that may justify expanding restrictions may include sensor failure, adverse environmental conditions (rain, snow, fog, or other conditions associated with limited visibility or reduced vehicle traction), etc.include.

[0222] Fig. 15 provides a flowchart for implementing control of the host vehicle based on the enhancement of one or more navigation constraints. At step 1501, the at least one processing device may receive a plurality of images representative of an environment of the host vehicle from a camera associated with the host vehicle. Analysis of the images at step 1503 may enable identification of a navigation state associated with the host vehicle. At step 1505, the at least one processor may determine navigation constraints associated with the navigation state of the host vehicle. The navigation constraints may include a first predefined navigation constraint implied by at least one aspect of the navigation state. At step 1507, analysis of the plurality of images may reveal the presence of at least one enhancement factor for the navigation constraint.

[0223] An implied navigation restriction may include any of the navigation restrictions discussed above (e.g., with respect to Fig. 12) or any other suitable navigation constraint. A navigation constraint expansion factor may include any indicator that one or more navigation constraints may be expanded / expanded in at least one aspect. The expansion or expansion of navigation constraints may be performed on a per-set basis (e.g., by adding new navigation constraints to a predetermined set of constraints) or on a per-constraint basis (e.g., by modifying a particular constraint such that the modified constraint is more restrictive than the original one, or by adding a new constraint that corresponds to a predetermined constraint, where the new constraint is more restrictive than the corresponding constraint in at least one aspect).Additionally or alternatively, the addition or extension of navigation constraints may refer to the selection from a set of predetermined constraints based on a hierarchy. For example, a set of extended constraints may be available for selection based on whether a navigation extension factor is detected in the host vehicle's environment or relative to it. Under normal conditions, where no extension factor is detected, the implied navigation constraints may be drawn from constraints applicable to normal conditions. On the other hand, where one or more extension factors for the constraint are detected, the implied constraints may be either generated from the extended constraints or predefined relative to the one or more extension factors.The enhanced restrictions may be more restrictive in at least one respect than the corresponding restrictions that apply under normal conditions.

[0224] In some embodiments, the at least one navigation constraint enhancement factor may include detecting (e.g., based on image analysis) the presence of ice, snow, or water on a surface of a road in the vicinity of the host vehicle. Such a determination may be based, for example, on detecting: reflectance areas higher than expected for dry roads (e.g., indicating ice or water on the roadway); white areas on the roadway indicating the presence of snow; shadows on the roadway consistent with the presence of longitudinal ditches (e.g., tire tracks in snow) on the roadway; water droplets or ice / snow particles on a windshield of the host vehicle; or any other suitable indicator of the presence of water or ice / snow on a surface of a roadway.

[0225] The at least one navigation limitation enhancement factor may also include the detection of particles on an exterior surface of a windshield of the host vehicle. Such particles may degrade the image quality of one or more image capture devices associated with the host vehicle. While this is described with respect to a windshield of the host vehicle, which is relevant for cameras mounted behind the windshield of the host vehicle, the detection of particles on other surfaces (e.g., a camera lens or lens cover, a headlight lens, a rear window, a taillight lens, or any other surface of the host vehicle visible to (or detected by a sensor) an image capture device associated with the host vehicle) may also indicate the presence of a navigation limitation enhancement factor.

[0226] The navigation limitation enhancement factor may also be detected as an attribute of one or more image capture devices. For example, a detected decrease in image quality of one or more images captured by an image capture device (e.g., a camera) associated with the host vehicle may also represent a navigation limitation enhancement factor. Degradation in image quality may be associated with a hardware failure or partial hardware failure associated with the image capture device or an assembly associated with the image capture device. Such degradation in image quality may also be caused by environmental conditions. For example, the presence of smoke, fog, rain, snow, etc., in the air surrounding the host vehicle may also result in reduced image quality relative to the road, pedestrians, target vehicles, etc.that may be present in the host vehicle's environment.

[0227] The navigation limitation extension factor may also relate to other aspects of the host vehicle. For example, in some situations, the navigation limitation extension factor may include a detected failure or partial failure of a system or sensor associated with the host vehicle. Such an extension factor may include, for example, the detection of a failure or partial failure of a speed sensor, GPS receiver, accelerometer, camera, radar, lidar, brakes, tires, or other system associated with the host vehicle that may impact the host vehicle's ability to navigate relative to navigation limitations associated with a navigation state of the host vehicle.

[0228] Where the presence of a navigation constraint enhancement factor is identified (e.g., at step 1507), a second navigation constraint may be determined or developed in response to the detection of the constraint enhancement factor. This second navigation constraint may be different from the first navigation constraint and may include at least one characteristic that is enhanced compared to the first navigation constraint. The second navigation constraint may be more restrictive than the first navigation constraint because the detection of a constraint enhancement factor in the environment of the host vehicle or associated with the host vehicle suggests that the host vehicle has reduced at least one navigation capability compared to normal operating conditions. Such reduced capabilities may include lower road grip (e.g.,Ice, snow, or water on the roadway; reduced tire pressure, etc.), impaired visibility (e.g., rain, snow, dust, smoke, fog, etc., which reduce the captured image quality), impaired detection capability (e.g., sensor failure or partial failure, reduced sensor performance, etc.), or any other reduction in the host vehicle's ability to navigate in response to a detected navigation condition.

[0229] If at least one enhancement factor for the constraint is detected at step 1507 and at least one constraint has been enhanced at step 1509, a navigation action for the host vehicle may be determined at step 1511. The navigation action for the host vehicle may be based on the identified navigation state and may satisfy the second (i.e., enhanced) navigation constraint. The navigation action may be implemented at step 1513 by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action.

[0230] As discussed, the use of navigation constraints and extended navigation constraints can be employed with navigation systems that are trained (e.g., through machine learning) or untrained (e.g., systems programmed to respond with predetermined actions in response to specific navigation conditions). Where trained navigation systems are used, the availability of extended navigation constraints for specific navigation situations can represent a mode switch from a trained system response to an untrained system response. For example, a trained navigation network can determine an initial navigation action for the host vehicle based on the initial navigation constraint. However, the action taken by the vehicle may be one that differs from the navigation action that satisfies the initial navigation constraint.Instead, the action taken may satisfy the extended second navigation constraint and be an action developed by an untrained system (e.g., in response to the detection of a particular condition in the host vehicle's environment, such as the presence of an extension factor for the navigation constraint).

[0231] There are many examples of navigation constraints that can be generated, augmented, or extended in response to the detection of a constraint extension factor in the host vehicle's environment. For example, if a predefined navigation constraint includes a buffer zone associated with a detected pedestrian, object, vehicle, etc., and at least a portion of the buffer zone extends a distance from the detected pedestrian / object / vehicle, an extended navigation constraint (either newly developed, retrieved from memory from a predetermined set, or generated as an extended version of a pre-existing constraint) can include a different or modified buffer zone.For example, the different or modified buffer zone may have a distance relative to the pedestrian / object / vehicle that is greater than the distance of the original or unmodified buffer zone relative to the detected pedestrian / object / vehicle. As a result, the host vehicle may be forced to navigate farther away from the detected pedestrian / object / vehicle in the face of the extended constraint if a suitable extension factor for the constraint is detected in the host vehicle's environment or relative to the host vehicle.

[0232] The at least one extended property may also include other types of modifications to the navigation restriction properties. For example, the extended property may include a reduction in the speed associated with at least one predefined navigation restriction. The extended property may also include a reduction in the maximum allowable deceleration / acceleration associated with at least one predefined navigation restriction. Navigation based on long-term planning

[0233] In some embodiments, the disclosed navigation system may not only respond to a detected navigation state in an environment of the host vehicle, but also determine one or more navigation actions based on a long-term plan. For example, the system may consider the potential impact on future navigation states of one or more navigation actions available as options for navigating relative to a detected navigation state. Considering the impact of available actions on future states may enable the navigation system to determine navigation actions based not only on a currently detected navigation state, but also based on a long-term plan.Navigation using long-term planning techniques may be particularly applicable where one or more reward functions are employed by the navigation system as a technique for selecting navigation actions from available options. Potential rewards may be analyzed relative to the available navigation actions that can be taken in response to a detected, current navigation state of the host vehicle. However, the potential rewards may also be analyzed relative to actions that can be taken in response to future navigation states resulting from the available actions for a current navigation state.As a result, the disclosed navigation system may, in some cases, select a navigation action in response to a detected navigation state, even if the selected navigation action may not provide the highest reward among the available actions that can be taken in response to the current navigation state. This may be particularly true if the system determines that the selected action may lead to a future navigation state that gives rise to one or more potential navigation actions that offer higher rewards than the selected action or, in some cases, any of the available actions relative to a current navigation state. The principle can be expressed more simply by taking a less favored action now to generate higher reward options in the future.Therefore, the revealed navigation system capable of long-term planning may choose a suboptimal short-term action when a long-term prediction suggests that a short-term loss of reward may lead to long-term reward gains.

[0234] In general, autonomous driving applications can involve a series of planning problems where the navigation system can decide on immediate actions to optimize a longer-term goal. For example, if a vehicle is confronted with a merging situation at a roundabout, the navigation system can decide on an immediate acceleration or braking command to initiate navigation into the roundabout. While the immediate response to the detected navigation condition at the roundabout may involve an acceleration or braking command in response to the detected condition, the long-term goal is a successful merging, and the long-term effect of the selected command is the success / failure of the merging. The planning problem can be approached by decomposing the problem into two phases.First, supervised learning can be applied to predict the near future based on the present (assuming the predictor is differentiable with respect to the representation of the present). Second, a complete trajectory of the agent can be modeled using a recurrent neural network, with unexplained factors modeled as (additive) input nodes. This can allow solutions to the long-term planning problem to be determined using supervised learning techniques and direct optimization via the recurrent neural network. Such an approach can also enable the learning of robust policies by incorporating adversarial elements into the environment.

[0235] Two of the most fundamental elements of autonomous driving systems are sensor sensing and planning. Sensor sensing is concerned with finding a compact representation of the current state of the environment, while planning is concerned with deciding which actions to take to optimize future goals. Supervised machine learning techniques are useful for solving sensor sensing problems. Algorithmic machine learning frameworks can also be used for the planning part, particularly reinforcement learning (RL) frameworks such as those described above.

[0236] RL can be performed in a sequence of consecutive rounds. In round t, the planner (also known as the agent or the driving policy module 803) can determine a state, s t ∈ S, which represents both the agent and the environment. He should then decide on an action a t∈ A. After performing the action, the agent receives an immediate reward, r t ∈ ℝ, and is converted into a new state, s t+1 , offset. As an example, the host vehicle may include an adaptive cruise control (ACC) system, where the vehicle should autonomously accelerate / brake to maintain an appropriate distance from a vehicle ahead while maintaining smooth driving. The state may be defined as a pair s t = (x t , v t ) ∈ ℝ 2 be modeled, where x t the distance to the vehicle in front and v t the speed of the host vehicle relative to the speed of the vehicle ahead. The action a t ∈ ℝ is the acceleration command (where the host vehicle slows down when a t < 0). The reward can be a function determined by |a t| (reflects the smoothness of the ride) and from s t (reflects that the host vehicle maintains a safe distance from the vehicle ahead). The planner's goal is to maximize the cumulative reward (perhaps up to a certain time horizon or a discounted sum of future rewards). To do so, the planner can rely on a policy π: S → A that maps a state to an action.

[0237] Supervised Learning (SL) can be considered a special case of RL, where t is sampled from a certain distribution over S and the reward function has the form r t = -ℓ(a t , y t ), where ℓ is a loss function and the learner determines the value of y t observed, which is the (possibly noisy) value of the optimal action taken when considering state s tThere can be several differences between a general RL model and a specific case of SL, and these differences can make the general RL problem more difficult.

[0238] In some SL situations, the actions (or predictions) taken by the learner may have no effect on the environment. In other words, t+1 and a t are independent. This can have two important implications. First, with SL, a sample (s1, y1), ..., (s m , y m ) must be collected in advance, and only then can the search for a policy (or predictor) with good accuracy relative to the sample begin. In contrast, in RL, the state s t+1usually depends on the action taken (and also on the previous state), which in turn depends on the policy used to generate the action. This links the data generation process with the policy learning process. Second, since the actions in SL do not affect the environment, the contribution of the choice of a t to the performance of π locally. Specifically, a t only the value of the immediate reward. In contrast, in RL, actions taken in round t can have a long-term impact on reward values ​​in future rounds.

[0239] In SL, knowing the “right” answer can y t together with the form of reward r t = -ℓ(a t , y t ) a complete knowledge of the reward for all possible choices of a t provide what the calculation of the derivative of the reward with respect to a tIn contrast, in RL, a "one-time" reward value can be anything that can be determined for a specific choice of action taken. This can be referred to as "bandit" feedback. This is one of the most important reasons for the need for "exploration" as part of long-term navigation planning, because when only "bandit" feedback is available in RL-based systems, the system may not always know whether the action taken was the best action to take.

[0240] Many RL algorithms are based, at least in part, on the mathematically elegant model of a Markov Decision Process (MDP). The Markov assumption states that the distribution of s t+1 is fully determined if s t and a tThis provides a closed-form expression for the cumulative reward of a given policy with respect to the stationary distribution over the states of the MDP. The stationary distribution of a policy can be expressed as a solution to a linear programming problem. This provides two families of algorithms: 1) optimization with respect to the primal problem, which can also be called policy search, and 2) optimization with respect to a dual problem whose variables are defined as the value function V π The value function determines the expected cumulative reward if the MDP starts from the initial state s and from there actions are chosen according to π. A related quantity is the state-action value function Q π(s,a), which determines the cumulative reward, assuming a start from state s, an immediately chosen action a, and actions chosen from there according to π. The Q-function can yield a characterization of the optimal policy (using the Bellman equation). In particular, the Q-function can show that the optimal policy is a deterministic function from S to A (in fact, it can be characterized as a "greedy" policy with respect to the optimal Q-function).

[0241] A potential advantage of the MDP model is that it allows coupling the future with the present using the Q-function. For example, assuming a vehicle is in state s, the value of Q π(s,a) specify the effect of performing action a on the future. Therefore, the Q-function can provide a local measure of the quality of an action a, making the RL problem more similar to a SL scenario.

[0242] Many RL algorithms approximate the V-function or the Q-function in one way or another. Value-based iteration algorithms, e.g., the Q-learning algorithm, can rely on the fact that the V- and Q-functions of the optimal policy can be fixed points of some operators derived from the Bellman equation. Actor-critic-policy iteration algorithms aim to learn a policy iteratively, where the "critic" is Q. πt at iteration t and the “actor” improves the policy based on this estimate.

[0243] Despite the mathematical elegance of MDPs and the convenience of switching to the Q-function representation, this approach can have several limitations. For example, an approximate notion of a Markovian behavioral state may be all that can be found in some cases. Furthermore, the transition of states may depend not only on the agent's action but also on the actions of other actors in the environment. For example, in the ACC example mentioned above, while the dynamics of the autonomous vehicle may be Markovian, the next state may depend on the behavior of the driver of the other car, which is not necessarily Markovian. One possible solution to this problem is to use partially observed MDPs, where a Markovian state is assumed to exist, but an observation distributed according to the hidden state is what can be seen.

[0244] A more direct approach can consider game-theoretic generalizations of MDPs (e.g., the framework of stochastic games). Indeed, algorithms for MDPs can be generalized to multi-agent games (e.g., minimax Q-learning or Nash Q-learning). Other approaches can involve explicit modeling of the other agents and vanishing-regret learning algorithms. Learning in a multi-agent setting can be more complex than in a single-agent setting.

[0245] A second limitation of the Q-function representation can arise from deviating from a tabular setting. The tabular setting exists when the number of states and actions is small, and Q can therefore be expressed as a table with |S| rows and |A| columns. However, if the natural representation of S and A involves Euclidean spaces and the state and action spaces are discretized, the number of states / actions can be exponential with the dimension. In such cases, it may be impractical to use a tabular representation. Instead, the Q-function can be approximated by some function from a parametric hypothesis class (e.g., neural networks of a particular architecture). For example, a deep Q-network (DQN) learning algorithm can be used. In DQN, the state space can be continuous, but the action space can remain a small discrete set.There may be approaches to dealing with continuous action spaces, but they may rely on approximating the Q-function. In any case, the Q-function can be complicated and sensitive to noise and can therefore be difficult to learn.

[0246] A different approach may be to tackle the RL problem using a recurrent neural network (RNN). In some cases, RNN can be combined with the concepts of multi-agent games and robustness to adverse environments from game theory. Furthermore, this approach may be one that does not explicitly rely on any Markovian assumptions.

[0247] In the following, an approach for navigation by planning based on prediction is described in more detail. In this approach, the state space S can be assumed to be a subset of ℝ d and the action space A is a subset of ℝk This may be a natural representation in many applications. As mentioned above, there are two key differences between RL and SL: (1) because past actions affect future rewards, information from the future may need to be propagated back into the past; and (2) the "bandit" nature of rewards can blur the dependence between (state, action) and reward, which can complicate the learning process.

[0248] As a first step in this approach, it could be noted that there are interesting problems where the bandit nature of rewards is not a problem. For example, the reward value (as will be discussed in more detail below) for the ACC application may be differentiable with respect to the current state and the current action. Indeed, even if the reward is given in a “bandit” manner, the problem of learning a differentiable function r̂(s,a) such that r̂(s t ,a t ) ≈ r tbe a relatively simple SL problem (e.g., a one-dimensional regression problem). Therefore, the first step of the approach can be to either define the reward as a function r̂(s,a) that is differentiable with respect to s and a, or to use a regression learning algorithm to learn a differentiable function r̂ that at least minimizes a regression loss over a sample, where the instance vector (s t , a t ) ∈ ℝ d × ℝ k and the target scalar r t In some situations, elements of exploration can be used to create a training set.

[0249] To address the connection between past and future, a similar idea can be used. For example, assume that a differentiable function N̂(s, a) can be learned such that N̂(s t , a t ) ≈ s t+1. Learning such a function can be characterized as an SL problem. N̂ can be viewed as a predictor for the near future. Next, a policy mapping S to A can be calculated using a parametric function π θ : S → A. Expressing π θ as a neural network can allow the expression of an episode of running the agent for T rounds using a recurrent neural network (RNN), where the next state is defined as s t+1 - N(s t , a t ) + v t is defined. Here v t ∈ ℝ d be defined by the environment and can express unpredictable aspects of the near future. The fact that s t+1 in a differentiable way from s t and a t can enable a connection between future reward values ​​and past actions. A parameter vector of the policy function πθ can be learned by backpropagation over the resulting RNN. Note that explicit probabilistic assumptions do not apply to v t In particular, there need not be a requirement for a Markov relationship. Instead, the recurrent network can be used to propagate “enough” information between past and future. Intuitively, N̂(s t , a t ) describe the predictable part of the near future, while v t can express the unpredictable aspects that may arise due to the behavior of other actors in the environment. The learning system should learn a policy that is robust to the behavior of the other actors. If ||v t|| is large, the connection between past actions and future reward values ​​may be too noisy to learn a meaningful policy. Expressing the dynamics of the system explicitly in a transparent manner can facilitate the incorporation of prior knowledge. For example, prior knowledge can simplify the problem of defining N̂.

[0250] As discussed above, the learning system can benefit from robustness relative to an adversarial environment, such as the environment of a host vehicle, which may include several other drivers who may act in unexpected ways. In a model that does not make probabilistic assumptions on v t imposed, environments can be considered in which v t is chosen in an adversarial manner. In some cases, restrictions on µ totherwise the opponent can make the planning problem difficult or even impossible. A natural restriction may be that ||µ t || is limited by a constant.

[0251] Robustness against adversarial environments can be useful in autonomous driving applications. Choosing µ t in an adversarial manner can even accelerate the learning process, as it can orient the learning system toward a robust, optimal policy. A simple game can be used to illustrate this concept. The state is s t ∈ ℝ, the action is a t ∈ ℝ, and the immediate loss function is 0.1|a t | + [|s t | - 2] +, where [x] + = max{x, 0} is the ReLU (Rectified Linear Unit) function. The next state is s t+1 = s t + a t + v t , where v t∈ [- 0.5, 0.5] for the environment is chosen in an adversarial manner. Here, the optimal policy can be written as a two-layer network with ReLU: a t = -[s t - 1.5] + + [-s t - 1.5] + . Note that if |s t | ∈ (1,5,2], the optimal action may have a larger immediate loss than the action a = 0. Therefore, the system can plan for the future and must not rely solely on the immediate loss. Note that the derivative of the loss with respect to a t 0.1 sign (a t ) and the derivative with respect to s t is 1[|s t | > 2] sign (s t ). In a situation where s t ∈ (1,5,2], the adversary choice of v would be t , v t = 0.5, and therefore there can be a non-zero loss in round t + 1 whenever a t > 1.5 - s tIn such cases, the derivation of the loss can relate directly to a t propagate back. Therefore, the adversary's choice of v t help the navigation system in cases where the choice of a t It is suboptimal to obtain a non-zero backpropagation. Such a relationship can help the navigation system select current actions based on the expectation that such a current action (even if it would result in a suboptimal reward or even a loss) will provide opportunities for more optimal actions in the future that lead to higher rewards.

[0252] Such an approach can be applied to virtually any navigation situation that may arise. The following describes the approach applied to an example: adaptive cruise control (ACC). In the ACC problem, the host vehicle may attempt to maintain a reasonable distance from a preceding target vehicle (e.g., 1.5 seconds to the target vehicle). Another objective may be to travel as smoothly as possible while maintaining the desired gap. A model representing this situation can be defined as follows. The state space is ℝ 3, and the action space is ℝ. The first coordinate of the state is the speed of the target vehicle, the second coordinate is the speed of the host vehicle, and the last coordinate is the distance between the host vehicle and the target vehicle (e.g., location of the host vehicle minus the location of the target vehicle along the road curve). The action to be taken by the host vehicle is acceleration and can be expressed as a t The size T τ can denote the time difference between consecutive rounds. While T can be adjusted to any suitable size, in an example T 0.1

[0253] seconds. The position s t can be used as st=(vttarget,vthost,xt) s t and the (unknown) acceleration of the target vehicle can be called attarget be referred to.

[0254] The entire dynamics of the system can be described by: vttarget=[vt−1target+Tat−1target]+ vthost=[vt−1host+Tat−1]+ xt=[xt−1+T(vt−1target−vt−1host)]+

[0255] This can be described as the sum of two vectors: st=([st−1[0]+Tst−1target]+[st−1[1]+Tat−1]+,[st−1[2]+T(st−1[0]−st−1[1])])=(st−1[0],[st−1[1 ]+Tat−1]+,[st−1[2]+T(st−1[0]−st−1[1])])︸N^(st−1,at)+([st−1[0]+Tat−1target]+−st−1[0],00)︸vt

[0256] The first vector is the predictable part, and the second vector is the unpredictable part. The reward in round t is defined as follows: −rt=0.1|at|+[|xt / xt*−1|−0.3]+,where xt*=max{1,1,5vthost}

[0257] The first term can lead to a penalty for non-zero accelerations, thus encouraging smooth driving. The second term depends on the ratio between the distance to the target vehicle xt and the desired distance xt* , which is defined as the maximum between a distance of 1 meter and a braking distance of 1.5 seconds. In some cases, this ratio may be exactly 1, but as long as this ratio is within [0.7, 1.3], the policy can waive any penalties, allowing the host vehicle some leeway in navigation—a property that can be important for achieving a smooth ride.

[0258] By implementing the approach outlined above, the host vehicle's navigation system (e.g., by operating the driving policy module 803 within the navigation system's processing unit 110) may select an action in response to a detected condition. The selected action may be based on analyzing not only the rewards associated with the available response actions relative to a sensor-detected navigation condition, but also on considering and analyzing future conditions, potential actions in response to the future conditions, and rewards associated with the potential actions.

[0259] Fig. 16 illustrates an algorithmic approach for navigation based on detection and long-term planning. For example, at step 1601, the at least one processing device 110 of the navigation system for the host vehicle may receive a plurality of images. These images may capture scenes representative of an environment of the host vehicle and may be provided by any of the image capture devices described above (e.g., cameras, sensors, etc.). Analysis of one or more of these images at step 1603 may enable the at least one processing device 110 to identify a present navigation state associated with the host vehicle (as described above).

[0260] At steps 1605, 1607, and 1609, various potential navigation actions may be determined in response to the sensor-detected navigation state. These potential navigation actions (e.g., a first navigation action through an Nth available navigation action) may be determined based on the sensor-detected state and the long-term goals of the navigation system (e.g., to complete a merge, smoothly follow a lead vehicle, overtake a target vehicle, avoid an object in the roadway, slow down for a detected stop sign, avoid a merging target vehicle, or any other navigation action that may advance the system's navigation goals).

[0261] For each of the determined potential navigation actions, the system may determine an expected reward. The expected reward may be determined according to any of the techniques described above and may involve analyzing a particular potential action relative to one or more reward functions. The expected rewards 1606, 1608, and 1610 may be determined for each of the potential navigation actions (e.g., the first, second, and Nth) determined in steps 1605, 1607, and 1609, respectively.

[0262] In some cases, the host vehicle's navigation system may select from the available potential actions based on values ​​associated with expected rewards 1606, 1608, and 1610 (or any other type of expected reward indicator). For example, in some situations, the action that provides the highest expected reward may be selected.

[0263] In other cases, particularly when the navigation system is engaged in long-term planning to determine navigation actions for the host vehicle, the system may not select the potential action that provides the highest expected reward. Instead, the system may look into the future to analyze whether there may be opportunities to realize higher rewards later if lower-reward actions are selected in response to a current navigation state. For example, a future state may be determined for any or all of the potential actions determined at steps 1605, 1607, and 1609. Each future state determined at steps 1613, 1615, and 1617 may represent a future navigation state based on the current navigation state determined by a respective potential action (e.g.,the potential actions determined at steps 1605, 1607 and 1609) is expected.

[0264] For each of the future states predicted at steps 1613, 1615, and 1617, one or more future actions (as navigation options available in response to the determined future state) may be determined and evaluated. At steps 1619, 1621, and 1623, for example, values ​​or any other type of indicator for expected rewards associated with one or more of the future actions may be developed (e.g., based on one or more reward functions). The expected rewards associated with one or more future actions may be evaluated by comparing values ​​of reward functions associated with each future action or by comparing other indicators associated with the expected rewards.

[0265] At step 1625, the navigation system for the host vehicle may select a navigation action for the host vehicle based on a comparison of expected rewards, not only based on the potential actions identified relative to a current navigation state (e.g., at steps 1605, 1607, and 1609), but also based on expected rewards determined as a result of potential future actions available in response to predicted future states (e.g., determined at steps 1613, 1615, and 1617). The selection at step 1625 may be based on the options and reward analysis performed at steps 1619, 1621, and 1623.

[0266] The selection of a navigation action at step 1625 may be based on a comparison of the expected rewards associated only with future action options. In such a case, the navigation system may select an action for the current state based solely on a comparison of the expected rewards resulting from actions for potential future navigation states. For example, the system may select the potential action identified at step 1605, 1607, or 1609 that is associated with the highest future reward value, as determined by the analysis at steps 1619, 1621, and 1623.

[0267] The selection of a navigation action at step 1625 may also be based solely on the comparison of the current action options (as noted above). In this situation, the navigation system may select the potential action identified at step 1605, 1607, or 1609 that is associated with the highest expected reward, 1606, 1608, or 1610. Such a selection may be made with little or no consideration of future navigation states or future expected rewards for navigation actions available in response to expected future navigation states.

[0268] On the other hand, the selection of a navigation action at step 1625 may, in some cases, be based on a comparison of the expected rewards associated with both future action options and current action options. This may, in fact, be one of the principles of navigation based on long-term planning. For example, expected rewards for future actions may be analyzed to determine if any may justify selecting a lower-reward action in response to the current navigation state in order to achieve a potentially higher reward in response to a subsequent navigation action expected to be available in response to future navigation states. As an example, a value or other indicator of expected reward 1606 may indicate a highest expected reward among rewards 1606, 1608, and 1610.On the other hand, expected reward 1608 may indicate a lowest expected reward among rewards 1606, 1608, and 1610. Instead of simply selecting the potential action determined at step 1605 (i.e., the action that produces the highest expected reward 1606), an analysis of future states, potential future actions, and future rewards may be used in making a navigation action selection at step 1625. In one example, it may be determined that a reward identified at step 1621 (in response to at least one future action to a future state determined at step 1615 based on the second potential action determined at step 1607) may be higher than expected reward 1606.Based on this comparison, the second potential action determined at step 1607 may be selected instead of the first potential action determined at step 1605, even though the expected reward 1606 is higher than the expected reward 1608. In one example, the potential navigation action determined at step 1605 may include merging in front of a detected target vehicle, while the potential navigation action determined at step 1607 may include merging behind the target vehicle.While the expected reward 1606 for merging in front of the target vehicle may be higher than the expected reward 1608 associated with merging behind the target vehicle, it may be determined that merging behind the target vehicle may result in a future state for which there may be action options that provide even higher potential rewards than the expected reward 1606, 1608, or other rewards based on available actions in response to a current, sensor-detected navigation state.

[0269] The selection from the potential actions at step 1625 may be based on a suitable comparison of the expected rewards (or any other metric or indicator of the benefit associated with one potential action versus another). In some cases, as described above, a second potential action may be selected over a first potential action if the second potential action is expected to provide at least one future action with an expected reward that is higher than the reward associated with the first potential action. In other cases, more complex comparisons may be used. For example, rewards associated with action options in response to predicted future states may be compared to more than one expected reward associated with a particular potential action.

[0270] In some scenarios, actions and expected rewards based on predicted future states may influence the selection of a potential action for a current state if at least one of the future actions is expected to yield a higher reward than any of the rewards expected as a result of the potential actions for a current state (e.g., expected rewards 1606, 1608, 1610, etc.). In some cases, the future action option that yields the highest expected reward (e.g., among the expected rewards associated with potential actions toward a sensor-detected current state, as well as among the expected rewards associated with potential future action options relative to potential future navigation states) may be used as a cue for selecting a potential action toward a current navigation state.That is, after identifying a future action option that provides the highest expected reward (or a reward above a predetermined threshold, etc.), the potential action that would result in the future state associated with the identified future action that provides the highest expected reward may be selected at step 1625.

[0271] In other cases, the selection of available actions may be made based on the determined differences between expected rewards. For example, a second potential action determined at step 1607 may be selected if a difference between an expected reward associated with a future action, determined at step 1621, and the expected reward 1606 is greater than a difference between the expected reward 1608 and the expected reward 1606 (assuming differences with + signs).In another example, a second potential action determined at step 1607 may be selected if a difference between an expected reward associated with a future action determined at step 1621 and an expected reward associated with a future action determined at step 1619 is greater than a difference between an expected reward 1608 and an expected reward 1606.

[0272] Several examples of selecting among potential actions for a current navigation state have been described. However, any other suitable comparison techniques or criteria may be used to select an available action through long-term planning based on an action and reward analysis that extends to predicted future states. Fig. 16 represents two layers in the long-term planning analysis (e.g., a first layer that considers the rewards from potential actions at a current state, and a second layer that considers the rewards from future action options in response to predicted future states), an additional analysis based on more layers may be possible. For example, instead of basing the long-term planning analysis on one or two layers, three, four, or more analysis layers could be used when selecting from available potential actions in response to a current navigation state.

[0273] After making a selection from the potential actions in response to a sensor-detected navigation state, the at least one processor may, at step 1627, initiate at least one adjustment of a navigation actuator of the host vehicle in response to the selected potential navigation action. The navigation actuator may include any suitable device for controlling at least one aspect of the host vehicle. For example, the navigation actuator may include at least one steering mechanism, a brake, or an accelerator. Navigation based on closed aggression from others

[0274] Target vehicles can be monitored by analyzing a captured image stream to determine indicators of aggressive driving behavior. Aggression is described herein as a qualitative or quantitative parameter, but other characteristics can also be used: perceived level of attention (potential driver impairment, distracted - mobile phone, asleep, etc.). In some cases, a target vehicle may be deemed to have a defensive posture, and in some cases, the target vehicle may be determined to have a more aggressive posture. Navigation actions can be selected or developed based on indicators of aggression. For example, in some cases, relative speed, relative acceleration, increases in relative acceleration, following distance, etc., relative to a host vehicle can be tracked to determine whether the target vehicle is aggressive or defensive.If it is determined that the target vehicle has an aggression level that exceeds, for example, a threshold, the host vehicle may be inclined to yield the right-of-way to the target vehicle. A target vehicle's aggression level may also be perceived based on a particular behavior of the target vehicle relative to one or more obstacles in a path or near the target vehicle (e.g., a preceding vehicle, an obstacle on the road, a traffic light, etc.).

[0275] As an introduction to this concept, an example experiment is described regarding the host vehicle merging into a roundabout, where the navigation goal is to navigate through and exit the roundabout. The situation may begin with the host vehicle approaching an entrance of the roundabout and may end with the host vehicle reaching an exit of the roundabout (e.g., the second exit). Success can be measured by whether the host vehicle maintains a safe distance from all other vehicles at all times, whether the host vehicle completes the route as quickly as possible, and whether the host vehicle adheres to a smooth acceleration policy. In this illustration, the N TTarget vehicles are randomly placed on the roundabout. To model a mix of adversary and typical behavior, a target vehicle can be modeled with probability p by an "aggressive" driving policy such that the aggressive target vehicle accelerates when the host vehicle attempts to merge in front of the target vehicle. With a probability of 1 - p, the target vehicle can be modeled with a "defensive" driving policy such that the target vehicle decelerates and allows the host vehicle to merge. In this experiment, p = 0.5, and the host vehicle's navigation system can be provided without information about the type of other drivers. The types of the other drivers can be randomly selected at the beginning of the episode.

[0276] The navigation state can be represented as the speed and position of the host vehicle (the agent) and the locations, speeds, and accelerations of the target vehicles. Maintaining target acceleration determinations can be important for distinguishing between aggressive and defensive drivers based on the current state. All target vehicles can move along a one-dimensional curve that outlines the path of the roundabout. The host vehicle can move along its own one-dimensional curve that intersects the curve of the target vehicles at the merging point, and this point is the origin of both curves. To model reasonable driving, the absolute value of the accelerations of all vehicles can be upper-bounded by a constant. Speeds can also be passed through a ReLU, since reversing is not allowed.Note that not allowing reversing may require long-term planning, as the agent cannot regret past actions.

[0277] As described above, the next state can be s t+1 into a sum of a predictable part N̂(s t , a t ) and an unpredictable part of t The expression N̂(s t , a t ), can represent the dynamics of vehicle locations and speeds (which can be well-defined in a differentiable way), while v t can represent the acceleration of the target vehicles. It can be verified that N̂(s t , a t ) can be expressed as a combination of ReLU functions via an affine transformation, therefore it is differentiable with respect to s t and a t . The vector v tcan be defined by a simulator in a non-differentiable manner and can implement aggressive behavior for some targets and defensive behavior for others. Two frames from such a simulator are shown in the Fig. 17A and Fig. 17B. In this example experiment, host vehicle 1701 learned to slow down when approaching the entrance to the roundabout. It also learned to yield to aggressive vehicles (e.g., vehicles 1703 and 1705) and to safely proceed when merging in front of defensive vehicles (e.g., vehicles 1706, 1708, and 1710). In the Fig. 17A and Fig. In the example shown in Figure 17B, the navigation system of the host vehicle 1701 is not provided with the type of target vehicles. Whether a particular vehicle is determined to be aggressive or defensive is instead determined by inference based on the detected position and acceleration of, for example, the target vehicles. Fig. 17A, the host vehicle 1701 may determine, based on position, speed, and / or relative acceleration, that the vehicle 1703 has an aggressive tendency, and therefore, the host vehicle 1701 may stop and wait for the target vehicle 1703 to pass rather than attempting to merge in front of the target vehicle 1703. In Fig. 17B, ​​however, the host vehicle 1701 has detected that the target vehicle 1710 traveling behind the vehicle 1703 has defensive tendencies (again based on the determined position, speed, and / or relative acceleration of the vehicle 1710) and has therefore completed a successful merge in front of the target vehicle 1710 and behind the target vehicle 1703.

[0278] Fig. Figure 18 is a flowchart illustrating an example algorithm for navigating a host vehicle based on predicted aggression of other vehicles. In the example of Fig. 18, an aggression level associated with at least one target vehicle may be inferred based on the determined behavior of the target vehicle relative to an object in the environment of the target vehicle. For example, at step 1801, at least one processing device (e.g., processing device 110) of the host vehicle's navigation system may receive a plurality of images representative of an environment of the host vehicle from a camera associated with the host vehicle. At step 1803, analysis of one or more of the received images may enable the at least one processor to identify a target vehicle (e.g., vehicle 1703) in the environment of the host vehicle 1701. At step 1805, analysis of one or more of the received images may enable the at least one processing device to identify at least one obstacle to the target vehicle in the environment of the host vehicle.The object may include debris on a roadway, a stoplight / traffic light, a pedestrian, another vehicle (e.g., a vehicle traveling in front of the target vehicle, a parked vehicle, etc.), a box on the roadway, a road barrier, a curb, or any other type of object that may be encountered in an environment of the host vehicle. At step 1807, analyzing one or more of the received images may enable the at least one processing device to determine at least one navigation characteristic of the target vehicle relative to the at least one identified obstacle to the target vehicle.

[0279] Various navigation characteristics can be used to infer the aggression level of a detected target vehicle in order to develop an appropriate navigation response to the target vehicle. For example, such navigation characteristics may include a relative acceleration between the target vehicle and the at least one identified obstacle, a distance of the target vehicle from the obstacle (e.g., a following distance of the target vehicle behind another vehicle), and / or a relative speed between the target vehicle and the obstacle, etc.

[0280] In some embodiments, the navigation characteristics of the target vehicles may be determined based on the outputs of sensors associated with the host vehicle (e.g., radar, speed sensors, GPS, etc.). However, in some cases, the navigation characteristics of the target vehicles may be determined partially or entirely based on the analysis of images of an environment of the host vehicle. For example, the image analysis techniques described above and, for example, in U.S. Patent No. 9,168,868, which is incorporated herein by reference, may be used to detect target vehicles within an environment of the host vehicle. Monitoring a location of a target vehicle in the captured images over time and / or monitoring locations in the captured images of one or more features associated with the target vehicle (e.g., taillights, headlights, bumper, wheels, etc.)) may enable a determination of the relative distances, velocities and / or accelerations between the target vehicles and the host vehicle or between the target vehicles and one or more other objects in an environment of the host vehicle.

[0281] An aggressiveness level of an identified target vehicle may be inferred from any suitable determined navigational characteristic of the target vehicle or any combination of determined navigational characteristics. For example, a determination of aggressiveness may be made based on any determined characteristic and one or more predetermined threshold levels, or any other suitable qualitative or quantitative analysis. In some embodiments, a target vehicle may be considered aggressive if the target vehicle is determined to be following the host vehicle or another vehicle at a distance less than a predetermined aggressive distance threshold.On the other hand, a target vehicle detected as following the host vehicle or another vehicle at a distance greater than a predetermined defensive distance threshold may be considered defensive. The predetermined aggressive distance threshold need not be the same as the predetermined defensive distance threshold. Additionally, one or both of the predetermined aggressive distance threshold and the predetermined defensive distance threshold may include a range of values ​​rather than a clear cutoff value. Furthermore, neither the predetermined aggressive distance threshold nor the predetermined defensive distance threshold need be fixed.Rather, these values ​​or ranges of values ​​may shift over time, and different thresholds / threshold ranges may be applied based on observed characteristics of a target vehicle. For example, the applied thresholds may depend on one or more other characteristics of the target vehicle. Higher observed relative velocities and / or accelerations may justify the application of larger thresholds / threshold ranges. Conversely, lower relative velocities and / or accelerations, including zero relative velocities and / or accelerations, may justify the application of smaller distance thresholds / threshold ranges when drawing the aggressive / defensive inference.

[0282] The aggressive / defensive inference can also be based on thresholds for relative speed and / or relative acceleration. A target vehicle can be considered aggressive if its detected relative speed and / or relative acceleration with respect to another vehicle exceeds a predetermined value or range. A target vehicle can be considered defensive if its detected relative speed and / or relative acceleration with respect to another vehicle falls below a predetermined value or range.

[0283] While the aggressive / defensive determination may be made based solely on a detected navigation characteristic, the determination may also depend on any combination of detected characteristics. For example, as noted above, in some cases, a target vehicle may be considered aggressive based solely on a determination that it is following another vehicle at a distance below a certain threshold or range. However, in other cases, the target vehicle may be considered aggressive if it is following another vehicle at a distance closer than a predetermined value (which may be the same as or different from the threshold applied when the determination is based solely on distance) and has a relative speed and / or relative acceleration greater than a predetermined value or range.Similarly, a target vehicle may be considered defensive based solely on a determination that it is following another vehicle at a distance greater than a certain threshold or range. However, in other cases, the target vehicle may be considered defensive if it is following another vehicle at a distance greater than a predetermined value (which may be the same as or different from the threshold applied when the determination is based on distance alone) and has a relative speed and / or relative acceleration less than a predetermined value or range. The system 100 may make an aggressive / defensive determination if, for example, a vehicle exceeds an acceleration or deceleration of 0.5 G (e.g., jerk 5 m / s). 3), a vehicle has a lateral acceleration of 0.5 G when changing lanes or cornering, a vehicle causes another vehicle to do any of the above, a vehicle changes lanes and causes another vehicle to do any of the above by a deceleration of more than 0.3 G or a jerk of 3 m / s 3 to yield the right of way and / or a vehicle changes lanes without stopping.

[0284] It should be understood that references to a quantity exceeding a range may indicate that the quantity either exceeds all values ​​associated with the range or falls within the range. Similarly, references to a quantity falling below a range may indicate that the quantity either falls below all values ​​associated with the range or falls within the range. Additionally, while the described examples of making an inference aggressively / defensively are described in terms of distance, relative acceleration, and relative speed, any other suitable quantities may be used. For example, a calculation of time to collision or any indirect indicator of distance, acceleration, and / or speed of the target vehicle may be used.It should also be noted that while the above examples focus on target vehicles relative to other vehicles, the aggressive / defensive inference can be made by determining the navigational characteristics of a target vehicle relative to any other type of obstacle (e.g., a pedestrian, a road barrier, a traffic light, debris, etc.).

[0285] With further reference to the Fig. 17A and Fig. 17B, ​​as the host vehicle 1701 approaches the roundabout, the navigation system, including its at least one processing device, may receive a stream of images from a camera associated with the host vehicle. Based on the analysis of one or more of the received images, any of the target vehicles 1703, 1705, 1706, 1708, and 1710 may be identified. Further, the navigation system may analyze the navigation characteristics of one or more of the identified target vehicles. The navigation system may determine that the gap between target vehicles 1703 and 1705 represents the first opportunity for a potential merge into the roundabout. The navigation system may analyze the target vehicle 1703 to determine aggression indicators associated with the target vehicle 1703.If the target vehicle 1703 is deemed aggressive, the host vehicle's navigation system may choose to yield to the vehicle 1703 rather than merging in front of the vehicle 1703. On the other hand, if the target vehicle 1703 is deemed defensive, the host vehicle's navigation system may attempt to complete a merging action in front of the vehicle 1703.

[0286] As host vehicle 1701 approaches the roundabout, the at least one processing device of the navigation system may analyze the captured images to determine the navigation characteristics associated with target vehicle 1703. For example, based on the images, it may be determined that vehicle 1703 is following vehicle 1705 at a distance that provides a sufficient gap for host vehicle 1701 to safely enter. In fact, it may be determined that vehicle 1703 is following vehicle 1705 at a distance that exceeds an aggressive following threshold, and therefore, based on this information, the host vehicle's navigation system may be inclined to identify target vehicle 1703 as defensive. However, in some situations, more than one navigation characteristic of a target vehicle may be analyzed to make the aggressive / defensive determination, as discussed above.By further analyzing the host vehicle's navigation system, it may determine that while the target vehicle 1703 is following the target vehicle 1705 at a non-aggressive distance, the vehicle 1703 has a relative speed and / or acceleration with respect to the vehicle 1705 that exceeds one or more thresholds associated with aggressive behavior. Indeed, the host vehicle 1701 may determine that the target vehicle 1703 is accelerating relative to the vehicle 1705 and closing the gap that exists between the vehicles 1703 and 1705. Based on further analysis of the relative speed, acceleration, and distance (and even a rate at which the gap between the vehicles 1703 and 1705 is closing), the host vehicle 1701 may determine that the target vehicle 1703 is behaving aggressively.Therefore, although there may be sufficient separation distance within which the host vehicle can navigate safely, the host vehicle 1701 may expect that merging in front of the target vehicle 1703 would result in an aggressively navigating vehicle directly behind the host vehicle. Further, based on behavior determined by image analysis or other sensor output, the target vehicle 1703 may be expected to continue accelerating toward the host vehicle 1701 or to continue traveling at a non-zero relative velocity toward the host vehicle 1701 if the host vehicle 1701 were to merge in front of the vehicle 1703. Such a situation may be undesirable from a safety perspective and may also result in discomfort to the host vehicle's passengers. For such reasons, the host vehicle 1701 may choose to yield to the vehicle 1703, as shown in FIG. Fig. 17B, ​​and to merge into the roundabout behind vehicle 1703 and in front of vehicle 1710, which is deemed defensive based on the analysis of one or more of its navigational characteristics.

[0287] With further reference to Fig. 18, the at least one processing device of the host vehicle's navigation system may, at step 1809, determine a navigation action for the host vehicle (e.g., merging in front of vehicle 1710 and behind vehicle 1703) based on the identified at least one navigation characteristic of the target vehicle relative to the identified obstacle. To implement the navigation action (at step 1811), the at least one processing device may, in response to the determined navigation action, initiate at least one adjustment of a navigation actuator of the host vehicle. For example, a brake may be applied to allow vehicle 1703 to Fig. 17A, and an accelerator may be operated together with the steering of the host vehicle's wheels to cause the host vehicle to enter the roundabout behind vehicle 1703, as in Fig. 17B shown.

[0288] As described in the examples above, the navigation of the host vehicle may be based on the navigation characteristics of a target vehicle relative to another vehicle or object. In addition, the navigation of the host vehicle may be based solely on the navigation characteristics of the target vehicle without specific reference to another vehicle or object. For example, at step 1807 of Fig. 18 The analysis of a plurality of images acquired from a host vehicle's environment enables the determination of at least one navigation characteristic of an identified target vehicle that is indicative of a level of aggression associated with the target vehicle. The navigation characteristic may include a speed, acceleration, etc., which need not be referenced with respect to another object or target vehicle to make an aggressive / defensive determination. For example, detected accelerations and / or speeds associated with a target vehicle that exceed a predetermined threshold or fall within or exceed a range of values ​​may indicate aggressive behavior.Conversely, detected accelerations and / or speeds associated with a target vehicle that fall below a predetermined threshold or fall within or exceed a range of values ​​may indicate defensive behavior.

[0289] Of course, in some cases, the detected navigation characteristic (e.g., a location, a distance, an acceleration, etc.) relative to the host vehicle may be referenced to make the aggressive / defensive determination. For example, a detected navigation characteristic of the target vehicle that indicates a level of aggression associated with the target vehicle may include an increase in the relative acceleration between the target vehicle and the host vehicle, a following distance of the target vehicle behind the host vehicle, a relative speed between the target vehicle and the host vehicle, etc. Navigation based on the limitation of accident liability

[0290] As described in the sections above, planned navigation actions can be tested against predetermined constraints to ensure compliance with certain rules. In some embodiments, this concept can be extended to considerations of potential accident liability. As discussed below, a primary goal of autonomous navigation is safety. Because absolute safety may be impossible (e.g., simply because a given host vehicle under autonomous control cannot control the other vehicles in its vicinity—it can only control its own actions), using potential accident liability as a consideration in autonomous navigation, and indeed as a constraint on planned actions, can help ensure that a given autonomous vehicle does not take actions deemed unsafe—e.g.,those for which potential accident liability could be attributed to the host vehicle. If the host vehicle only takes actions that are safe and are determined not to result in an accident through its own fault or the host vehicle's responsibility, then the desired levels of accident avoidance (e.g., less than 10) can be achieved. -9 per driving hour).

[0291] The challenges arising from most current approaches to autonomous driving include a lack of safety guarantees (or at least the inability to provide the desired levels of safety) and also a lack of scalability. Consider the problem of guaranteeing safe driving with multiple agents. Since society is unlikely to tolerate fatalities from road traffic accidents caused by machines, an acceptable level of safety is crucial for the acceptance of autonomous vehicles. While one goal may be to achieve zero accidents, this may be impossible, since multiple agents are typically involved in an accident, and one can imagine situations in which an accident is caused solely through the fault of other agents. For example, the host vehicle 1901, as in Fig. 19, on a multi-lane highway, and while the host vehicle 1901 can control its own actions relative to the target vehicles 1903, 1905, 1907, and 1909, it cannot control the actions of the surrounding target vehicles. As a result, the host vehicle 1901 may not be able to avoid a crash with at least one of the target vehicles if, for example, the vehicle 1905, on a collision course with the host vehicle, suddenly merges into the host vehicle's lane. To address this difficulty, a typical response by practitioners in the autonomous vehicle field is to resort to a statistical, data-driven approach in which safety validation becomes more stringent as more data is collected over an extended period of driving.

[0292] To understand the problem of a data-driven safety approach, however, one must first consider that the probability of a fatality caused by an accident per hour of (human) driving is known to be 10 -6 It is reasonable to assume that the fatality rate should be reduced by three orders of magnitude, to a probability of 10-9 per hour, for society to accept that machines replace humans in the task of driving. This estimate is similar to the assumed fatality rate of airbags and aviation standards. For example, 10 -9the probability that a wing will spontaneously detach from an aircraft in mid-air. However, attempts to guarantee safety using a data-driven statistical approach that provides additional confidence by accumulating miles flown are impractical. The amount of data required to determine a probability of 10 -9 deaths per driving hour is proportional to its inverse (ie 10 9Hours of data), which is roughly on the order of 30 billion miles. Furthermore, a multi-agent system interacts with its environment and likely cannot be validated offline (unless a realistic simulator is available that replicates real human driving with all its facets and complexities, such as reckless driving—but the problem of validating the simulator would be even more difficult than creating a safe autonomous vehicle agent). And any change to the planning and control software requires new data acquisition on the same order of magnitude, which is clearly unwieldy and impractical. Furthermore, developing a system from data inevitably suffers from a lack of interpretability and explainability of the actions taken—if an autonomous vehicle (AV) has a fatal accident, we need to know the reason.Consequently, a model-based safety approach is required, but existing “functional safety” and ASIL requirements in the automotive industry are not designed to deal with multi-agent environments.

[0293] A second primary challenge in developing a safe driving model for autonomous vehicles is the need for scalability. The premise underlying AVs goes beyond "building a better world" and is instead based on the premise that mobility without drivers can be sustainably secured at lower costs than with them. This premise is inevitably coupled with the notion of scalability—in the sense of supporting the mass production of AVs (in the millions) and, more importantly, supporting negligible additional costs to enable driving in a new city.Therefore, the cost of data processing and sensor acquisition is important if AVs are to be mass-produced, with the cost of validation and the ability to drive “everywhere” rather than just in a few cities also being a necessary prerequisite to ensure a sustainable business.

[0294] The problem with most current approaches lies in a brute-force mindset along three axes: (i) the required data processing density, (ii) the way high-resolution maps are defined and created, and (iii) the required sensor specifications. A brute-force approach runs counter to scalability and shifts the balance toward a future where unlimited on-board data processing is ubiquitous, where the cost of building and maintaining HD maps becomes negligible and scalable, and where exotic, highly advanced sensors are developed and produced at negligible automotive-grade costs. A future where any of the above comes to pass is certainly plausible, but all of the above is likely a low-probability event.Therefore, there is a need to provide a formal model that combines safety and scalability into an AV program that society can accept and is scalable in the sense of supporting millions of cars driving throughout the developed world.

[0295] The disclosed embodiments represent a solution that can provide the desired safety levels (or even exceed safety goals) and can also be scalable for systems involving millions of autonomous vehicles (or more). In the area of ​​safety, a model called "Responsibility-Sensitive Safety" (RSS) is introduced, which formalizes the notion of "accident fault," is interpretable and explainable, and incorporates a sense of "responsibility" into the actions of a robot agent. The definition of RSS is agnostic to the way it is implemented—which is a crucial feature to facilitate the goal of creating a convincing global safety model. RSS is motivated by the determination (as in Fig. 19) that the agents play an asymmetric role in an accident, with typically only one of the agents being responsible for the accident and therefore to be held accountable for it. The RSS model also includes a formal treatment of "careful driving" under restricted perceptual conditions, where not all agents are always visible (for example, due to occlusion). A key goal of the RSS model is to guarantee that an agent never causes an accident for which it is "at fault" or responsible. A model can only be useful if it is accompanied by an efficient policy (e.g., a function that maps the "sensor detection state" to an action) that conforms to RSS. For example, an action that appears harmless at the current moment might lead to a catastrophic event in the distant future ("butterfly effect").RSS can be useful for creating a set of local constraints for the short-term future that can guarantee (or at least virtually guarantee) that no accidents will occur in the future as a result of the host vehicle's actions.

[0296] Another contribution revolves around the introduction of a "semantic" language consisting of units, measurements, and action spaces, as well as specifications for how these are incorporated into the planning, sensor acquisition, and operation of the AV. To get a sense of semantics, consider in this context how a person taking driving lessons is instructed to think about a "driving policy." These instructions are not geometric—they do not take the form of "drive 13.7 meters at the current speed and then accelerate at a rate of 0.8 m / s." 2Instead, the instructions are semantic in nature—"follow the car in front of you" or "overtake the car on your left." The typical language of human driving guidelines deals with longitudinal and lateral goals rather than geometric units of acceleration vectors. A formal semantic language can be useful on several fronts, ranging from the computational complexity of planning, which does not scale exponentially with time and the number of agents, to the way safety and comfort interact, to the way the computation of sensor sensing is defined, and to the specification of the sensor modalities and how they interact in a fusion methodology. A fusion methodology (based on the semantic language) can ensure that the RSS model achieves the required probability of 10 -9fatalities per driving hour, all while performing an offline validation of a dataset in the order of 10 5 Hours of driving data.

[0297] For example, in a reinforcement learning environment, a Q-function (e.g., a function that evaluates the long-term quality of performing an action a∈A when the agent is in state s ∈ S; given such a Q-function, the natural choice of an action may be to take the one with the highest quality π(s) = argmax a Q(s, a)) can be defined over a semantic space in which the number of trajectories to be examined at a given time is determined by 10 4is limited. The signal-to-noise ratio in this space can be high, allowing effective machine learning approaches to successfully model the Q-function. In the case of sensor sensing computation, semantics can allow distinguishing between errors that impact safety versus errors that impact ride comfort. We define a PAC (Probably Approximate Correct - PAC) model (adapted from Valiant's PAC learning terminology) for sensor sensing that is tied to the Q-function and show how measurement errors are incorporated into planning in a way that conforms to RSS while still allowing for ride comfort optimization.The language of semantics may be important for the success of certain aspects of this model, as other standard error measurements, such as errors with respect to a global coordinate system, may not conform to the PAC sensor acquisition model. Additionally, the semantic language can be an important enabler for defining HD maps that can be created using low-bandwidth sensor data and thus crowdsourced and support scalability.

[0298] In summary, the disclosed embodiments may include a formal model that covers key components of an AV: sensing, planning, and acting. The model can help ensure that, from a planning perspective, there are no accidents caused by the AV itself. And even with a PAC sensor sensing model, the described fusion methodology may only require offline data collection at a very reasonable scale, even in the case of sensor sensing errors, to conform to the described safety model. Furthermore, the model can combine safety and scalability through the language of semantics, thus providing a complete methodology for a safe and scalable AV. Finally, it is worth noting that developing an accepted safety model adopted by industry and regulators may be a necessary condition for AV success.

[0299] The RSS model can generally follow a classic perceive-plan-act robot control method. The sensor sensing system can be responsible for understanding the current state of a host vehicle's environment. The planning part, which can be called a "driving policy" and implemented by a set of hard-coded instructions, by a trained system (e.g., a neural network), or a combination thereof, can be responsible for determining the best next step given the available options to reach a driving goal (e.g., how to change from the left lane to a right lane to exit a highway). The acting part is responsible for implementing the plan (e.g., the system of actuators and one or more controllers for steering, accelerating, and / or braking, etc., of a vehicle to implement a selected navigation action).The embodiments described below focus mainly on the sensor acquisition and planning parts.

[0300] Accidents can be due to sensor detection errors or planning errors. Planning is a multi-agent endeavor because there are other road users (humans and machines) that react to the actions of an AV. The RSS model described is designed, among other things, to address the safety of the planning part. This can be referred to as multi-agent safety. With a statistical approach, the estimation of the probability of planning errors can be done "online." After each software update, billions of miles must be driven with the new version to provide an acceptable level for estimating the frequency of planning errors. This is clearly not feasible. Alternatively, the RSS model can provide a 100% (or near-100%) guarantee that the planning module will not make errors due to the AV's fault (the term "fault" is formally defined).The RSS model can also provide an efficient means for its validation that does not rely on online testing.

[0301] Errors in a sensor detection system may be easier to validate because the sensor detection can be independent of the vehicle's actions, and therefore we can validate the probability of a serious sensor detection error using offline data. However, even collecting offline data from more than 10 9 Driving lessons are difficult. As part of the description of a disclosed sensor system, a fusion approach is described that can be validated using a significantly smaller amount of data.

[0302] The described RSS system can also be scalable to millions of cars. For example, the described semantic driving policy and the applied safety constraints can be aligned with the sensor detection and mapping requirements, which can be scaled to millions of cars even with today's technology.

[0303] A fundamental building block of such a system is a thorough safety definition, i.e., a minimum standard that AV systems must adhere to. The following technical lemma shows that a statistical approach to validating an AV system is not feasible, even for validating a simple claim such as "the system has N accidents per hour." This implies that a model-based safety definition is the only feasible tool for validating an AV system.

[0304] Lemma 1 Let X be a probability space and let A be an event such that Pr(A) = p1 < 0.1. Suppose we take m=1p1 uiv samples from X as sample, and let Z = ∑i=1m1[x∈A]. Then the following applies: Pr(Z=0)≥e−2.

[0305] Proof: We use the inequality 1 - x ≥ e -2x (proven for completeness in Appendix A.1) and obtain Pr(Z=0)=(1−p1)m≥e−2p1m=e−2.

[0306] Corollary 1 Suppose an AV system AV1 causes an accident with a small, yet insufficient probability p1. Any deterministic validation procedure given 1 / p1 samples will, with constant probability, fail to distinguish between AV1 and another AV system AV0 that never causes accidents.

[0307] To gain a perspective on the typical values ​​for such probabilities, we assume that we have an accident probability of 10 -9 per hour and a particular AV system only has a probability of 10 -8 Even if the system 10 8 driving lessons, there is a constant probability that the validation process will not be able to indicate that the system is dangerous.

[0308] Finally, it should be noted that this difficulty lies in invalidating a single, specific, dangerous AV system. A complete solution cannot be considered a single system, as new versions, bug fixes, and updates will be necessary. Every change, even to a single line of code, creates a new system from a validator's perspective. Therefore, a solution that is statistically validated must do so online using new samples after each small fix or change to account for the shift in the distribution of states detected and reached by the new system. Repeatedly and systematically obtaining such a large number of samples (and even then, with constant probability, failing to validate the system) is impractical.

[0309] Furthermore, any statistical claim must be formalized to be measured. Asserting a statistical property about the number of accidents a system has is considerably weaker than asserting "it drives safely." To do this, one must formally define what safety is. Absolute security is impossible

[0310] An action a taken by a car c can be considered absolutely safe if no accident can follow the action at a later time. It can be seen that it is impossible to achieve absolute safety by observing simple driving scenarios, for example as in Fig. 19. From the perspective of vehicle 1901, no action can ensure that none of the surrounding cars will crash into it. Solving this problem by prohibiting the autonomous car from entering such situations is also impossible. Since any highway with more than two lanes will eventually lead to this, prohibiting this scenario is tantamount to demanding that the car stay in the garage. The implications may seem disappointing at first glance. Nothing is absolutely safe. However, such a requirement for absolute safety, as defined above, might be too stringent, as can be seen from the fact that human drivers do not adhere to the requirement for absolute safety. Instead, humans behave according to a concept of safety that depends on responsibility. Responsibility-Sensitive Safety (RSS)

[0311] An important aspect that is missing from the absolute safety concept is the non-symmetry of most accidents – it is usually one of the drivers who is responsible for a collision and must bear the blame. In the example of Fig. 19, the middle car 1901 is not at fault if, for example, the left car 1909 suddenly drives into it. To formalize the fact of considering its lack of responsibility, AV 1901's behavior of staying in its own lane can be considered safe. To this end, a formal concept of "accident fault" or accident responsibility is described, which can serve as a prerequisite for a safe driving approach.

[0312] As an example, consider the simple case of two cars c f , c r , traveling one behind the other at the same speed along a straight road. Suppose c f, the car in front, suddenly brakes because of an obstacle appearing on the road and manages to avoid it. Unfortunately, c r not enough distance to c f held, is unable to react in time and crashes into the back of c f . It is clear that c r is at fault; it is the responsibility of the rear car to maintain a safe distance from the front car and to be prepared for unexpected but appropriate braking.

[0313] Next, consider a much broader family of scenarios: driving on a multi-lane road where cars can freely change lanes, merge into other cars' lanes, travel at different speeds, and so on. To simplify the following discussion, assume a straight road on a flat surface, where the lateral and longitudinal axes are the x- and y-axes, respectively. This can be achieved under mild conditions by defining a homomorphism between the actual curved road and a straight road. Additionally, consider a discrete time period. Definitions can help distinguish between two intuitively different sets of cases: simple cases where no significant lateral maneuver is performed, and more complex cases involving lateral movement.

[0314] Definition 1 (Car corridor) The corridor of a car c is the area [c x,left ,c x,right ] × [±∞], where c x,left, c x,right are the positions of the leftmost and rightmost corners of c.

[0315] Definition 2 (Merging) A car c1 (e.g. car 2003 in the Fig. 20A and Fig. 20B) enters the corridor of car c0 (e.g. car 2001 enters the Fig. 20A and Fig. 20B) at time t if it does not intersect the corridor of c0 at time t - 1 and intersects it at time t.

[0316] A further distinction can be made between the front and rear parts of the corridor. The term "the direction of a cut-in" can describe a movement toward the respective corridor boundary. These definitions can define cases with lateral movement. For the simple case where no such occurrence occurs, such as the simple case of one car following another, the safe longitudinal distance is defined:

[0317] Definition 3 (Safe longitudinal distance) A longitudinal distance 2101 ( Fig. 21) between a car c r (Car 2103) and another car c f (Car 2105), which is in the frontal corridor of c r is safe with respect to a reaction time p if at any of c f executed braking command a, |a| < a max,brake , if c r its maximum brake is applied from time p to standstill, it is not with cf will collide.

[0318] The following Lemma 2 calculates d as a function of the speeds of c r , c f , the reaction time ρ and the maximum acceleration a max,brake . Both p and a max,brake are constants that should be determined by regulation to some reasonable values. In further examples, either one of the reaction times ρ and the maximum acceleration a max,brake be set for a specific vehicle or vehicle type or can be adjusted / set according to measurements or other input parameters regarding the vehicle condition, road condition, user (e.g., driver or passenger) preferences, etc.

[0319] Lemma 2 Let c r a vehicle positioned on the longitudinal axis behind c f is located. Let a max,brake , a max,accelthe maximum braking and acceleration commands, and let ρ be the reaction time of c r . Be u r , and f be the longitudinal speeds of the cars and let l f , l r their lengths. Define u p,max = u r + ρ · a max,accel and define Tr=p+vp,maxamax,brake and Tf=vfamax,brake. Let L = (I r + I f ) / 2. Then the minimum safe longitudinal distance for c is r : dm={LL+Tf[(vp,max−vf)+ρamax,brake]−ρ2amax,brake2+(Tr−Tf)(vp,max−(Tf−ρ)amax,brake2 if Tr≤Tfotherwise

[0320] Proof: Let d t the distance at time t. To prevent an accident, the t > L for each t. To d minTo construct the speed curve, we need to find the tightest required lower bound for d0. It is clear that d0 must be at least L. As long as the two cars do not stop after T ≥ ρ seconds, the speed of the car in front is u f - T a max,brake , while the speed of c r through u p,max - (T- ρ) a max,accel is bounded upwards. The distance between the cars after T seconds is therefore bounded downwards by: dT:=d0+T2(2vf−T amax,brake)−[ρvp,max+T−ρ2(2vρ,max−(T−ρ)amax,brake)]=d0+T[(vf−vρ,max)−ρamax,brake]+ρ2amax,brake2

[0321] Note that T r the time is when c r comes to a standstill (a speed of 0) and T f is the time at which the other vehicle comes to a stop. Note that a max,brake (T r - T f ) = u p,max - and f + ρa max,brake is, so if T r ≤ Tf , it is sufficient to require that d0 > L. If T r > T f , then: dTr=d0+Tf[(vf−vρ,max)−ρamax,brake]+ρ2amax,brake2−(Tr−Tf)(vρ,max−(Tf−ρ)amax,brake)2.

[0322] Demand from d Tr > L and rearranging the terms completes the proof.

[0323] Finally, a relational operator is defined that allows comparisons with a certain notion of “margin”: when comparing lengths, speeds, and so on, it is necessary to accept very similar quantities as “equal”.

[0324] Definition 4 (µ-comparison) The µ-comparison of two numbers a, b is a >µb, if a > b + µ, a < µ b, if a < b - µ and a = µ b, if |a - b| ≤ µ.

[0325] The following comparisons (argmin, argmax, etc.) are µ-comparisons for some suitable µ. Suppose an accident occurred between cars c1 and c2. To consider who is at fault for the accident, the relevant moment to be examined is defined. This is a point in time preceding the accident and intuitively the "point of no return"; after this point, nothing could be done to prevent the accident.

[0326] Definition 5 (Fault period) The fault period of an accident is the earliest point in time preceding the accident in which: • there was an overlap between one of the cars and the corridor of the other and • the longitudinal distance was not certain.

[0327] There is clearly such a period, since both conditions apply at the time of the accident. Periods of fault can be divided into two separate categories: • Those in which there is also a cut-in, that is, they are the first moment of intersection of a car with the corridor of the other, and at an unsafe distance. • Those in which no cutting-in occurs, i.e. there was already an overlap with the corridor, at a safe longitudinal distance, and the distance had changed to an unsafe one at the time of the fault.

[0328] Definition 6 (µ-Loss due to Lateral Velocity) Suppose a cut-in occurs between vehicles c1 and c2. We say that c1 loses µ-loss due to lateral velocity if its lateral velocity, relative to the direction of the cut-in, is µ higher than that of c2.

[0329] It's worth noting that the direction of the velocity is important: For example, a velocity of -1.1 (both cars collide) is a tie. However, if the velocities are 1.1 +µ / 2, the car with the positive direction, toward the other car, is at fault. Intuitively, this definition allows us to blame a car that collides laterally with another at high speed.

[0330] Definition 7 ((µ1, µ2)-Gain by Lateral Position) Suppose a merge occurs between the vehicles c1, c2. We say that c1 wins by Lateral Position (µ1, µ2) if its lateral position with respect to the center of the merge lane (the center closest to the corridor relevant for the merge) is smaller than µ1 (as an absolute value) and smaller than that of c2 by µ2.

[0331] Intuitively, we will not blame a car if it is very close to the lane center (µ1) and much closer than the other car (by µ2).

[0332] Definition 8 (Fault) The fault or responsibility for an accident between cars c1 and c2 is a function of the state at the time of fault and is defined as follows: • If the fault time is not a merge time, the rear car is at fault. • If the fault time is also a cut-in time, both cars are at fault unless, for one of the cars, without loss of generality c1, the following two conditions apply for some predefined µ: • It does not lose lateral speed, • It wins through lateral position.

[0333] In this case, c1 is spared. In other words, if an unsafe merge occurs, both vehicles are at fault unless one of the vehicles is not (significantly) faster laterally and is (significantly) closer to the lane center. This captures the desired behavior: when following a car, a safe following distance is to be maintained, and when merging into a corridor of a car simply traveling in its own lane, only a safe following distance is to be maintained. An automated, control-based system for following the safety guidelines described above should not lead to overly defensive driving, as discussed in more detail below. Dealing with limited perception

[0334] After considering the highway example, a second example addresses the problem of limited perception. A very common human reaction when blamed for an accident falls into the category of "but I couldn't see him." This is often true. Human perception capabilities are limited, sometimes due to an unconscious decision to focus on another part of the road, sometimes due to inattention, and sometimes due to physical limitations—it's impossible to see a pedestrian hiding behind a parked car. Of these human limitations, advanced automatic sensor detection systems may be subject only to the latter: 360° visibility of the road, combined with the fact that computers are never inattentive, places them above human perception capabilities.Referring again to the example "but I couldn't see him," a suitable response is "well, you should have been more careful." To formalize what is meant by careful in relation to limited perception, consider the example in . Fig. 22. Car 2201 (c0) is attempting to merge from a parking space onto a (possibly) busy road, but cannot see if there are any cars on the road because its view is obscured by building 2203. Suppose it is a narrow urban road with a speed limit of 30 km / h. The behavior of a human driver is to slowly merge onto the road, gaining more and more field of view until they perceive that the restrictions have been removed. A significant moment in time should be defined—the first time the obscured object is revealed to us; after its revealing, it is treated as any other object that can be perceived.

[0335] Definition 9 (Reveal Time) The reveal time of an object is the first time we see it.

[0336] Definition 10 (Guilty due to unreasonable speed) Suppose that car c1 (car 2205) was traveling...

Claims

[1] A system for navigating a host vehicle, the system comprising: at least one processor comprising a circuit and a memory, wherein the memory includes instructions executable by the circuit to cause the at least one processor to: Receiving at least one image captured by an image capture device, wherein the at least one image is representative of an environment of the host vehicle; analyzing the at least one image to identify at least one property associated with the environment of the host vehicle; Determining a navigation action for the host vehicle based on: at least one property associated with the environment of the host vehicle and a steering limit corresponding to a maximum permissible lateral acceleration for the host vehicle; and Causing one or more actuators associated with the host vehicle to implement the particular navigation action. [2] The system of claim 1, wherein the at least one property associated with the environment of the host vehicle includes a property associated with a target vehicle in the environment of the host vehicle. [3] The system of claim 2, wherein the characteristic associated with the target vehicle includes a vehicle type. [4] The system of claim 2, wherein the characteristic associated with the target vehicle includes a vehicle size. [5] The system of claim 2, wherein the property associated with the target vehicle includes a vehicle model. [6] The system of claim 2, wherein the characteristic associated with the target vehicle is determined based on the analysis of the at least one image. [7] The system of claim 2, wherein the characteristic associated with the target vehicle is determined based on at least one of a LIDAR output and a RADAR output. [8] The system of claim 1, wherein the at least one property associated with the environment of the host vehicle includes a property of an object in the environment of the host vehicle. [9] The system of claim 8, wherein the property of the object is determined based on the analysis of the at least one image. [10] The system of claim 8, wherein the property of the object is determined based on at least one of a LIDAR output and a RADAR output. [11] The system of claim 1, wherein the at least one property associated with the environment of the host vehicle includes a property of a road surface in the environment of the host vehicle. [12] The system of claim 11, wherein the property of the road surface is determined based on the analysis of the at least one image. [13] The system of claim 11, wherein the property of the road surface is determined based on at least one of a LIDAR output and a RADAR output. [14] The system of claim 1, wherein the at least one characteristic includes a curve in a roadway in the vicinity of the host vehicle. [15] The system of claim 1, wherein the maximum allowable lateral acceleration is determined based on the at least one property associated with the environment of the host vehicle. [16] The system of claim 1, wherein the maximum allowable lateral acceleration is determined based on a user preference. [17] The system of claim 1, wherein the steering limit is based on at least one of a wheelbase associated with the host vehicle and a turning radius associated with the host vehicle. [18] The system of claim 1, wherein the steering limit is based on an operating parameter associated with the host vehicle. [19] The system of claim 18, wherein the operating parameter includes at least one of a load weight of the host vehicle, a number of passengers in the host vehicle, or a weight distribution within the host vehicle. [20] The system of claim 1, wherein the steering limitation is based on an indication of wheel slip obtained from the host vehicle. [21] The system of claim 20, wherein the indication of wheel slip is obtained based on the analysis of the at least one image. [22] The system of claim 1, wherein the particular navigation action includes at least one of a lane change maneuver, a merging maneuver, or a passing maneuver. [23] The system of claim 1, wherein determining the navigation action of the host vehicle includes determining a plurality of navigation actions based on at least one long-term goal for the host vehicle, a driving policy, and a safety policy, wherein the plurality of navigation actions are optimized using the driving policy to achieve the long-term goal, and wherein the plurality of navigation actions are selected according to the safety policy to ensure that the plurality of navigation actions are safe. [24] The system of claim 23, wherein selecting the plurality of navigation actions according to the safety policy to ensure that the plurality of navigation actions are safe includes checking the plurality of navigation actions to allow the host vehicle to maintain a minimum safe distance between the host vehicle and at least one object. [25] The system of claim 23, wherein the steering limit is configured such that the plurality of navigation actions does not trigger a steering operation that causes the host vehicle to exceed the maximum allowable lateral acceleration. [26] The system of claim 1, wherein the determined navigation action includes a steering maneuver, and wherein determining the navigation action for the host vehicle includes: Determining, based on the characteristic associated with the host vehicle's environment, whether implementing the steering maneuver in accordance with the steering limitation will result in the host vehicle maintaining a safe distance from at least one object in the host vehicle's environment; and If implementing the steering maneuver in accordance with the steering limitation will not result in the host vehicle maintaining the safety distance to at least one object in the vicinity of the host vehicle, refrain from initiating the implementation of the steering maneuver in accordance with the steering limitation. [27] The system of claim 26, wherein the safety distance includes at least one of a longitudinal safety distance and a lateral safety distance. [28] A non-transitory computer-readable medium storing instructions executable by at least one processor to perform a method of navigating a host vehicle, the method comprising: Receiving at least one image captured by an image capture device, wherein the at least one image is representative of an environment of the host vehicle; analyzing the at least one image to identify at least one property associated with the environment of the host vehicle; Determining a navigation action for the host vehicle based on: at least one property associated with the environment and a steering limit corresponding to a maximum permissible lateral acceleration for the host vehicle; and Causing one or more actuators associated with the host vehicle to implement the particular navigation action. [29] A method for navigating a host vehicle, the method comprising: Receiving at least one image captured by an image capture device, wherein the at least one image is representative of an environment of the host vehicle; analyzing the at least one image to identify at least one property associated with the environment of the host vehicle; Determining a navigation action for the host vehicle based on: at least one property associated with the environment and a steering limit corresponding to a maximum permissible lateral acceleration for the host vehicle; and Causing one or more actuators associated with the host vehicle to implement the particular navigation action.

Images