Steering limiter for vehicle navigation

Abstract

Systems and methods for navigating a host vehicle are provided. In one embodiment, a system may include at least one processor configured to: receive at least one image acquired by an image capture device, the at least one image representing an environment of the host vehicle; analyzing the at least one image to identify at least one characteristic associated with the environment of the host vehicle; determining a navigation action for the host vehicle based on the at least one characteristic associated with the environment of the host vehicle, and a steering limit corresponding to a maximum allowable lateral acceleration for the host vehicle; and causing one or more actuators associated with the host vehicle to perform the determined navigation action.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 327,959, filed on April 6, 2022. The foregoing application is incorporated herein by reference in its entirety. Background Art Technical Field

[0003] The present disclosure relates generally to vehicle navigation and, more particularly, to systems and methods for controlling steering maneuvers to reduce jerkiness to a host vehicle and / or passengers.

[0004] Background Information

[0005] As technology continues to advance, the goal of fully autonomous vehicles capable of navigating roadways is imminent. Autonomous vehicles may need to consider a variety of factors and, based on those factors, make appropriate decisions to safely and accurately reach their intended destination. For example, autonomous vehicles may need to process and interpret visual information (e.g., captured by cameras), information from radar or lidar, and may also use information obtained from other sources (e.g., GPS devices, speed sensors, accelerometers, suspension sensors, etc.). Furthermore, to navigate to their destination, autonomous vehicles may also need to identify their location within a specific roadway (e.g., a specific lane in a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, obey traffic signals and signs, cross from one road to another at appropriate intersections or junctions, and respond to any other situations that arise or develop during vehicle operation. Furthermore, navigation systems may need to adhere to certain imposed constraints. In some cases, these constraints may involve interactions between the host vehicle and one or more other objects (such as other vehicles, pedestrians, etc.). In other cases, the constraints may involve responsibility rules to be followed in carrying out one or more navigation actions for the host vehicle.

[0006] In the field of autonomous driving, there are two important considerations for viable autonomous vehicle systems. The first is the standardization of safety assurance, including the requirements that each self-driving car must meet to ensure safety, and how to verify these requirements. The second is scalability, as cost-incurring engineering solutions will not scale to millions of cars and could hinder widespread adoption of autonomous vehicles, or even less widespread adoption. Therefore, interpretable mathematical models for safety assurance and the design of systems that adhere to safety assurance requirements while scalable to millions of cars are needed. Summary of the Invention

[0007] Embodiments consistent with the present disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation features. For example, consistent with the disclosed embodiments, the disclosed system may include one, two, or more cameras that monitor the vehicle's environment. The disclosed system may provide a navigation response based on, for example, analysis of images captured by one or more of the cameras. The navigation response may also take into account other data, including, for example, global positioning system (GPS) data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data.

[0008] In one embodiment, a system for navigating a host vehicle may include at least one processor including circuitry and memory. The memory may include instructions executable by the circuitry to cause the at least one processor to: receive at least one image captured by an image capture device, the at least one image representing an environment of the host vehicle; analyze the at least one image to identify at least one characteristic associated with the environment of the host vehicle; determine a navigation action for the host vehicle based on: the at least one characteristic associated with the environment of the host vehicle and a steering limit corresponding to a maximum allowable lateral acceleration for the host vehicle; and cause one or more actuators associated with the host vehicle to implement the determined navigation action.

[0009] In one embodiment, a method for navigating an autonomous host vehicle may include: receiving at least one image acquired by an image capture device, the at least one image representing an environment of the host vehicle; analyzing the at least one image to identify at least one characteristic associated with the environment of the host vehicle; determining a navigation action for the host vehicle based on: the at least one characteristic associated with the environment and a steering limit corresponding to a maximum allowable lateral acceleration for the host vehicle; and causing one or more actuators associated with the host vehicle to implement the determined navigation action.

[0010] Consistent with other disclosed embodiments, a non-transitory computer-readable storage medium may store program instructions that are executable by at least one processing device to perform any of the steps and / or methods described herein.

[0011] The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various disclosed embodiments. In the drawings:

[0013] Figure 1 is a schematic representation of an exemplary system consistent with the disclosed embodiments.

[0014] Figure 2A is a schematic side view representation of an exemplary vehicle including a system consistent with the disclosed embodiments.

[0015] Figure 2B In accordance with the disclosed embodiments Figure 2A Schematic top-view representation of the vehicle and systems shown.

[0016] Figure 2C is a schematic top view representation of another embodiment of a vehicle including a system consistent with the disclosed embodiments.

[0017] Figure 2D is a schematic top view representation of yet another embodiment of a vehicle including a system consistent with the disclosed embodiments.

[0018] Figure 2E is a schematic top view representation of yet another embodiment of a vehicle including a system consistent with the disclosed embodiments.

[0019] Figure 2F is a schematic representation of an exemplary vehicle control system consistent with the disclosed embodiments.

[0020] Figure 3A is a schematic representation of the interior of a vehicle including a rearview mirror and a user interface for a vehicle imaging system, consistent with the disclosed embodiments.

[0021] Figure 3B is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and against a vehicle windshield, consistent with the disclosed embodiments.

[0022] Figure 3C From different perspectives consistent with the disclosed embodiments Figure 3B Illustration of the camera mount shown.

[0023] Figure 3D is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and against a vehicle windshield, consistent with the disclosed embodiments.

[0024] Figure 4 An exemplary block diagram of a memory configured to store instructions for performing one or more operations consistent with the disclosed embodiments.

[0025] Figure 5A A flowchart illustrating an exemplary process for eliciting one or more navigation responses based on monocular image analysis, consistent with the disclosed embodiments.

[0026] Figure 5BA flow chart illustrating an exemplary process for detecting one or more vehicles and / or pedestrians in a set of images, consistent with the disclosed embodiments.

[0027] Figure 5C A flow chart illustrating an exemplary process for detecting road markings and / or lane geometry in a set of images, consistent with the disclosed embodiments.

[0028] Figure 5D A flow chart illustrating an exemplary process for detecting traffic lights in a set of images, consistent with the disclosed embodiments.

[0029] Figure 5E A flow chart illustrating an exemplary process for eliciting one or more navigation responses based on a vehicle path, consistent with the disclosed embodiments.

[0030] Figure 5F 1 is a flow chart illustrating an example process for determining whether a leading vehicle is changing lanes, consistent with the disclosed embodiments.

[0031] Figure 6 A flow chart illustrating an exemplary process for eliciting one or more navigation responses based on stereo image analysis, consistent with the disclosed embodiments.

[0032] Figure 7 A flow chart illustrating an exemplary process for eliciting one or more navigational responses based on analysis of three sets of images, consistent with the disclosed embodiments.

[0033] Figure 8 A block diagram representation of modules that may be implemented by one or more specially programmed processing devices of a navigation system for an autonomous vehicle consistent with the disclosed embodiments.

[0034] Figure 9 A diagram of navigation options consistent with the disclosed embodiments.

[0035] Figure 10 A diagram of navigation options consistent with the disclosed embodiments.

[0036] Figure 11A 、 11B 11C provide schematic representations of navigation options for a host vehicle in a merge strip consistent with the disclosed embodiments.

[0037] Figure 11D A schematic description of a double merging scenario consistent with the disclosed embodiments is provided.

[0038] Figure 11E A diagram is provided of options that may be useful in a double merging scenario consistent with the disclosed embodiments.

[0039] Figure 12 A representative image captured of a host vehicle's environment is provided along with a diagram of potential navigation constraints consistent with the disclosed embodiments.

[0040] Figure 13 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0041] Figure 14 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0042] Figure 15 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0043] Figure 16 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0044] Figure 17A and 17B A schematic illustration of a host vehicle navigating into a roundabout is provided, consistent with the disclosed embodiments.

[0045] Figure 18 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0046] Figure 19 An example of a host vehicle driving on a multi-lane highway is shown, consistent with the disclosed embodiments.

[0047] Figure 20A and 20B An example of a vehicle cutting in front of another vehicle is shown, consistent with the disclosed embodiments.

[0048] Figure 21 An example of a vehicle following another vehicle consistent with the disclosed embodiments is shown.

[0049] Figure 22 An example of a vehicle exiting a parking lot and merging onto a potentially busy roadway is shown, consistent with the disclosed embodiments.

[0050] Figure 23 A vehicle is shown traveling on a road consistent with the disclosed embodiments.

[0051] Figures 24A to 24D Four example scenarios consistent with the disclosed embodiments are presented.

[0052] Figure 25 Example scenarios consistent with the disclosed embodiments are presented.

[0053] Figure 26Example scenarios consistent with the disclosed embodiments are presented.

[0054] Figure 27 Example scenarios consistent with the disclosed embodiments are presented.

[0055] Figure 28A and 28B An example of a scenario in which a vehicle follows another vehicle is presented, consistent with the disclosed embodiments.

[0056] Figure 29A and 29B Example attribution in a cut-through scenario consistent with the disclosed embodiments is presented.

[0057] Figure 30A and 30B Example attribution in a cut-through scenario consistent with the disclosed embodiments is presented.

[0058] Figures 31A to 31D Example attribution in a drift scenario consistent with the disclosed embodiments is presented.

[0059] Figure 32A and 32B Example attribution in a two-way traffic scenario consistent with the disclosed embodiments is presented.

[0060] Figure 33A and 33B Example attribution in a two-way traffic scenario consistent with the disclosed embodiments is presented.

[0061] Figure 34A and 34B Example attribution in a route priority scenario consistent with the disclosed embodiments is presented.

[0062] Figure 35A and 35B Example attribution in a route priority scenario consistent with the disclosed embodiments is presented.

[0063] Figure 36A and 36B Example attribution in a route priority scenario consistent with the disclosed embodiments is presented.

[0064] Figure 37A and 37B Example attribution in a route priority scenario consistent with the disclosed embodiments is presented.

[0065] Figure 38A and 38B Example attribution in a route priority scenario consistent with the disclosed embodiments is presented.

[0066] Figure 39A and 39BExample attribution in a route priority scenario consistent with the disclosed embodiments is presented.

[0067] Figure 40A and 40B Example attribution in a traffic light scenario consistent with the disclosed embodiments is presented.

[0068] Figure 41A and 41B Example attribution in a traffic light scenario consistent with the disclosed embodiments is presented.

[0069] Figure 42A and 42B Example attribution in a traffic light scenario consistent with the disclosed embodiments is presented.

[0070] Figures 43A to 43C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is presented.

[0071] Figures 44A to 44C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is presented.

[0072] Figures 45A to 45C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is presented.

[0073] Figures 46A to 46D An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is presented.

[0074] Figure 47A and 47B An example scenario is presented in which a vehicle follows another vehicle, consistent with the disclosed embodiments.

[0075] Figure 48 A flow chart illustrating an exemplary process for navigating a host vehicle consistent with the disclosed embodiments is shown.

[0076] Figures 49A to 49D An example scenario is presented in which a vehicle follows another vehicle, consistent with the disclosed embodiments.

[0077] Figure 50 A flow chart illustrating an exemplary process for braking a host vehicle consistent with the disclosed embodiments is shown.

[0078] Figure 51 A flow chart illustrating an exemplary process for navigating a host vehicle consistent with the disclosed embodiments is shown.

[0079] Figures 52A to 52D An example approach buffer zone for a host vehicle consistent with the disclosed embodiments is shown.

[0080] Figure 53A and 53B Example scenarios including proximity buffers consistent with the disclosed embodiments are presented.

[0081] Figure 54A and 54B Example scenarios including proximity buffers consistent with the disclosed embodiments are presented.

[0082] Figure 55 A flow chart is provided for selectively overriding human driver control of a host vehicle consistent with the disclosed embodiments.

[0083] Figure 56 A flow chart illustrating an exemplary process for navigating a host vehicle consistent with the disclosed embodiments is shown.

[0084] Figures 57A to 57C Example scenarios consistent with the disclosed embodiments are presented.

[0085] Figure 58 A flow chart illustrating an exemplary process for navigating a host vehicle consistent with the disclosed embodiments is shown.

[0086] Figure 59 Example scenarios for navigating a vehicle using one or more control loops consistent with the disclosed embodiments are presented.

[0087] Figure 60 A flow chart illustrating an exemplary process for generating steering commands consistent with the disclosed embodiments is shown.

[0088] Figure 61 is a schematic representation of the steering motion of a vehicle consistent with the disclosed embodiments.

[0089] Figure 62 is a block diagram of an exemplary vehicle consistent with the disclosed embodiments.

[0090] Figure 63 A flow chart illustrating an exemplary process for navigating a host vehicle consistent with the disclosed embodiments is shown.

[0091] Figure 64 An example environment is presented in which a vehicle may restrict steering motion consistent with the disclosed embodiments.

[0092] Figure 65 is a schematic representation of a vehicle's steering motion that may be limited by one or more steering limits consistent with the disclosed embodiments.

[0093] Figure 66 Example scenarios are presented in which a host vehicle may forgo enforcing steering restrictions consistent with the disclosed embodiments.

[0094] Figure 67 is a flow chart illustrating an example process for navigating a host vehicle consistent with the disclosed embodiments. DETAILED DESCRIPTION

[0095] The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and the following description to refer to the same or similar parts. Although several illustrative embodiments are described herein, modifications, adaptations, and other implementations are possible. For example, components shown in the drawings may be replaced, added to, or modified, and the illustrative methods described herein may be modified by replacing, reordering, removing, or adding steps to the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. Instead, the appropriate scope is defined by the appended claims.

[0096] Autonomous Vehicle Overview

[0097] As used throughout this disclosure, the term "autonomous vehicle" refers to a vehicle that is capable of implementing at least one navigation change without driver input. A "navigation change" refers to a change in one or more of the vehicle's steering, braking, or acceleration / deceleration. To be autonomous, a vehicle need not be fully autonomous (e.g., fully operational without a driver or without driver input). Rather, an autonomous vehicle includes a vehicle that can operate under driver control during certain time periods and without driver control during other time periods. An autonomous vehicle may also include a vehicle that controls only some aspects of vehicle navigation, such as steering (e.g., to maintain the vehicle's course between vehicle lane constraints) or some steering operations under certain conditions (but not all conditions), but may leave other aspects to the driver (e.g., braking or braking under certain conditions). In some cases, an autonomous vehicle may handle some or all aspects of the vehicle's braking, speed control, and / or steering.

[0098] Because human drivers typically rely on visual cues and observation to control their vehicles, the transportation infrastructure is constructed accordingly, with lane markings, traffic signs, and traffic lights designed to provide visual information to drivers. Given these design features of the transportation infrastructure, autonomous vehicles may include cameras and processing units that analyze visual information captured from the vehicle's environment. This visual information may include, for example, images representing components of the transportation infrastructure observable by the driver (e.g., lane markings, traffic signs, traffic lights, etc.) as well as other obstacles (e.g., other vehicles, pedestrians, debris, etc.). Autonomous vehicles may also utilize stored information, such as information that provides a model of the vehicle's environment when navigating. For example, a vehicle may use GPS data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data to provide information about its environment as it travels, and the vehicle (and other vehicles) may use this information to locate itself on the model. Some vehicles may also be capable of communicating with each other, sharing information, and alerting peer vehicles to hazards or changes in the vehicle's surroundings.

[0099] System Overview

[0100] Figure 1 is a block diagram representation of a system 100 consistent with exemplary disclosed embodiments. System 100 may include various components, depending on the requirements of a particular implementation. In some embodiments, system 100 may include a processing unit 110, an image acquisition unit 120, a location sensor 130, one or more memory units 140, 150, a map database 160, a user interface 170, and a wireless transceiver 172. Processing unit 110 may include one or more processing devices. In some embodiments, processing unit 110 may include an application processor 180, an image processor 190, or any other suitable processing device. Similarly, image acquisition unit 120 may include any number of image acquisition devices and components, depending on the requirements of a particular application. In some embodiments, image acquisition unit 120 may include one or more image capture devices (e.g., cameras, CCDs, or any other type of image sensor), such as image capture device 122, image capture device 124, and image capture device 126. System 100 may also include a data interface 128 that communicatively connects processing unit 110 to image acquisition unit 120. For example, the data interface 128 may include any one or more wired and / or wireless links for transmitting image data acquired by the image acquisition unit 120 to the processing unit 110 .

[0101] The wireless transceiver 172 may include one or more devices configured to exchange transmissions with one or more networks (e.g., cellular networks, the Internet, etc.) over an air interface using radio frequencies, infrared frequencies, magnetic fields, or electric fields. The wireless transceiver 172 may use any known standard (e.g., Wi-Fi, The host vehicle may transmit and / or receive data using a standard communication protocol (e.g., Bluetooth Smart, 802.15.4, ZigBee, etc.). Such transmissions may include communications from the host vehicle to one or more remotely located servers. Such transmissions may also include (one-way or two-way) communications between the host vehicle and one or more target vehicles in the host vehicle's environment (e.g., to facilitate coordination of the host vehicle's navigation with or in conjunction with target vehicles in the host vehicle's environment), or even broadcast transmissions to unspecified recipients in the transmitting vehicle's vicinity.

[0102] Both the application processor 180 and the image processor 190 may include various types of hardware-based processing devices. For example, either or both of the application processor 180 and the image processor 190 may include a microprocessor, a preprocessor (such as an image preprocessor), a graphics processor, a central processing unit (CPU), support circuits, a digital signal processor, an integrated circuit, memory, or any other type of device suitable for running application programs and suitable for image processing and analysis. In some embodiments, the application processor 180 and / or the image processor 190 may include any type of single-core or multi-core processor, a mobile device microcontroller, a central processing unit, etc. Various processing devices may be used, including, for example, those available from manufacturers such as and the various processing devices may include various architectures (e.g., x86 processors, wait).

[0103] In some embodiments, the application processor 180 and / or the image processor 190 may include a processor purchased from Any of the EyeQ series of processor chips. These processor designs each include multiple processing units with local memory and instruction sets. Such processors may include video inputs for receiving image data from multiple image sensors and may also include video output capabilities. In one example, Using 90nm micron technology operating at 332Mhz. The architecture consists of two floating point hyperthreaded 32-bit RISC CPUs ( cores), five Visual Compute Engines (VCEs), and three vector microcode processors Denali 64-bit mobile DDR controller, 128-bit internal Sonics interconnect, dual 16-bit video input and 18-bit video output controllers, 16-channel DMA and several peripherals. MIPS34K CPU manages five VCEs, three VMPs TM and DMA, a second MIPS34K CPU and multi-channel DMA and other peripherals. Five VCEs, three and MIPS34K CPUs can perform the intensive visual computations required for multi-function bundled applications. In another example, (It is a third generation processor and its performance is Six times of ) can be used in the disclosed embodiments. In other examples, and / or Of course, any newer or future EyeQ processing device can also be used with the disclosed embodiments.

[0104] Any processing device disclosed herein may be configured to perform certain functions. Configuring a processing device (such as any described EyeQ processor or other controller or microprocessor) to perform certain functions may include programming computer-executable instructions and making those instructions available for execution by the processing device during operation of the processing device. In some embodiments, configuring the processing device may include directly programming the processing device with architectural instructions. In other embodiments, configuring the processing device may include storing executable instructions on a memory accessible to the processing device during operation. For example, the processing device may access the memory during operation to obtain and execute the stored instructions. In either case, a processing device configured to perform the sensing, image analysis, and / or navigation functions disclosed herein represents a specialized hardware-based system that controls multiple hardware-based components of a host vehicle.

[0105] Although Figure 1 Two separate processing devices are depicted as being included in processing unit 110, but more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to perform the tasks of application processor 180 and image processor 190. In other embodiments, these tasks may be performed by more than two processing devices. Furthermore, in some embodiments, system 100 may include one or more processing units 110 without including other components, such as image acquisition unit 120.

[0106] The processing unit 110 may include various types of devices. For example, the processing unit 110 may include various devices such as a controller, an image preprocessor, a central processing unit (CPU), support circuits, a digital signal processor, an integrated circuit, memory, or any other type of device for image processing and analysis. The image preprocessor may include a video processor for capturing, digitizing, and processing images from an image sensor. The CPU may include any number of microcontrollers or microprocessors. The support circuits may include any number of circuits known in the art, including caches, power supplies, clocks, and input / output circuits. The memory may store software that, when executed by the processor, controls the operation of the system. The memory may include a database and image processing software. The memory may include any number of random access memories, read-only memories, flash memories, disk drives, optical storage devices, tape storage devices, removable storage devices, and other types of storage devices. In one embodiment, the memory may be separate from the processing unit 110. In another embodiment, the memory may be integrated into the processing unit 110.

[0107] Each memory 140, 150 may include software instructions that, when executed by a processor (e.g., application processor 180 and / or image processor 190), may control various aspects of the operation of system 100. For example, these memory units may include various databases and image processing software, as well as trained systems such as neural networks or deep neural networks. The memory units may include random access memory, read-only memory, flash memory, disk drives, optical storage devices, tape storage devices, removable storage devices, and / or any other type of storage device. In some embodiments, the memory units 140, 150 may be separate from the application processor 180 and / or image processor 190. In other embodiments, these memory units may be integrated into the application processor 180 and / or image processor 190.

[0108] Position sensor 130 may include any type of device suitable for determining a location associated with at least one component of system 100. In some embodiments, position sensor 130 may include a GPS receiver. Such a receiver can determine the user's location and velocity by processing signals broadcast by global positioning system satellites. Position information from position sensor 130 may be made available to application processor 180 and / or image processor 190.

[0109] In some embodiments, system 100 may include components such as a speed sensor (e.g., a speedometer) for measuring the speed of vehicle 200. System 100 may also include one or more accelerometers (single-axis or multi-axis) for measuring the acceleration of vehicle 200 along one or more axes.

[0110] The memory units 140, 150 may include a database indicating the locations of known landmarks or data organized in any other form. Sensory information of the environment (such as images, radar signals, depth information from lidar, or stereo processing of two or more images) may be processed together with position information (such as GPS coordinates, the vehicle's ego motion, etc.) to determine the vehicle's current location relative to the known landmarks and refine the vehicle's location. Certain aspects of this technology are included in a process called REM. TM The positioning technology is sold by the patentee of this application.

[0111] The user interface 170 may include any device suitable for providing information to or receiving input from one or more users of the system 100. In some embodiments, the user interface 170 may include user input devices, including, for example, a touch screen, a microphone, a keyboard, a pointing device, a scroll wheel, a camera, knobs, buttons, etc. Using such input devices, a user may be able to provide information input or commands to the system 100 by typing instructions or information, providing voice commands, selecting menu options on a screen (using buttons, a pointer, or eye tracking capabilities), or via any other suitable technique for communicating information to the system 100.

[0112] The user interface 170 may be equipped with one or more processing devices configured to provide information to and receive information from a user and process the information for use, for example, by the application processor 180. In some embodiments, such processing devices may execute instructions for recognizing and tracking eye movements, receiving and interpreting voice commands, recognizing and interpreting touches and / or gestures made on a touch screen, responding to keyboard inputs or menu selections, etc. In some embodiments, the user interface 170 may include a display, a speaker, a haptic device, and / or any other device for providing output information to a user.

[0113] Map database 160 may comprise any type of database for storing map data useful to system 100. In some embodiments, map database 160 may include data relating to the locations of various items in a reference coordinate system, including roads, water features, geographic features, businesses, points of interest, restaurants, gas stations, and the like. Map database 160 may store not only the locations of such items but also descriptors associated with these items, including, for example, names associated with any of the stored features. In some embodiments, map database 160 may be physically located with other components of system 100. Alternatively or additionally, map database 160 or portions thereof may be remotely located relative to other components of system 100 (e.g., processing unit 110). In such embodiments, information from map database 160 may be downloaded via a wired or wireless data connection to a network (e.g., via a cellular network and / or the internet, etc.). In some cases, map database 160 may store sparse data models, including polynomial representations of certain road features (e.g., lane markings) or target trajectories for the host vehicle. The map database 160 may also include stored representations of various recognized landmarks that may be used to determine or update the known position of the host vehicle relative to the target trajectory. The landmark representation may include data fields such as landmark type, landmark location, and other possible identifiers.

[0114] Image capture devices 122, 124, and 126 may each include any type of device suitable for capturing at least one image from the environment. Furthermore, any number of image capture devices may be used to acquire images for input to the image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. Figures 2B to 2E Image capture devices 122, 124, and 126 are further described.

[0115] One or more cameras (e.g., image capture devices 122, 124, and 126) may be part of a sensing block included on the vehicle. Various other sensors may be included in the sensing block, and any or all of these sensors may be relied upon to develop the vehicle's sensed navigational state. In addition to cameras (forward, side, rear, etc.), the sensing block may also include other sensors such as radar, lidar, and acoustic sensors. Furthermore, the sensing block may include one or more components configured to transmit and / or receive information related to the vehicle's environment. For example, such components may include a wireless transceiver (RF, etc.) that can receive sensor-based information or any other type of information related to the host vehicle's environment from a source remotely located relative to the host vehicle. Such information may include sensor output information or related information received from vehicle systems external to the host vehicle. In some embodiments, such information may include information received from a remote computing device, a centralized server, or the like. Furthermore, the cameras may be configured in many different ways: single-camera unit, multiple cameras, camera clusters, long field of view (FOV), short field of view (FOV), wide angle, fisheye, and the like.

[0116] The system 100 or its various components may be incorporated into a variety of different platforms. In some embodiments, the system 100 may be included on a vehicle 200, such as Figure 2A For example, vehicle 200 may be equipped with processing unit 110 and any other components of system 100, as described above with respect to Figure 1 While in some embodiments, the vehicle 200 may be equipped with only a single image capture device (e.g., a camera), in other embodiments (such as a combination of Figures 2B to 2E In those discussed above, multiple image capture devices may be used. For example, Figure 2A As shown, either of the image capture devices 122 and 124 of vehicle 200 may be part of an ADAS (Advanced Driver Assistance System) imaging set.

[0117] The image capture device included on the vehicle 200 as part of the image acquisition unit 120 may be located at any suitable location. In some embodiments, such as Figures 2A to 2E and Figures 3A to 3C As shown, the image capture device 122 can be located in the vicinity of the rearview mirror. This location can provide a line of sight similar to the line of sight of the driver of the vehicle 200, which can assist in determining what is and is not visible to the driver. The image capture device 122 can be positioned anywhere near the rearview mirror, but placing the image capture device 122 on the driver's side of the mirror can further assist in obtaining an image representative of the driver's field of view and / or line of sight.

[0118] Other locations for the image capture devices of image acquisition unit 120 may also be used. For example, image capture device 124 may be located on or in the bumper of vehicle 200. Such locations may be particularly suitable for image capture devices with a wide field of view. The line of sight of an image capture device located on the bumper may differ from the driver's line of sight, and therefore the bumper image capture device and the driver may not always see the same objects. Image capture devices (e.g., image capture devices 122, 124, and 126) may also be located in other locations. For example, the image capture device may be located on or in one or both of the side mirrors of vehicle 200, on the roof of vehicle 200, on the hood of vehicle 200, on the trunk of vehicle 200, on the side of vehicle 200, mounted on, positioned behind, or positioned in front of any of the windows of vehicle 200, mounted in or near light fixtures on the front and / or rear of vehicle 200, etc.

[0119] In addition to the image capture device, the vehicle 200 may also include various other components of the system 100. For example, the processing unit 110 may be included on the vehicle 200, or integrated with or separate from the vehicle's engine control unit (ECU). The vehicle 200 may also be equipped with a location sensor 130, such as a GPS receiver, and may also include a map database 160 and memory units 140 and 150.

[0120] As previously discussed, the wireless transceiver 172 can transmit and / or receive data via one or more networks (e.g., a cellular network, the Internet, etc.). For example, the wireless transceiver 172 can upload data collected by the system 100 to one or more servers and download data from the one or more servers. Via the wireless transceiver 172, the system 100 can receive, for example, periodic or on-demand updates to data stored in the map database 160, the memory 140, and / or the storage 150. Similarly, the wireless transceiver 172 can upload any data from the system 100 (e.g., images captured by the image acquisition unit 120, data received by the position sensor 130 or other sensors, the vehicle control system, etc.) and / or any data processed by the processing unit 110 to one or more servers.

[0121] The system 100 may upload data to a server (e.g., to the cloud) based on a privacy level setting. For example, the system 100 may implement privacy level settings to regulate or limit the type of data (including metadata) sent to the server that may uniquely identify the vehicle and / or the driver / owner of the vehicle. Such settings may be set by a user via, for example, the wireless transceiver 172, by factory default settings, or by data received by the wireless transceiver 172.

[0122] In some embodiments, the system 100 may upload data according to a "high" privacy level, and under the set setting, the system 100 may transmit data (e.g., location information related to a route, captured images, etc.) without any details about a specific vehicle and / or driver / owner. For example, when uploading data according to the "high" privacy setting, the system 100 may not include the vehicle identification number (VIN) or the name of the driver or owner of the vehicle, and instead may transmit data such as captured images and / or restricted location information related to a route.

[0123] Other privacy levels are also contemplated. For example, the system 100 may transmit data to a server according to a "medium" privacy level and include additional information not included at a "high" privacy level, such as the make and / or model of the vehicle and / or the type of vehicle (e.g., passenger car, sport utility vehicle, truck, etc.). In some embodiments, the system 100 may upload data according to a "low" privacy level. At the "low" privacy level setting, the system 100 may upload data and include information sufficient to uniquely identify a particular vehicle, owner / driver, and / or part or all of the route traveled by the vehicle. Such "low" privacy level data may include, for example, one or more of the VIN, the driver / owner's name, the vehicle's starting point prior to departure, the vehicle's intended destination, the vehicle's make and / or model, the vehicle's type, etc.

[0124] Figure 2A is a schematic side view representation of an exemplary vehicle imaging system consistent with the disclosed embodiments. Figure 2B for Figure 2A A schematic top view of the embodiment shown is shown. Figure 2B As shown, the disclosed embodiments may include a vehicle 200 including a system 100 in its body having a first image capture device 122 positioned in the vicinity of a rearview mirror and / or near a driver of the vehicle 200, a second image capture device 124 positioned on or in a bumper area (e.g., one of the bumper areas 210) of the vehicle 200, and a processing unit 110.

[0125] like Figure 2C As shown, both image capture devices 122 and 124 may be positioned in the vicinity of a rearview mirror and / or near the driver of vehicle 200. Figure 2B and 2C Two image capture devices 122 and 124 are shown in FIG, but it should be understood that other embodiments may include more than two image capture devices. Figure 2D and 2E In the illustrated embodiment, first, second, and third image capture devices 122 , 124 , and 126 are included in the system 100 of a vehicle 200 .

[0126] like Figure 2D As shown, image capture device 122 may be positioned in the vicinity of a rearview mirror and / or near the driver of vehicle 200, and image capture devices 124 and 126 may be positioned on or in a bumper area (e.g., one of bumper areas 210) of vehicle 200. Figure 2E As shown, image capture devices 122, 124, and 126 may be positioned in the vicinity of rearview mirrors and / or near the driver's seat of vehicle 200. The disclosed embodiments are not limited to any particular number and configuration of image capture devices, and the image capture devices may be positioned in any suitable location within and / or on vehicle 200.

[0127] It should be understood that the disclosed embodiments are not limited to vehicles and may be applied in other contexts. It should also be understood that the disclosed embodiments are not limited to a specific type of vehicle 200 and may be applicable to all types of vehicles, including cars, trucks, trailers, and other types of vehicles.

[0128] The first image capture device 122 may include any suitable type of image capture device. The image capture device 122 may include an optical axis. In one embodiment, the image capture device 122 may include an Aptina M9V024WVGA sensor with a global shutter. In other embodiments, the image capture device 122 may provide a resolution of 1280x960 pixels and may include a rolling shutter. The image capture device 122 may include various optical elements. In some embodiments, one or more lenses may be included, for example, to provide a desired focal length and field of view for the image capture device. In some embodiments, the image capture device 122 may be associated with a 6 mm lens or a 12 mm lens. In some embodiments, the image capture device 122 may be configured to capture an image having a desired field of view (FOV) 202, such as Figure 2DAs shown. For example, the image capture device 122 may be configured to have a regular FOV, such as in the range of 40 degrees to 56 degrees, including a 46 degree FOV, a 50 degree FOV, a 52 degree FOV, or larger. Alternatively, the image capture device 122 may be configured to have a narrow FOV in the range of 23 degrees to 40 degrees, such as a 28 degree FOV or a 36 degree FOV. In addition, the image capture device 122 may be configured to have a wide FOV in the range of 100 degrees to 180 degrees. In some embodiments, the image capture device 122 may include a wide-angle bumper camera or a camera with a FOV of up to 180 degrees. In some embodiments, the image capture device 122 may be a 7.2M pixel image capture device with an aspect ratio of approximately 2:1 (e.g., HxV=3800x1900 pixels) with a horizontal FOV of approximately 100 degrees. Such an image capture device may be used to replace a three-image capture device configuration. Due to significant lens distortion, in embodiments where the image capture device uses a radially symmetric lens, the vertical FOV of such an image capture device may be significantly less than 50 degrees. For example, such a lens may not be radially symmetric, which would allow for a vertical FOV greater than 50 degrees and a horizontal FOV of 100 degrees.

[0129] The first image capture device 122 can capture a plurality of first images relative to a scene associated with the vehicle 200. Each of the plurality of first images can be captured as a series of image scan lines, which can be captured using a rolling shutter. Each scan line can include a plurality of pixels.

[0130] The first image capture device 122 may have a scan rate associated with the acquisition of each of the first series of image scan lines. The scan rate may refer to the rate at which the image sensor may acquire image data associated with each pixel included in a particular scan line.

[0131] For example, image capture devices 122, 124, and 126 may include any suitable type and number of image sensors, including CCD sensors or CMOS sensors. In one embodiment, a CMOS image sensor may be employed with a rolling shutter, such that each pixel in a row is read one at a time, and the rows are scanned on a row-by-row basis until the entire image frame has been captured. In some embodiments, the rows may be captured sequentially from top to bottom relative to the frame.

[0132] In some embodiments, one or more of the image capture devices disclosed herein (e.g., image capture devices 122, 124, and 126) may constitute a high-resolution imager and may have a resolution greater than 5 M pixels, 7 M pixels, 10 M pixels, or greater.

[0133] The use of a rolling shutter can cause pixels in different rows to be exposed and captured at different times, which can result in skew and other image artifacts in the captured image frame. On the other hand, when image capture device 122 is configured to operate with a global or synchronized shutter, all pixels can be exposed for the same amount of time and during a common exposure period. Therefore, the image data in a frame collected from a system employing a global shutter represents a snapshot of the entire FOV (such as FOV 202) at a specific time. In contrast, in a rolling shutter application, each row in the frame is exposed and the data is captured at a different time. As a result, moving objects may appear distorted in an image capture device with a rolling shutter. This phenomenon is described in more detail below.

[0134] Second image capture device 124 and third image capture device 126 may be any type of image capture device. Similar to first image capture device 122, each of image capture devices 124 and 126 may include an optical axis. In one embodiment, each of image capture devices 124 and 126 may include an Aptina M9V024 WVGA sensor with a global shutter. Alternatively, each of image capture devices 124 and 126 may include a rolling shutter. Similar to image capture device 122, image capture devices 124 and 126 may be configured to include various lenses and optical elements. In some embodiments, the lenses associated with image capture devices 124 and 126 may provide a FOV (such as FOVs 204 and 206) that is the same as or narrower than the FOV associated with image capture device 122 (such as FOV 202). For example, image capture devices 124 and 126 may have a FOV of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less.

[0135] Image capture devices 124 and 126 may capture a plurality of second and third images relative to a scene associated with vehicle 200. Each of the plurality of second and third images may be captured as second and third series of image scan lines, which may be captured using a rolling shutter. Each scan line or row may have a plurality of pixels. Image capture devices 124 and 126 may have second and third scan rates associated with the capture of each of the image scan lines included in the second and third series.

[0136] Each image capture device 122, 124, and 126 may be positioned at any suitable location and orientation relative to vehicle 200. The relative positioning of image capture devices 122, 124, and 126 may be selected to facilitate fusing information acquired from the image capture devices. For example, in some embodiments, the FOV associated with image capture device 124 (such as FOV 204) may partially or completely overlap with the FOV associated with image capture device 122 (such as FOV 202) and the FOV associated with image capture device 126 (such as FOV 206).

[0137] Image capture devices 122, 124, and 126 may be located at any suitable relative heights on vehicle 200. In one embodiment, there may be height differences between image capture devices 122, 124, and 126, which may provide sufficient parallax information to enable stereo analysis. Figure 2A As shown, the two image capture devices 122 and 124 are at different heights. For example, there may also be lateral displacement differences between the image capture devices 122, 124, and 126, thereby providing additional parallax information for stereo analysis by the processing unit 110. The difference in lateral displacement may be denoted as d x ,like Figure 2C and 2D In some embodiments, there may be a forward or backward displacement (e.g., a range displacement) between image capture devices 122, 124, and 126. For example, image capture device 122 may be positioned 0.5 meters to 2 meters or more behind image capture device 124 and / or image capture device 126. This type of displacement may enable one of the image capture devices to cover a potential blind spot of the other image capture devices.

[0138] Image capture device 122 may have any suitable resolution capability (e.g., the number of pixels associated with the image sensor), and the resolution of the image sensor associated with image capture device 122 may be higher, lower, or the same as the resolution of the image sensors associated with image capture devices 124 and 126. In some embodiments, the image sensors associated with image capture device 122 and / or image capture devices 124 and 126 may have a resolution of 640 x 480, 1024 x 768, 1280 x 960, or any other suitable resolution.

[0139] The frame rate (e.g., the rate at which an image capture device acquires a set of pixel data for one image frame before continuing to capture pixel data associated with the next image frame) may be controllable. The frame rate associated with image capture device 122 may be higher, lower, or the same as the frame rates associated with image capture devices 124 and 126. The frame rates associated with image capture devices 122, 124, and 126 may depend on a variety of factors that may affect the timing of the frame rates. For example, one or more of image capture devices 122, 124, and 126 may include a selectable pixel delay period that is applied before or after the acquisition of image data associated with one or more pixels of the image sensors in image capture devices 122, 124, and / or 126. Generally speaking, image data corresponding to each pixel may be acquired according to a clock rate for the device (e.g., one pixel per clock cycle). Additionally, in embodiments including a rolling shutter, one or more of image capture devices 122, 124, and 126 may include a selectable horizontal blanking period applied before or after the acquisition of image data associated with rows of pixels of image sensors in image capture devices 122, 124, and / or 126. Furthermore, one or more of image capture devices 122, 124, and / or 126 may include a selectable vertical blanking period applied before or after the acquisition of image data associated with image frames of image capture devices 122, 124, and 126.

[0140] These timing controls may enable synchronization of the frame rates associated with image capture devices 122, 124, and 126, even when the line scan rate of each image capture device is different. Additionally, as will be discussed in greater detail below, these selectable timing controls, among other factors (e.g., image sensor resolution, maximum line scan rate, etc.), may enable synchronization of image capture from areas where the FOV of image capture device 122 overlaps with one or more of the FOVs of image capture devices 124 and 126, even when the field of view of image capture device 122 is different from the FOVs of image capture devices 124 and 126.

[0141] The frame rate timing in image capture devices 122, 124, and 126 may depend on the resolution of the associated image sensors. For example, assuming similar line scan rates for both devices, if one device includes an image sensor with a resolution of 640 x 480 and the other device includes an image sensor with a resolution of 1280 x 960, it will take more time to acquire a frame of image data from the sensor with the higher resolution.

[0142] Another factor that may affect the timing of image data acquisition in image capture devices 122, 124, and 126 is the maximum line scan rate. For example, acquiring a line of image data from the image sensors included in image capture devices 122, 124, and 126 will require some minimum amount of time. Assuming no pixel delay period is added, this minimum amount of time for acquiring a line of image data will be related to the maximum line scan rate for the particular device. Devices with higher maximum line scan rates may be able to provide higher frame rates than devices with lower maximum line scan rates. In some embodiments, one or more of image capture devices 124 and 126 may have a maximum line scan rate that is higher than the maximum line scan rate associated with image capture device 122. In some embodiments, the maximum line scan rate of image capture devices 124 and / or 126 may be 1.25 times, 1.5 times, 1.75 times, or 2 times, or more, the maximum line scan rate of image capture device 122.

[0143] In another embodiment, image capture devices 122, 124, and 126 may have the same maximum line scan rate, but image capture device 122 may be operated at a scan rate less than or equal to its maximum scan rate. The system may be configured such that one or more of image capture devices 124 and 126 operate at a line scan rate equal to the line scan rate of image capture device 122. In other cases, the system may be configured such that the line scan rate of image capture device 124 and / or image capture device 126 may be 1.25 times, 1.5 times, 1.75 times, or 2 times, or more, the line scan rate of image capture device 122.

[0144] In some embodiments, image capture devices 122, 124, and 126 may be asymmetric. That is, they may include cameras with different fields of view (FOVs) and focal lengths. For example, the fields of view of image capture devices 122, 124, and 126 may include any desired area relative to the environment of vehicle 200. In some embodiments, one or more of image capture devices 122, 124, and 126 may be configured to acquire image data from the environment in front of vehicle 200, behind vehicle 200, to the sides of vehicle 200, or a combination thereof.

[0145] Furthermore, the focal length associated with each image capture device 122, 124, and / or 126 may be selectable (e.g., by including an appropriate lens, etc.) so that each device captures images of objects at a desired range of distances relative to the vehicle 200. For example, in some embodiments, the image capture devices 122, 124, and 126 may capture images of close-up objects within a few meters from the vehicle. The image capture devices 122, 124, and 126 may also be configured to capture images of objects at greater ranges from the vehicle (e.g., 25 m, 50 m, 100 m, 150 m, or more). In addition, the focal lengths of image capture devices 122, 124, and 126 may be selected such that one image capture device (e.g., image capture device 122) may capture images of objects that are relatively close to the vehicle (e.g., within 10 m or within 20 m), while other image capture devices (e.g., image capture devices 124 and 126) may capture images of objects that are farther from the vehicle 200 (e.g., greater than 20 m, 50 m, 100 m, 150 m, etc.).

[0146] According to some embodiments, the FOV of one or more image capture devices 122, 124, and 126 may have a wide angle. For example, having a FOV of 140 degrees may be advantageous, particularly for image capture devices 122, 124, and 126 that may be used to capture images of areas in the vicinity of vehicle 200. For example, image capture device 122 may be used to capture images of areas to the right or left of vehicle 200, and in such embodiments, it may be desirable for image capture device 122 to have a wide FOV (e.g., at least 140 degrees).

[0147] The field of view associated with each of image capture devices 122, 124, and 126 may depend on the respective focal length. For example, as the focal length increases, the corresponding field of view decreases.

[0148] Image capture devices 122, 124, and 126 may be configured to have any suitable field of view. In one specific example, image capture device 122 may have a horizontal FOV of 46 degrees, image capture device 124 may have a horizontal FOV of 23 degrees, and image capture device 126 may have a horizontal FOV between 23 and 46 degrees. In another instance, image capture device 122 may have a horizontal FOV of 52 degrees, image capture device 124 may have a horizontal FOV of 26 degrees, and image capture device 126 may have a horizontal FOV between 26 and 52 degrees. In some embodiments, the ratio of the FOV of image capture device 122 to the FOV of image capture device 124 and / or image capture device 126 may vary from 1.5 to 2.0. In other embodiments, the ratio may vary between 1.25 and 2.25.

[0149] System 100 can be configured such that the field of view of image capture device 122 at least partially or completely overlaps the field of view of image capture device 124 and / or image capture device 126. In some embodiments, system 100 can be configured such that the fields of view of image capture devices 124 and 126, for example, fall within (e.g., are narrower than) the field of view of image capture device 122 and share a common center therewith. In other embodiments, image capture devices 122, 124, and 126 can capture adjacent FOVs or can have partial overlap in their FOVs. In some embodiments, the fields of view of image capture devices 122, 124, and 126 can be aligned such that the center of the narrower FOV image capture devices 124 and / or 126 is located in the lower half of the field of view of the wider FOV device 122.

[0150] Figure 2F FIG. 1 is a schematic representation of an exemplary vehicle control system consistent with the disclosed embodiments. Figure 2F As indicated, the vehicle 200 may include a throttle system 220, a braking system 230, and a steering system 240. The system 100 may provide input (e.g., control signals) to one or more of the throttle system 220, the braking system 230, and the steering system 240 via one or more data links (e.g., any wired and / or wireless links or links for transmitting data). For example, based on analysis of images acquired by the image capture devices 122, 124, and / or 126, the system 100 may provide control signals to one or more of the throttle system 220, the braking system 230, and the steering system 240 to navigate the vehicle 200 (e.g., by causing acceleration, turning, lane changes, etc.). In addition, the system 100 may receive input from one or more of the throttle system 220, the braking system 230, and the steering system 240 indicating an operating condition of the vehicle 200 (e.g., speed, whether the vehicle 200 is braking and / or turning, etc.). hereinafter in conjunction with Figures 4 to 7 Provide further details.

[0151] like Figure 3AAs shown, vehicle 200 may also include a user interface 170 for interacting with the driver or passengers of vehicle 200. For example, user interface 170 in a vehicle application may include a touch screen 320, a knob 330, buttons 340, and a microphone 350. The driver or passengers of vehicle 200 may also use handles (e.g., located on or near the steering column of vehicle 200, including, for example, a turn signal handle), buttons (e.g., located on the steering wheel of vehicle 200), etc. to interact with system 100. In some embodiments, microphone 350 may be positioned adjacent to rearview mirror 310. Similarly, in some embodiments, image capture device 122 may be located near rearview mirror 310. In some embodiments, user interface 170 may also include one or more speakers 360 (e.g., speakers of a vehicle audio system). For example, system 100 may provide various notifications (e.g., alarms) via speakers 360.

[0152] Figures 3B to 3D 3 is an illustration of an exemplary camera mount 370 configured to be positioned behind a rearview mirror (e.g., rearview mirror 310) and against a vehicle windshield, consistent with the disclosed embodiments. Figure 3B As shown, camera mount 370 may include image capture devices 122, 124, and 126. Image capture devices 124 and 126 may be positioned behind glare shield 380, which may be flush against the vehicle windshield and include a combination of film and / or anti-reflective material. For example, glare shield 380 may be positioned so that it is aligned against a vehicle windshield having a matching slope. In some embodiments, each of image capture devices 122, 124, and 126 may be positioned behind glare shield 380, such as, for example, Figure 3D The disclosed embodiments are not limited to any particular configuration of image capture devices 122 , 124 , and 126 , camera mount 370 , and glare shield 380 . Figure 3C For the front view Figure 3B An illustration of a camera mount 370 is shown.

[0153] As will be appreciated by those skilled in the art having the benefit of this disclosure, numerous variations and / or modifications may be made to the aforementioned disclosed embodiments. For example, not all components are required for the operation of system 100. Furthermore, any component may be located in any suitable part of system 100, and these components may be rearranged into various configurations while providing the functionality of the disclosed embodiments. Therefore, the aforementioned configurations are examples, and regardless of the configurations discussed above, system 100 may provide a wide range of functionality for analyzing the surrounding environment of vehicle 200 and navigating vehicle 200 in response to that analysis.

[0154] As discussed in further detail below and consistent with the various disclosed embodiments, system 100 can provide various features related to autonomous driving and / or driver assistance technologies. For example, system 100 can analyze image data, location data (e.g., GPS location information), map data, speed data, and / or data from sensors included in vehicle 200. System 100 can collect data for analysis from, for example, image acquisition unit 120, location sensor 130, and other sensors. Furthermore, system 100 can analyze the collected data to determine whether vehicle 200 should take a specific action, and then automatically take the determined action without human intervention. For example, while vehicle 200 is navigating without human intervention, system 100 can automatically control braking, acceleration, and / or steering of vehicle 200 (e.g., by sending control signals to one or more of throttle system 220, braking system 230, and steering system 240). Furthermore, system 100 can analyze the collected data and, based on the analysis of the collected data, issue warnings and / or alerts to vehicle occupants. Additional details regarding various embodiments provided by system 100 are provided below.

[0155] Forward multiple imaging system

[0156] As discussed above, system 100 can provide driver assistance functionality using a multi-camera system. A multi-camera system can use one or more cameras facing forward of the vehicle. In other embodiments, the multi-camera system can include one or more cameras facing the side or rear of the vehicle. In one embodiment, for example, system 100 can use a dual-camera imaging system, where the first and second cameras (e.g., image capture devices 122 and 124) can be positioned at the front and / or sides of the vehicle (e.g., vehicle 200). Other camera configurations are consistent with the disclosed embodiments, and the configurations disclosed herein are examples. For example, system 100 can include a configuration with any number of cameras (e.g., one, two, three, four, five, six, seven, eight, etc.). Furthermore, system 100 can include camera "clusters." For example, a camera cluster (including any appropriate number of cameras, e.g., one, four, eight, etc.) can be forward-facing relative to the vehicle, or can face any other direction (e.g., rearward, sideways, angled, etc.). Thus, system 100 can include multiple camera clusters, each oriented in a specific direction to capture images from a specific area of ​​the vehicle's environment.

[0157] The first camera may have a field of view that is larger than, smaller than, or partially overlaps with the field of view of the second camera. Additionally, the first camera may be connected to a first image processor to perform monocular image analysis on images provided by the first camera, and the second camera may be connected to a second image processor to perform monocular image analysis on images provided by the second camera. The outputs (e.g., processed information) of the first and second image processors may be combined. In some embodiments, the second image processor may receive images from both the first and second cameras to perform stereoscopic analysis. In another embodiment, system 100 may utilize a three-camera imaging system, each of which has a different field of view. Thus, such a system can make decisions based on information derived from objects located at varying distances both in front of and to the sides of the vehicle. References to monocular image analysis may refer to situations in which image analysis is performed based on images captured from a single viewpoint (e.g., from a single camera). Stereoscopic image analysis may refer to situations in which image analysis is performed based on two or more images captured using one or more variations of image capture parameters. For example, captured images suitable for stereoscopic image analysis may include images captured from two or more different positions, images captured from different fields of view, images captured using different focal lengths and disparity information, and the like.

[0158] For example, in one embodiment, system 100 may implement a three-camera configuration using image capture devices 122 to 126. In such a configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees, or other values ​​selected from a range of approximately 20 to 45 degrees, etc.), image capture device 124 may provide a wide field of view (e.g., 150 degrees, or other values ​​selected from a range of approximately 100 to approximately 180 degrees), and image capture device 126 may provide an intermediate field of view (e.g., 46 degrees, or other values ​​selected from a range of approximately 35 to approximately 60 degrees). In some embodiments, image capture device 126 may serve as the primary or main camera. Image capture devices 122 to 126 may be positioned behind rearview mirror 310 and positioned substantially side-by-side (e.g., 6 cm apart). Furthermore, in some embodiments, as discussed above, one or more of image capture devices 122 to 126 may be mounted behind a glare shield 380 flush with the windshield of vehicle 200. Such a mask may serve to minimize the effect of any reflections from the interior of the vehicle on the image capture devices 122 - 126 .

[0159] In another embodiment, as combined Figure 3B and 3CAs discussed, the wide field of view camera (e.g., image capture device 124 in the example above) can be mounted lower than the narrow field of view and main field of view cameras (e.g., image devices 122 and 126 in the example above). This configuration provides a clear line of sight from the wide field of view camera. To reduce reflections, the camera can be mounted close to the windshield of vehicle 200 and a polarizer on the camera can be included to reduce reflected light.

[0160] A three-camera system can provide specific performance characteristics. For example, some embodiments may include the ability to verify the detection of an object by one camera based on the detection results from another camera. In the three-camera configuration discussed above, the processing unit 110 may include, for example, three processing devices (e.g., three EyeQ series processor chips, as discussed above), where each processing device is dedicated to processing images captured by one or more of the image capture devices 122 to 126.

[0161] In a three-camera system, the first processing device can receive images from both the main camera and the narrow field of view camera and perform vision processing for the narrow FOV camera to, for example, detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Furthermore, the first processing device can calculate pixel disparity between the images from the main camera and the narrow camera and create a 3D reconstruction of the vehicle 200's environment. The first processing device can then combine the 3D reconstruction with 3D map data or with 3D information calculated based on information from another camera.

[0162] The second processing device can receive images from the primary camera and perform visual processing to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the second processing device can calculate camera displacement and, based on this displacement, calculate pixel disparity between successive images and create a 3D reconstruction of the scene (e.g., structure from motion). The second processing device can send the structure from motion 3D reconstruction to the first processing device for combination with the stereoscopic 3D image.

[0163] The third processing device may receive images from the wide FOV camera and process the images to detect vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. The third processing device may further execute additional processing instructions to analyze the images to identify moving objects in the images, such as vehicles changing lanes, pedestrians, etc.

[0164] In some embodiments, having image-based information streams captured and processed independently may provide an opportunity to provide redundancy in the system. Such redundancy may include, for example, using a first image capture device and images processed from that device to verify and / or supplement information obtained by capturing and processing image information from at least a second image capture device.

[0165] In some embodiments, system 100 may use two image capture devices (e.g., image capture devices 122 and 124) to provide navigation assistance for vehicle 200, and a third image capture device (e.g., image capture device 126) to provide redundancy and validate the analysis of data received from the other two image capture devices. For example, in such a configuration, image capture devices 122 and 124 may provide images for stereo analysis by system 100 for navigating vehicle 200, while image capture device 126 may provide images for monocular analysis by system 100 to provide redundancy and validation of information derived from images captured by image capture devices 122 and / or 124. That is, image capture device 126 (and a corresponding processing device) may be considered to provide a redundant subsystem to provide a check on the analysis derived from image capture devices 122 and 124 (e.g., to provide an automatic emergency braking (AEB) system). Additionally, in some embodiments, redundancy and verification of the received data may be supplemented based on information received from one or more sensors (e.g., radar, lidar, acoustic sensors, information received from one or more transceivers outside the vehicle, etc.).

[0166] Those skilled in the art will recognize that the camera configurations, camera placements, number of cameras, camera locations, etc. described above are merely examples. These components and other components described with respect to the overall system can be assembled and used in a variety of different configurations without departing from the scope of the disclosed embodiments. Further details regarding the use of a multi-camera system to provide driver assistance and / or autonomous vehicle functionality are as follows.

[0167] Figure 4 An exemplary functional block diagram of memory 140 and / or 150 that may be stored / programmed with instructions for performing one or more operations consistent with the disclosed embodiments. Although the following refers to memory 140, those skilled in the art will recognize that instructions may be stored in memory 140 and / or 150.

[0168] like Figure 4 As shown, memory 140 may store a monocular image analysis module 402, a stereo image analysis module 404, a velocity and acceleration module 406, and a navigation response module 408. The disclosed embodiments are not limited to any particular configuration of memory 140. Furthermore, application processor 180 and / or image processor 190 may execute instructions stored in any of modules 402 to 408 included in memory 140. Those skilled in the art will appreciate that references to processing unit 110 in the following discussion may refer individually or collectively to application processor 180 and image processor 190. Therefore, the steps of any of the following processes may be performed by one or more processing devices.

[0169] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis on a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensory information (e.g., information from radar) to perform the monocular image analysis. Figures 5A to 5D As described, the monocular image analysis module 402 may include instructions for detecting a set of features within the set of images (such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other features associated with the vehicle's environment). Based on this analysis, the system 100 (e.g., via the processing unit 110) may cause one or more navigation responses in the vehicle 200, such as turns, lane changes, changes in acceleration, etc., as discussed below in conjunction with the navigation response module 408.

[0170] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis on a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensory information (e.g., information from radar, lidar, etc.) to perform monocular image analysis. Figures 5A to 5D As described, the monocular image analysis module 402 may include instructions for detecting a set of features within the set of images (such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other features associated with the vehicle's environment). Based on this analysis, the system 100 (e.g., via the processing unit 110) may cause one or more navigation responses in the vehicle 200, such as turns, lane changes, changes in acceleration, etc., as discussed below in conjunction with determining navigation responses.

[0171] In one embodiment, stereo image analysis module 404 may store instructions (such as computer vision software) that, when executed by processing unit 110, perform stereo image analysis of a first set of images and a second set of images acquired by a combination of image capture devices selected from any one of image capture devices 122, 124, and 126. In some embodiments, processing unit 110 may combine information from the first set of images and the second set of images with additional sensory information (e.g., information from radar) to perform stereo image analysis. For example, stereo image analysis module 404 may include instructions for performing stereo image analysis based on a first set of images acquired by image capture device 124 and a second set of images acquired by image capture device 126. As described below in conjunction with Figure 6 As described, stereo image analysis module 404 may include instructions for detecting a set of features within the first set of images and the second set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, etc. Based on this analysis, processing unit 110 may cause one or more navigation responses in vehicle 200, such as turns, lane changes, changes in acceleration, etc., as discussed below in conjunction with navigation response module 408. Furthermore, in some embodiments, stereo image analysis module 404 may implement techniques associated with either a trained system (such as a neural network or a deep neural network) or an untrained system.

[0172] In one embodiment, the velocity and acceleration module 406 may store software configured to analyze data received from one or more computing and electromechanical devices in the vehicle 200 that are configured to cause changes in the velocity and / or acceleration of the vehicle 200. For example, the processing unit 110 may execute instructions associated with the velocity and acceleration module 406 to calculate a target velocity for the vehicle 200 based on data derived from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include, for example, the target position, velocity, and / or acceleration, the position and / or velocity of the vehicle 200 relative to nearby vehicles, pedestrians, or road objects, position information for the vehicle 200 relative to lane markings on the road, etc. Among other things, the processing unit 110 may calculate the target velocity for the vehicle 200 based on sensory input (e.g., information from radar) and input from other systems of the vehicle 200, such as the throttle system 220, the braking system 230, and / or the steering system 240 of the vehicle 200. Based on the calculated target speed, the processing unit 110 may transmit electronic signals to the throttle system 220, the braking system 230 and / or the steering system 240 of the vehicle 200 to trigger a change in speed and / or acceleration by, for example, physically depressing the brakes or releasing the accelerator of the vehicle 200.

[0173] In one embodiment, navigation response module 408 may store software executable by processing unit 110 to determine a desired navigation response based on data derived from the execution of monocular image analysis module 402 and / or stereo image analysis module 404. Such data may include position and velocity information associated with nearby vehicles, pedestrians, and road objects, target position information for vehicle 200, and the like. Additionally, in some embodiments, the navigation response may be based (in part or in whole) on map data, a predetermined position of vehicle 200, and / or a relative velocity or acceleration between vehicle 200 and one or more objects detected from the execution of monocular image analysis module 402 and / or stereo image analysis module 404. Navigation response module 408 may also determine the desired navigation response based on sensory input (e.g., information from radar) and input from other systems of vehicle 200, such as throttle system 220, braking system 230, and steering system 240 of vehicle 200. Based on the desired navigation response, processing unit 110 may transmit electronic signals to throttle system 220, brake system 230, and steering system 240 of vehicle 200 to trigger the desired navigation response to achieve a predetermined angle of rotation, for example, by turning the steering wheel of vehicle 200. In some embodiments, processing unit 110 may use the output of navigation response module 408 (e.g., the desired navigation response) as input to the execution of velocity and acceleration module 406 for calculating the change in velocity of vehicle 200.

[0174] Furthermore, any modules disclosed herein (e.g., modules 402, 404, and 406) may implement techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems.

[0175] Figure 5A A flow chart illustrating an exemplary process 500A for causing one or more navigation responses based on monocular image analysis consistent with the disclosed embodiments is provided. At step 510, the processing unit 110 may receive a plurality of images via the data interface 128 between the processing unit 110 and the image acquisition unit 120. For example, a camera included in the image acquisition unit 120 (such as the image capture device 122 having the field of view 202) may capture a plurality of images of an area in front of the vehicle 200 (or, for example, to the side or rear of the vehicle) and transmit them to the processing unit 110 via a data connection (e.g., digital, wired, USB, wireless, Bluetooth, etc.). The processing unit 110 may execute the monocular image analysis module 402 at step 520 to analyze the plurality of images, as described below in conjunction with Figures 5B to 5D By performing this analysis, processing unit 110 may detect a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, etc.

[0176] Processing unit 110 may also execute monocular image analysis module 402 at step 520 to detect various road hazards, such as, for example, parts of truck tires, fallen road signs, loose cargo, small animals, etc. Road hazards can vary in structure, shape, size, and color, which may make detecting such hazards more challenging. In some embodiments, processing unit 110 may execute monocular image analysis module 402 to perform multi-frame analysis on multiple images to detect road hazards. For example, processing unit 110 may estimate camera motion between consecutive image frames and calculate pixel disparity between frames to construct a 3D map of the road. Processing unit 110 may then use the 3D map to detect hazards on the road surface and above the road surface.

[0177] At step 530, the processing unit 110 may execute the navigation response module 408 to perform the navigation response based on the analysis performed at step 520 and the above-mentioned combination. Figure 4 The described techniques are used to cause one or more navigation responses in vehicle 200. Navigation responses may include, for example, turns, lane changes, changes in acceleration, etc. In some embodiments, processing unit 110 may use data derived from execution of rate and acceleration module 406 to cause one or more navigation responses. In addition, multiple navigation responses may occur simultaneously, sequentially, or any combination thereof. For example, processing unit 110 may cause vehicle 200 to change out of a lane and then accelerate by, for example, sequentially transmitting control signals to steering system 240 and throttle system 220 of vehicle 200. Alternatively, processing unit 110 may cause vehicle 200 to brake while changing lanes by, for example, simultaneously transmitting control signals to braking system 230 and steering system 240 of vehicle 200.

[0178] Figure 5B Flowchart illustrating an exemplary process 500B for detecting one or more vehicles and / or pedestrians in a set of images consistent with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500B. At step 540, processing unit 110 may determine a set of candidate objects representing possible vehicles and / or pedestrians. For example, processing unit 110 may scan one or more images, compare the images to one or more predetermined patterns, and identify possible locations within each image that may contain objects of interest (e.g., vehicles, pedestrians, or portions thereof). The predetermined patterns may be designed to achieve a high "false hit" rate and a low "miss" rate. For example, processing unit 110 may use a low threshold for similarity to the predetermined patterns to identify a candidate object as a possible vehicle or pedestrian. Doing so may allow processing unit 110 to reduce the probability of missing (e.g., not identifying) a candidate object representing a vehicle or pedestrian.

[0179] At step 542, processing unit 110 may filter the set of candidate objects based on classification criteria to exclude certain candidates (e.g., irrelevant or less relevant objects). Such criteria may be derived from various attributes associated with object types stored in a database (e.g., a database stored in memory 140). Attributes may include object shape, size, texture, location (e.g., relative to vehicle 200), etc. Thus, processing unit 110 may use one or more sets of criteria to reject incorrect candidates from a set of candidate objects.

[0180] At step 544, processing unit 110 may analyze the plurality of image frames to determine whether an object in a set of candidate objects represents a vehicle and / or a pedestrian. For example, processing unit 110 may track detected candidate objects across consecutive frames and accumulate frame-by-frame data associated with the detected objects (e.g., size, position relative to vehicle 200, etc.). Additionally, processing unit 110 may estimate parameters for the detected objects and compare the frame-by-frame position data of the objects with the predicted positions.

[0181] At step 546, processing unit 110 may construct a set of measurements for the detected objects. Such measurements may include, for example, position, velocity, and acceleration values ​​associated with the detected objects (relative to vehicle 200). In some embodiments, processing unit 110 may construct the measurements based on an estimation technique such as a Kalman filter or linear quadratic estimation (LQE) using a series of time-based observations and / or based on available modeling data for different object types (e.g., cars, trucks, pedestrians, bicycles, road signs, etc.). The Kalman filter may be based on a measurement of the scale of the object, where the scale measurement is proportional to the time to collision (e.g., the amount of time it takes for vehicle 200 to reach the object). Thus, by performing steps 540 to 546, processing unit 110 may identify vehicles and pedestrians that appear within a set of captured images and derive information related to the vehicles and pedestrians (e.g., position, velocity, size). Based on the identified and derived information, processing unit 110 may cause one or more navigation responses in vehicle 200, as described above in conjunction with Figure 5A described.

[0182] At step 548, processing unit 110 may perform an optical flow analysis on one or more images to reduce the probability of detecting "false hits" and missing candidate objects representing vehicles or pedestrians. Optical flow analysis may refer to, for example, analyzing motion patterns relative to vehicle 200 in one or more images associated with other vehicles and pedestrians, and where the motion patterns are different from the motion of the road surface. Processing unit 110 may calculate the motion of candidate objects by observing the different positions of the objects across multiple image frames captured at different times. Processing unit 110 may use the position and time values ​​as inputs into a mathematical model to calculate the motion of the candidate objects. Therefore, optical flow analysis may provide another method for detecting vehicles and pedestrians near vehicle 200. Processing unit 110 may perform optical flow analysis in conjunction with steps 540 to 546 to provide redundancy for detecting vehicles and pedestrians and increase the reliability of system 100.

[0183] Figure 5C Flowchart illustrating an exemplary process 500C for detecting road markings and / or lane geometry in a set of images consistent with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500C. At step 550, processing unit 110 may detect a set of objects by scanning one or more images. In order to detect lane markings, lane geometry, and other relevant road marking segments, processing unit 110 may filter the set of objects to exclude those determined to be irrelevant (e.g., small potholes, small rocks, etc.). At step 552, processing unit 110 may group together the segments detected in step 550 that belong to the same road marking or lane marking. Based on the grouping, processing unit 110 may develop a model, such as a mathematical model, representing the detected segments.

[0184] At step 554, the processing unit 110 may construct a set of measurements associated with the detected segment. In some embodiments, the processing unit 110 may create a projection of the detected segment from the image plane to the real-world plane. The projection may be characterized using a cubic polynomial with coefficients corresponding to physical properties (such as the position, slope, curvature, and curvature derivatives of the detected road). When generating the projection, the processing unit 110 may take into account changes in the road surface and the pitch rate and roll rate associated with the vehicle 200. In addition, the processing unit 110 may model the road elevation by analyzing position and motion cues present on the road surface. Furthermore, the processing unit 110 may estimate the pitch rate and roll rate associated with the vehicle 200 by tracking a set of feature points in one or more images.

[0185] At step 556, processing unit 110 may perform a multi-frame analysis by, for example, tracking the detected segments across consecutive image frames and accumulating frame-by-frame data associated with the detected segments. As processing unit 110 performs a multi-frame analysis, the set of measurements constructed at step 554 may become more reliable and associated with increasingly higher confidence levels. Thus, by performing steps 550 through 556, processing unit 110 may identify road markings that appear within a set of captured images and derive lane geometry information. Based on the identified and derived information, processing unit 110 may cause one or more navigation responses in vehicle 200, as described above in conjunction with Figure 5A described.

[0186] At step 558, processing unit 110 may consider additional information sources to further develop a safety model for vehicle 200 in the context of its surroundings. Processing unit 110 may use the safety model to define scenarios in which system 100 can safely perform autonomous control of vehicle 200. To develop the safety model, in some embodiments, processing unit 110 may consider the positions and motions of other vehicles, detected road edges and obstacles, and / or a general road shape description extracted from map data (such as data from map database 160). By considering additional information sources, processing unit 110 can provide redundancy for detecting road markings and lane geometry and increase the reliability of system 100.

[0187] Figure 5D A flow chart illustrating an exemplary process 500D for detecting traffic lights in a set of images consistent with the disclosed embodiments is provided. Processing unit 110 may execute monocular image analysis module 402 to implement process 500D. At step 560, processing unit 110 may scan the set of images and identify objects that appear at locations in the images that may contain traffic lights. For example, processing unit 110 may filter the identified objects to construct a set of candidate objects, excluding those that are unlikely to correspond to traffic lights. This filtering may be performed based on various attributes associated with traffic lights, such as shape, size, texture, and position (e.g., relative to vehicle 200). Such attributes may be based on multiple examples of traffic lights and traffic control signals and stored in a database. In some embodiments, processing unit 110 may perform a multi-frame analysis on the set of candidate objects that reflect possible traffic lights. For example, processing unit 110 may track candidate objects across consecutive image frames, estimate the real-world positions of the candidate objects, and filter out objects that are moving (thus unlikely to be traffic lights). In some embodiments, processing unit 110 may perform color analysis on the candidate objects and identify the relative positions of the detected colors that appear within the possible traffic lights.

[0188] At step 562, processing unit 110 may analyze the geometric features of the intersection. This analysis may be based on any combination of: (i) the number of lanes detected on either side of vehicle 200, (ii) detected markings on the road (such as arrows), and (iii) a description of the intersection extracted from map data (such as data from map database 160). Processing unit 110 may use information derived from the execution of monocular analysis module 402 to perform the analysis. Additionally, processing unit 110 may determine a correspondence between the traffic lights detected at step 560 and the lanes present near vehicle 200.

[0189] As the vehicle 200 approaches the intersection, at step 564, the processing unit 110 may update a confidence level associated with the analyzed intersection geometry and detected traffic lights. For example, the number of traffic lights estimated to be present at the intersection compared to the number actually present at the intersection may affect the confidence level. Therefore, based on the confidence level, the processing unit 110 may delegate control to the driver of the vehicle 200 in order to improve safety conditions. By performing steps 560 to 564, the processing unit 110 may identify the traffic lights that appear within the set of captured images and analyze the intersection geometry. Based on the identification and analysis, the processing unit 110 may cause one or more navigation responses in the vehicle 200, as described above in conjunction with Figure 5A described.

[0190] Figure 5E Flowchart showing an exemplary process 500E for inducing one or more navigation responses in vehicle 200 based on a vehicle path consistent with the disclosed embodiments. At step 570, processing unit 110 may construct an initial vehicle path associated with vehicle 200. The vehicle path may be represented by a set of points expressed as coordinates (x, z), and the distance d between two points in the set of points may be i May fall within the range of 1 meter to 5 meters. In one embodiment, the processing unit 110 may use two polynomials (such as a left road polynomial and a right road polynomial) to construct an initial vehicle path. The processing unit 110 may calculate the geometric midpoint between the two polynomials and, if available, offset each point included in the resulting vehicle path by a predetermined offset (e.g., a smart lane offset) (a zero offset may correspond to driving in the middle of the lane). The offset may be in a direction perpendicular to the segment between any two points in the vehicle path. In another embodiment, the processing unit 110 may use a polynomial and an estimated lane width to offset each point of the vehicle path by half the estimated lane width plus a predetermined offset (e.g., a smart lane offset).

[0191] At step 572, the processing unit 110 may update the vehicle path constructed at step 570. The processing unit 110 may reconstruct the vehicle path constructed at step 570 using a higher resolution so that the distance d between two points in the set of points representing the vehicle path is k Smaller than the distance d mentioned above i For example, the distance d k The processing unit 110 may reconstruct the vehicle path using a parabolic spline algorithm, which may generate a cumulative distance vector S corresponding to the total length of the vehicle path (ie, based on a set of points representing the vehicle path).

[0192] At step 574, the processing unit 110 may determine a look-ahead point (expressed as coordinates (x l ,z l )). Processing unit 110 may extract a look-ahead point from the accumulated distance vector S, and the look-ahead point may be associated with a look-ahead distance and a look-ahead time. The look-ahead distance (which may have a lower limit in the range of 10 meters to 20 meters) may be calculated as the product of the speed of vehicle 200 and the look-ahead time. For example, as the speed of vehicle 200 decreases, the look-ahead distance may also decrease (e.g., until it reaches the lower limit). The look-ahead time (which may be in the range of 0.5 seconds to 1.5 seconds) may be inversely proportional to the gain of one or more control loops associated with inducing a navigation response in vehicle 200, such as a heading error tracking control loop. For example, the gain of the heading error tracking control loop may depend on the bandwidth of the yaw rate loop, the steering actuator loop, the lateral dynamics of the car, etc. Therefore, the higher the gain of the heading error tracking control loop, the shorter the look-ahead time.

[0193] At step 576, the processing unit 110 may determine the heading error and yaw rate command based on the look-ahead point determined at step 574. The processing unit 110 may calculate the heading error and yaw rate command by calculating the arc tangent of the look-ahead point, e.g., arctan(x l / z l ) to determine the heading error. Processing unit 110 may determine the yaw rate command as the product of the heading error and the high-level control gain. If the look-ahead distance is not at the lower limit, the high-level control gain may be equal to: (2 / look-ahead time). Otherwise, the high-level control gain may be equal to: (2*vehicle 200 speed / look-ahead distance).

[0194] Figure 5F FIGURE 5 is a flow chart illustrating an exemplary process 500F for determining whether a leading vehicle is changing lanes consistent with the disclosed embodiments. At step 580, the processing unit 110 may determine navigation information associated with a leading vehicle (e.g., a vehicle traveling in front of the vehicle 200). For example, the processing unit 110 may use the above-described method in conjunction with the vehicle 200 to determine whether the leading vehicle is changing lanes. Figure 5A and 5B The processing unit 110 may also use the above combined Figure 5E The described techniques determine one or more road polynomials, look-ahead points (associated with vehicle 200 ), and / or snail trajectories (eg, a set of points describing a path taken by a leading vehicle).

[0195] At step 582, processing unit 110 may analyze the navigation information determined at step 580. In one embodiment, processing unit 110 may calculate the distance between the snail trajectory and the road polynomial (e.g., along the trajectory). If the variance of this distance along the trajectory exceeds a predetermined threshold (e.g., 0.1 to 0.2 meters on a straight road, 0.3 to 0.4 meters on a moderately curving road, and 0.5 to 0.6 meters on a road with a sharp curve), processing unit 110 may determine that the preceding vehicle may be changing lanes. In the event that multiple vehicles are detected traveling ahead of vehicle 200, processing unit 110 may compare the snail trajectories associated with each vehicle. Based on this comparison, processing unit 110 may determine that a vehicle whose snail trajectory does not match the snail trajectories of the other vehicles may be changing lanes. Processing unit 110 may also compare the curvature of the snail trajectory (associated with the preceding vehicle) with the expected curvature of the road segment on which the preceding vehicle is traveling. The expected curvature may be extracted from map data (e.g., data from map database 160 ), from a road polynomial, from snail trajectories of other vehicles, from prior knowledge about the road, etc. If the curvature of the snail trajectory differs from the expected curvature of the road segment by more than a predetermined threshold, processing unit 110 may determine that the leading vehicle may be changing lanes.

[0196] In another embodiment, the processing unit 110 may compare the instantaneous position of the preceding vehicle with a look-ahead point (associated with the vehicle 200) over a specific time period (e.g., 0.5 seconds to 1.5 seconds). If the distance between the instantaneous position of the preceding vehicle and the look-ahead point changes over the specific time period, and the cumulative sum of the changes exceeds a predetermined threshold (e.g., 0.3 meters to 0.4 meters on a straight road, 0.7 meters to 0.8 meters on a moderately curved road, and 1.3 meters to 1.7 meters on a road with a sharp curve), the processing unit 110 may determine that the preceding vehicle is likely changing lanes. In another embodiment, the processing unit 110 may analyze the geometric characteristics of the snail's trajectory by comparing the lateral distance traveled along the snail's trajectory with the expected curvature of the snail's trajectory. The expected curvature radius may be determined according to the following calculation: (δ z 2 +δ x2 ) / 2 / (δ x ), where δ x represents the lateral distance traveled, and δ z Indicates the longitudinal distance traveled. If the difference between the lateral distance traveled and the expected curvature exceeds a predetermined threshold (e.g., 500 meters to 700 meters), the processing unit 110 may determine that the leading vehicle is likely changing lanes. In another embodiment, the processing unit 110 may analyze the position of the leading vehicle. If the position of the leading vehicle obscures the road polynomial (e.g., the leading vehicle is overlaid on top of the road polynomial), the processing unit 110 may determine that the leading vehicle is likely changing lanes. If the position of the leading vehicle is such that another vehicle is detected in front of the leading vehicle and the snail trajectories of the two vehicles are not parallel, the processing unit 110 may determine that the (closer) leading vehicle is likely changing lanes.

[0197] At step 584, processing unit 110 may determine whether the leading vehicle 200 is changing lanes based on the analysis performed at step 582. For example, processing unit 110 may make a determination based on a weighted average of the individual analyses performed at step 582. Under such an approach, for example, a determination made by processing unit 110 that the leading vehicle is likely changing lanes based on a particular type of analysis may be assigned a value of "1" (and "0" to indicate a determination that the leading vehicle is unlikely to be changing lanes). Different analyses performed at step 582 may be assigned different weights, and the disclosed embodiments are not limited to any particular combination of analyses and weights. Furthermore, in some embodiments, the analysis may utilize a trained system (e.g., a machine learning or deep learning system) that can, for example, estimate a future path ahead of the vehicle's current location based on images captured at the current location.

[0198] Figure 6 A flow chart illustrating an exemplary process 600 for inducing one or more navigation responses based on stereo image analysis consistent with the disclosed embodiments is shown. At step 610, processing unit 110 may receive a first plurality of images and a second plurality of images via data interface 128. For example, cameras included in image acquisition unit 120 (such as image capture devices 122 and 124 having fields of view 202 and 204) may capture a first plurality of images and a second plurality of images of the area in front of vehicle 200 and transmit them to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first plurality of images and the second plurality of images via two or more data interfaces. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0199] At step 620, the processing unit 110 may execute the stereo image analysis module 404 to perform stereo image analysis on the first plurality of images and the second plurality of images to create a 3D map of the road in front of the vehicle and detect features within the image, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, etc. The stereo image analysis may be combined with the above Figures 5A to 5D The steps described above are performed in a similar manner. For example, processing unit 110 can execute stereo image analysis module 404 to detect candidate objects (e.g., vehicles, pedestrians, road markings, traffic lights, road hazards, etc.) within the first and second pluralities of images, filter out a subset of candidate objects based on various criteria, and perform multi-frame analysis, construct measurements, and determine confidence levels for the remaining candidate objects. In performing the above steps, processing unit 110 can consider information from both the first and second pluralities of images, rather than information from a single set of images. For example, processing unit 110 can analyze differences in pixel-level data (or other subsets of data from the two streams of captured images) for candidate objects appearing in both the first and second pluralities of images. As another example, processing unit 110 can estimate the position and / or velocity of the object (e.g., relative to vehicle 200) by observing that the candidate object appears in one of the multiple images but not the other, or other differences that may exist relative to objects appearing in both image streams. For example, position, velocity, and / or acceleration relative to vehicle 200 may be determined based on the trajectory, position, movement characteristics, etc. of features associated with objects appearing in one or both of the image streams.

[0200] At step 630, the processing unit 110 may execute the navigation response module 408 to perform the navigation response based on the analysis performed at step 620 and the above-mentioned combination. Figure 4 The described techniques may be used to cause one or more navigation responses in the vehicle 200. The navigation responses may include, for example, turns, lane changes, changes in acceleration, changes in speed, braking, etc. In some embodiments, the processing unit 110 may use data derived from execution of the speed and acceleration module 406 to cause one or more navigation responses. Additionally, multiple navigation responses may occur simultaneously, sequentially, or any combination thereof.

[0201] Figure 7A flow chart illustrating an exemplary process 700 for inducing one or more navigation responses based on analysis of three sets of images, consistent with the disclosed embodiments, is shown. At step 710, processing unit 110 may receive a first plurality of images, a second plurality of images, and a third plurality of images via data interface 128. For example, cameras included in image acquisition unit 120 (such as image capture devices 122, 124, and 126 having fields of view 202, 204, and 206) may capture a first plurality of images, a second plurality of images, and a third plurality of images of areas in front of and / or to the sides of vehicle 200 and transmit them to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first plurality of images, the second plurality of images, and the third plurality of images via three or more data interfaces. For example, each of image capture devices 122, 124, and 126 may have an associated data interface for transmitting data to processing unit 110. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0202] At step 720, the processing unit 110 may analyze the first plurality of images, the second plurality of images, and the third plurality of images to detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, etc. This analysis may be performed in conjunction with the above. Figures 5A to 5D and Figure 6 For example, the processing unit 110 may perform a monocular image analysis on each of the first plurality of images, the second plurality of images, and the third plurality of images (e.g., via the execution of the monocular image analysis module 402 and based on the above combined Figures 5A to 5D Alternatively, the processing unit 110 may perform stereoscopic image analysis on the first plurality of images and the second plurality of images, the second plurality of images and the third plurality of images, and / or the first plurality of images and the third plurality of images (e.g., via execution of the stereoscopic image analysis module 404 and based on the above in combination with Figure 64 and 5. The processing unit 110 may perform a combination of monocular image analysis and stereo image analysis. For example, the processing unit 110 may perform monocular image analysis on the first plurality of images (e.g., via execution by the monocular image analysis module 402) and stereo image analysis on the second and third pluralities of images (e.g., via execution by the stereo image analysis module 404). The configuration of the image capture devices 122, 124, and 126—including their respective locations and fields of view 202, 204, and 206—may affect the type of analysis performed on the first, second, and third pluralities of images. The disclosed embodiments are not limited to a particular configuration of the image capture devices 122, 124, and 126, or the type of analysis performed on the first, second, and third pluralities of images.

[0203] In some embodiments, processing unit 110 may test system 100 based on the images acquired and analyzed at steps 710 and 720. Such testing may provide an indicator of the overall performance of system 100 for a particular configuration of image capture devices 122, 124, and 126. For example, processing unit 110 may determine the proportion of "false hits" (e.g., instances in which system 100 incorrectly determines the presence of a vehicle or pedestrian) and "misses."

[0204] At step 730, the processing unit 110 may cause one or more navigation responses in the vehicle 200 based on information derived from both the first plurality of images, the second plurality of images, and the third plurality of images. The selection of both the first plurality of images, the second plurality of images, and the third plurality of images may depend on various factors, such as, for example, the number, type, and size of objects detected in each of the plurality of images. The processing unit 110 may also base its selection on image quality and resolution, the effective field of view reflected in the image, the number of frames captured, the extent to which one or more objects of interest actually appear in the frames (e.g., the percentage of frames in which the objects appear, the proportion of the objects appearing in each such frame, etc.), etc.

[0205] In some embodiments, processing unit 110 may select information derived from both the first, second, and third pluralities of images by determining the extent to which information derived from one image source is consistent with information derived from other image sources. For example, processing unit 110 may combine processed information derived from each of image capture devices 122, 124, and 126 (whether through monocular analysis, stereo analysis, or any combination thereof) and determine visual indicators (e.g., lane markings, detected vehicles and their locations and / or paths, detected traffic lights, etc.) that are consistent across the images captured by each of image capture devices 122, 124, and 126. Processing unit 110 may also exclude information that is inconsistent across the captured images (e.g., vehicles changing lanes, lane models indicating vehicles that are too close to vehicle 200, etc.). Thus, processing unit 110 may select information derived from both the first, second, and third pluralities of images based on the determination of consistent and inconsistent information.

[0206] The navigation response may include, for example, a turn, a lane change, a change in acceleration, etc. The processing unit 110 may, based on the analysis performed at step 720 and the above combined Figure 4 The processing unit 110 may also use data derived from execution of the velocity and acceleration module 406 to cause one or more navigation responses. In some embodiments, the processing unit 110 may cause one or more navigation responses based on the relative position, relative velocity, and / or relative acceleration between the vehicle 200 and an object detected within any of the first, second, and third pluralities of images. The multiple navigation responses may occur simultaneously, sequentially, or any combination thereof.

[0207] Reinforcement learning and trained navigation systems

[0208] Subsequent sections discuss automated driving and systems and methods for enabling autonomous control of a vehicle, whether that control is fully autonomous (a self-driving vehicle) or partially autonomous (e.g., one or more driver assistance systems or functions). Figure 8As shown, the autonomous driving task can be divided into three main modules, including a sensing module 801, a driving strategy module 803, and a control module 805. In some embodiments, modules 801, 803, and 805 can be stored in the memory unit 140 and / or the memory unit 150 of the system 100, or modules 801, 803, and 805 (or portions thereof) can be stored remotely from the system 100 (e.g., stored in a server accessible to the system 100 via, for example, the wireless transceiver 172). In addition, any module disclosed herein (e.g., modules 801, 803, and 805) can implement techniques associated with a trained system (such as a neural network or a deep neural network) or an untrained system.

[0209] The sensing module 801, which may be implemented using the processing unit 110, can handle various tasks related to sensing the navigational state of the host vehicle's environment. Such tasks may rely on input from various sensors and sensing systems associated with the host vehicle. These inputs may include images or image streams from one or more onboard cameras, GPS location information, accelerometer output, user feedback, or user input to one or more user interface devices, radar, lidar, etc. The sensing (which may include data from cameras and / or any other available sensors) may be collected, analyzed, and converted into a "sensed state" along with map information, describing information extracted from the scene in the host vehicle's environment. The sensed state may include sensed information related to target vehicles, lane markings, pedestrians, traffic lights, road geometry, lane shape, obstacles, distance to other objects / vehicles, relative velocity, relative acceleration, and any other potential sensed information. Supervised machine learning may be implemented to generate a sensed state output based on the sensed data provided to the sensing module 801. The output of the sensing module may represent the sensed navigational "state" of the host vehicle, which may be passed to the driving strategy module 803.

[0210] While the sensed state can be developed based on image data received from one or more cameras or image sensors associated with the host vehicle, any suitable sensor or combination of sensors can be used to develop the sensed state for use in navigation. In some embodiments, the sensed state can be developed without relying on captured image data. In fact, any of the navigation principles described herein can be applied to sensed states developed based on captured image data as well as sensed states developed using other non-image-based sensors. The sensed state can also be determined via a source external to the host vehicle. For example, the sensed state can be developed based entirely or in part on information received from a source remote from the host vehicle (e.g., based on sensor information, processed state information, etc. shared from other vehicles, shared from a central server, or from any other information source relevant to the navigation state of the host vehicle).

[0211] The driving strategy module 803, discussed in more detail below and which can be implemented using the processing unit 110, can implement a desired driving strategy to determine one or more navigation actions to be taken by the host vehicle in response to the sensed navigation state. If there are no other agents (e.g., target vehicles or pedestrians) in the host vehicle's environment, the sensed state input to the driving strategy module 803 can be handled in a relatively straightforward manner. When the sensed state requires negotiation with one or more other agents, the task becomes more complex. Techniques for generating the output of the driving strategy module 803 can include reinforcement learning (discussed in more detail below). The output of the driving strategy module 803 can include at least one navigation action for the host vehicle and can include a desired acceleration (which can be converted into an updated velocity of the host vehicle), a desired yaw rate of the host vehicle, a desired trajectory, and other potential desired navigation actions.

[0212] Based on the output from the driving strategy module 803, the control module 805, which may also be implemented using the processing unit 110, can develop control instructions for one or more actuators or controlled devices associated with the host vehicle. Such actuators and devices may include an accelerator, one or more steering controls, brakes, a signal transmitter, a display, or any other actuator or device that can be controlled as part of navigation operations associated with the host vehicle. Aspects of control theory can be used to generate the output of the control module 805. The control module 805 can be responsible for developing and outputting instructions to the controllable components of the host vehicle to implement the desired navigation objectives or requirements of the driving strategy module 803.

[0213] Returning to the driving policy module 803, in some embodiments, a trained system trained via reinforcement learning can be used to implement the driving policy module 803. In other embodiments, the driving policy module 803 can be implemented without machine learning methods by using a specified algorithm to "manually" address various scenarios that may arise during autonomous navigation. However, while feasible, this approach can result in overly simplistic driving policies and may lack the flexibility of a trained system based on machine learning. For example, a trained system may be better equipped to handle complex navigation states and may be better able to: determine whether a taxi is parked or stopped to pick up or drop off a passenger; determine whether a pedestrian intends to cross the street in front of the host vehicle; balance the unexpected behavior of other drivers with their defensiveness; negotiate dense traffic involving target vehicles and / or pedestrians; decide when to suspend certain navigation rules or enhance others; anticipate unsensed but anticipated situations (e.g., whether a pedestrian will emerge from behind a car or obstacle); and so on. A trained system based on reinforcement learning may also be better equipped to address continuous and high-dimensional state spaces and continuous action spaces.

[0214] Training the system using reinforcement learning can involve learning a driving policy to map from sensed states to navigation actions. The driving policy is a function π:S→A, where S is a set of states and is the action space (e.g., desired velocity, acceleration, yaw command, etc.). The state space is S = S s ×S p , where S s is the sensing state, and S p is additional information about the state saved by the policy. In the case of working in discrete time intervals, at time t, the current state S can be observed t ∈S, and the policy can be applied to obtain the desired action a t =π(S t ).

[0215] The system can be trained by exposing it to various navigation states, having it apply policies, and providing rewards (based on a reward function designed to reward desired navigation behavior). Based on the reward feedback, the system can "learn" policies and be trained to produce the desired navigation actions. For example, the learning system can observe the current state S t ∈S and based on the policy To decide action a t ∈A. Based on the action decided (and the execution of the action), the environment moves to the next state S t+1 ∈ S for the learning system to observe. For each action developed in response to the observed state, the feedback to the learning system is a reward signal r1,r2,….

[0216] The goal of reinforcement learning (RL) is to discover a policy π. It is usually assumed that at time t, there exists a reward function r t , the reward function measures the state s t and take action a t However, at time t, action a is taken t will affect the environment and, therefore, the value of future states. Therefore, when deciding what action to take, not only the current reward should be considered, but also the future reward. In some cases, when the system determines that taking a lower reward option now could yield a larger reward in the future, it should take an action even if that action is associated with a lower reward than another available option. To formalize this, observe that the policy π and the initial state s give rise to The distribution on which the vector (r1,…,r T ) is the probability of observing rewards r1,…,r T The probability of (if the agent starts from state s0=s and follows the policy π from then on). The value of the initial state s can be defined as:

[0217]

[0218] For some fixed γ∈(0,1), we can define it by discounting future rewards instead of restricting the time horizon to T:

[0219]

[0220] In any case, the optimal strategy is the following solution:

[0221]

[0222] where the expectation exceeds the initial state s.

[0223] There are several possible approaches for training a driving policy system. For example, one can use imitation methods (e.g., behavioral cloning), where the system learns from state / action pairs, where the action is the action chosen by a good agent (e.g., a human) in response to a particular observed state. Suppose a human driver is observed. Through this observation, many examples of the form (s t ,a t )(where s t is a state and a t is the action of the human driver) and use it as the basis for training a driving policy system. For example, supervised learning can be used to learn a policy π such that π(s t ≈a t). This approach has many potential advantages. First, it does not require the definition of a reward function. Second, learning is supervised and occurs offline (no agent is required during learning). A disadvantage of this approach is that different human drivers, and even the same human driver, are not deterministic in their choice of strategy. Therefore, learning ||π(s t )-a t || Very small functions are often infeasible. Moreover, even small errors can accumulate over time, resulting in large errors.

[0224] Another technique that can be used is policy-based learning. Here, the policy can be expressed in parameter form and optimized directly using a suitable optimization technique (e.g., stochastic gradient descent). This approach is to directly solve There are, of course, many ways to solve this problem. One advantage of this approach is that it tackles the problem directly and therefore often leads to good practical results. A potential disadvantage is that it often requires "online policy" training, i.e., the learning of π is an iterative process where at iteration j we obtain a non-perfect policy π. j , and in order to construct the next strategy π j , we must interact with the environment and based on π j To make a move.

[0225] The system can also be trained by value-based learning (learning Q or V functions). If a good approximation to the optimal value function V* can be learned, an optimal policy can be constructed (e.g., by relying on the Bellman equation). Some forms of value-based learning can be performed offline (referred to as "off-policy" training). Some drawbacks of value-based methods may arise from their strong reliance on Markov assumptions and the required approximation of complex functions (approximating the value function can be more difficult than directly approximating the policy).

[0226] Another technique may include model-based learning and planning (learning the probabilities of state transitions and solving the optimization problem of finding the best V). A combination of these techniques can also be used to train a learning system. In this approach, the dynamics of the process can be learned, i.e., using (s t ,a t ) and generate the next state s t+1 Once this function is learned, an optimization problem can be solved to find the policy π that best represents its value. This is called "planning". An advantage of this approach can be that the learning part is supervised and can be learned by observing the triples (s t ,a t ,s t+1) for offline application. Similar to the "imitation" approach, a drawback of this approach may be that small errors in the learning process may accumulate and produce an underperforming policy.

[0227] Another approach for training the driving policy module 803 can include decomposing the driving policy function into semantically meaningful components. This allows for manual implementation of portions of the policy, which ensures policy safety, and for implementation of other portions of the policy using reinforcement learning techniques, which allows for adaptability to a wide range of scenarios, a personalized balance between defensive and aggressive behavior, and personalized negotiation with other drivers. From a technical perspective, reinforcement learning methods can combine several approaches and provide a tractable training protocol, where the majority of the training can be performed using recorded data or a custom simulator.

[0228] In some embodiments, the training of the driving policy module 803 can rely on an "option" mechanism. To illustrate, consider a simple scenario of a driving policy for a two-lane highway. In a direct RL approach, the policy π maps the state to In the modified method, the following strategy can be constructed:

[0229] Automatic Cruise Control (ACC) Strategy ACC :S→A: This strategy always outputs a yaw rate of 0 and only changes the speed in order to implement smooth and accident-free driving.

[0230] ACC+Left Strategy O L :S→A: The longitudinal commands for this strategy are the same as those for ACC. The yaw rate is simply to enforce centering of the vehicle to the middle of the left lane while ensuring safe lateral movement (e.g., not moving left if there is a car to the left).

[0231] ACC+Right Strategy O R :S→A:with o L Same, but the vehicle may center toward the middle of the right lane.

[0232] These strategies can be called "options". Based on these "options", we can learn the strategy π of choosing options o :S→O, where O is a set of available options. In one case, O={o ACC ,o L ,o R}. Option selector strategy π o By setting To define the actual policy π:S→A.

[0233] In practice, the policy function can be decomposed into an option graph 901, such as Figure 9 shown. Figure 10 Another example option graph 1000 is shown in FIG. An option graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There is a special node, called the root node 903 of the graph. This node has no incoming nodes. The decision process traverses the graph starting from the root node until it reaches a "leaf" node, which is a node with no outgoing decision lines. Figure 9 As shown, leaf nodes may include, for example, nodes 905, 907, and 909. Upon encountering a leaf node, the driving strategy module 803 may output acceleration and steering commands associated with the desired navigation maneuver associated with the leaf node.

[0234] For example, internal nodes such as nodes 911, 913, and 915 may result in the implementation of a strategy for selecting children among their available options. The set of available children for an internal node includes all nodes associated with the particular internal node via a decision line. For example, Figure 9 The internal node 913 designated as “Merge” includes three child nodes 909 , 915 and 917 (“Keep”, “Overtake Right” and “Overtake Left”, respectively), each of which is connected to the node 913 via a decision line.

[0235] The flexibility of the decision-making system can be obtained by enabling nodes to adjust their positions in the hierarchical structure of the option graph. For example, any one of the nodes can be allowed to declare itself as "critical". Each node can implement a "critical" function, which outputs "true" if the node is in a critical section for its policy implementation. For example, the node responsible for overtaking can declare itself as critical during maneuvers. This may impose constraints on the set of available children of node u, which may include all nodes v, which are children of node u and for which there is a path from v to a leaf node, which passes through all nodes designated as critical. On the one hand, this approach allows the desired path on the graph to be declared at each time step, and on the other hand, the stability of the strategy can be retained, especially when implementing the critical part of the strategy.

[0236] By defining an option graph, the problem of learning a driving policy π:S→A can be decomposed into the problem of defining a policy for each node in the graph, where the policy at the internal node should be selected from the available child nodes. For some of the nodes, the corresponding policy can be implemented manually (e.g., by specifying a set of actions in response to the observed state through an if-then type algorithm), while for other nodes, a trained system constructed through reinforcement learning can be used to implement the policy. The choice between a manual approach or a trained / learned approach can depend on the safety aspects related to the task and the relative simplicity of the task. The option graph can be constructed in such a way that some of the nodes are easy to implement, while other nodes can rely on a trained model. This approach can ensure safe operation of the system.

[0237] The following discussion provides relevant Figure 9 Further details on the role of the option map within the driving strategy module 803 are provided. As discussed above, the input to the driving strategy module is a "sensed state," which summarizes, for example, a map of the environment obtained from available sensors. The output of the driving strategy module 803 is a set of expectations (optionally along with a set of hard constraints) that define the trajectory as a solution to the optimization problem.

[0238] As described above, the option graph represents a hierarchical set of decisions organized as a DAG. There is a special node called the "root" of the graph. The root node is the only node with no incoming edges (e.g., decision lines). The decision process traverses the graph starting from the root node until it reaches a "leaf" node, that is, a node with no outgoing edges. Each internal node should implement a policy that selects children from the available children of the internal node. Each leaf node should implement a policy that defines a set of expectations (e.g., a set of navigation goals for the host vehicle) based on the entire path from the root to the leaf. This set of expectations, together with a set of hard constraints defined directly based on the sensed states, establishes an optimization problem whose solution is a trajectory for the vehicle. The hard constraints can be used to further improve the safety of the system, and the expectations can be used to provide the system with driving comfort and human-like driving behavior. The trajectory provided as the solution to the optimization problem then defines the commands that should be provided to the steering, braking, and / or engine actuators to complete the trajectory.

[0239] return Figure 9Option graph 901 represents an option graph for a two-lane highway, including a merging lane (meaning that at some point, a third lane merges into the right or left lane of the highway). Root node 903 first determines whether the host vehicle is in a normal road scenario or an approaching merging scenario. This is an example of a decision that can be made based on the sensing state. Normal road node 911 includes three child nodes: Hold node 909, Pass Left node 917, and Pass Right node 915. Hold refers to a situation where the host vehicle wishes to continue driving in the same lane. The Hold node is a leaf node (without outgoing edges / lines). Therefore, the Hold node defines a set of expectations. The first expectation defined by the Hold node may include a desired lateral position—for example, as close to the center of the current lane as possible. There may also be a desire to navigate smoothly (for example, within a predefined or allowed maximum acceleration). The Hold node may also define how the host vehicle should react to other vehicles. For example, the Hold node may check for sensed target vehicles and assign semantic meaning to each sensed target vehicle, which can be translated into components of the trajectory.

[0240] Various semantic meanings can be assigned to target vehicles in the host vehicle's environment. For example, in some embodiments, the semantic meanings can include any of the following assignments: 1) Not relevant: indicates that the sensed vehicle in the scene is currently not relevant; 2) Next lane: indicates that the sensed vehicle is in an adjacent lane and should maintain an appropriate offset relative to this vehicle (the exact offset can be calculated in the optimization problem that constructs the trajectory given the desired and hard constraints, and can be vehicle-dependent - the hold leaf of the option graph sets the semantic type of the target vehicle, which defines the desired relative to the target vehicle); 3) Yield: The host vehicle will attempt to yield to the sensed target vehicle by, for example, reducing speed (especially if the host vehicle determines that the target vehicle may cut into the host vehicle's lane); 4) Hold lane: The host vehicle will attempt to gain right-of-way by, for example, increasing speed; 5) Follow: The host vehicle desires to maintain a smooth drive following this target vehicle; 6) Pass left / right: This means that the host vehicle wishes to initiate a lane change to the left or right lane. Pass left node 917 and Pass right node 915 are internal nodes for which no desired expectations have yet been defined.

[0241] The next node in option graph 901 is select gap node 919. This node may be responsible for selecting a gap between two target vehicles in a particular target lane that the host vehicle desires to enter. By selecting a node of the form IDj, for some value of j, the host vehicle reaches the leaf that assigns the desired outcome to the trajectory optimization problem, e.g., the host vehicle desires to maneuver to reach the selected gap. This maneuver may involve first accelerating / braking in the current lane and then moving to the target lane at the appropriate time to enter the selected gap. If select gap node 919 cannot find a suitable gap, it moves to terminate node 921, which defines the desired outcome of moving back to the center of the current lane and canceling the desired outcome of the overtaking.

[0242] Returning to the merge node 913, when the host vehicle is approaching a merge, the host vehicle has several options that may depend on the particular situation. Figure 11A As shown, the host vehicle 1105 is traveling along a two-lane road, and no other target vehicles are detected in the main lane of the two-lane road or in the merging lane 1111. In this case, the driving strategy module 803 may select the maintain node 909 upon reaching the merge node 913. In other words, if no target vehicle is sensed when merging onto the roadway, it may be desirable to maintain its current lane.

[0243] exist Figure 11B In Figure 1, the situation is slightly different. Here, the host vehicle 1105 senses one or more target vehicles 1107 entering the main roadway 1112 from the merging lane 1111. In this case, once the driving strategy module 803 encounters the merging node 913, the driving strategy module may choose to initiate a left passing maneuver to avoid the merging situation.

[0244] exist Figure 11C In FIG, the host vehicle 1105 encounters one or more target vehicles 1107 entering the main roadway 1112 from the merging lane 1111. The host vehicle 1105 also detects a target vehicle 1109 traveling in a lane adjacent to the host vehicle's lane. The host vehicle also detects one or more target vehicles 1110 traveling in the same lane as the host vehicle 1105. In this case, the driving policy module 803 can decide to adjust the speed of the host vehicle 1105 to give way to the target vehicle 1107 and continue to travel in front of the target vehicle 1115. This can be accomplished by, for example, progressing to the select interval node 919, which in turn selects the interval between ID0 (vehicle 1107) and ID1 (vehicle 1115) as the appropriate merging interval. In this case, the appropriate interval for the merging situation defines the purpose of the trajectory planner optimization problem.

[0245] As discussed above, nodes of the option graph can declare themselves as "critical", which ensures that the selected options pass through critical nodes. Formally, each node can implement the function IsCritical. After performing a forward pass from root to leaf on the option graph and solving the trajectory planner's optimization problem, a backward pass can be performed from the leaf back to the root. Along this backward pass, the IsCritical function of all nodes in the pass can be called, and a list of all critical nodes can be saved. In the forward path corresponding to the next time frame, the driving policy module 803 may be required to select a path from the root node to the leaf that passes through all critical nodes.

[0246] Figures 11A to 11C Can be used to illustrate the potential benefits of this approach. For example, when an overtaking action is initiated and the driving strategy module 803 reaches the leaf corresponding to IDk, it is not desirable to select, for example, to hold node 909 while the host vehicle is in the middle of an overtaking maneuver. To avoid this jump, the IDj node can designate itself as critical. During the maneuver, the success of the trajectory planner can be monitored, and if the overtaking maneuver progresses as expected, the function IsCritical will return a "true" value. This approach can ensure that in the next time frame, the overtaking maneuver will continue (rather than jumping to another possibly inconsistent maneuver before completing the initially selected maneuver). On the other hand, if the monitoring of the maneuver indicates that the selected maneuver did not progress as expected, or if the maneuver has become unnecessary or impossible, the function IsCritical can return a "false" value. This can allow the selection interval node to select a different interval in the next time frame, or to terminate the overtaking maneuver completely. On the one hand, this approach can allow the desired path on the option graph to be declared at each time step, and on the other hand, it can help improve the stability of the strategy in the critical part of the execution.

[0247] Hard constraints, which will be discussed in more detail below, can be distinguished from navigation expectations. For example, hard constraints can ensure safe driving by applying an additional layer of filtering to planned navigation actions. The hard constraints involved can be determined based on the sensed state, which can be programmed and defined manually rather than by using a trained system constructed based on reinforcement learning. However, in some embodiments, the trained system can learn the applicable hard constraints to apply and follow. This approach can encourage the driving policy module 803 to reach selected actions that already comply with the applicable hard constraints, which can reduce or eliminate selected actions that may need to be modified later to comply with the applicable hard constraints. However, as a redundant safety measure, hard constraints can be applied to the output of the driving policy module 803 even if the driving policy module 803 has been trained to account for predetermined hard constraints.

[0248] There are many examples of potential hard constraints. For example, a hard constraint may be defined in conjunction with a guardrail at the edge of the road. Under no circumstances may the host vehicle be allowed to pass the guardrail. Such a rule would create a hard lateral constraint on the trajectory of the host vehicle. Another example of a hard constraint may include a road bump (e.g., a speed control bump) that may create a hard constraint on the speed of driving before and across the bump. Hard constraints may be considered safety critical and, therefore, may be defined manually rather than relying solely on a trained system that learns the constraints during training.

[0249] In contrast to hard constraints, the purpose of a desire may be to achieve or attain comfortable driving. As discussed above, an example of a desire may include a goal of positioning the host vehicle within a lane at a lateral position corresponding to the center of the host vehicle's lane. Another desire may include the ID of the interval to be coordinated. Note that the host vehicle is not required to be exactly in the center of the lane, but the desire to be as close to the center of the lane as possible may ensure that the host vehicle tends to migrate to the center of the lane even if it deviates from the center of the lane. The desire may not be safety critical. In some embodiments, the desire may require negotiation with other drivers and pedestrians. One method for constructing the desire may rely on an option graph, and the policies implemented in at least some nodes of the graph may be based on reinforcement learning.

[0250] For nodes of the option graph 901 or 1000 that are implemented as nodes trained based on learning, the training process may include breaking the problem into a supervised learning phase and a reinforcement learning phase. In the supervised learning phase, it is possible to learn from (s t ,a t )arrive A differentiable mapping such that This can be analogous to "model-based" reinforcement learning. However, in the forward loop of the network, Can be s t+1 The actual value is replaced by the actual value, thus eliminating the problem of error accumulation. The predictive role of is to propagate information from the future back to past actions. In this sense, the algorithm can be a combination of "model-based" reinforcement learning and "policy-based learning."

[0251] An important element that can be provided in some scenarios is a differentiable path from future losses / rewards back to the decision about the action. With the option graph structure, the implementation of options involving safety constraints is usually not differentiable. To overcome this problem, the selection of children in the learned policy node can be random. That is, the node can output a probability vector p that assigns the probability of using when selecting each of the children of a particular node. Suppose a node has k children, and let a (1) ,…,a(k) is the action of the path from each child to the leaf. Therefore, the predicted action is This can produce a differentiable path from action to p. In practice, for i~p, action a can be chosen as a (i) , and a and The difference between them can be called additive noise.

[0252] For a given s t ,a t Time For training, supervised learning can be used with real-world data. For training, a policy from a node simulator can be used. Later, fine-tuning of the policy can be done using real-world data. Two ideas can make the simulation more realistic. First, through imitation, a "behavioral cloning" paradigm can be used to build an initial policy using large real-world datasets. In some cases, the resulting agent can be suitable. In other cases, the resulting agent at least forms a very good initial policy for other agents on the road. Second, using self-play, training can be enhanced with our own policy. For example, a policy can be trained based on a simulator, given an initial implementation of other agents (cars / pedestrians) that may be experienced. Some of the other agents can be replaced with the new policy, and the process can be repeated. Therefore, the policy can be continuously improved because it should respond to a wider variety of other agents with different levels of sophistication.

[0253] Furthermore, in some embodiments, the system can implement a multi-agent approach. For example, the system can consider data from various sources and / or images captured from multiple angles. Furthermore, some disclosed embodiments can provide energy economy because anticipation of events that do not directly involve the host vehicle but may have an impact on the host vehicle can be factored in, or even anticipation of events that may lead to unpredictable situations involving other vehicles can be factored in (e.g., radar can "see through" the lead vehicle and anticipate unavoidable events, or even anticipate a high probability of an event that will affect the host vehicle).

[0254] Trained system with imposed navigation constraints

[0255] In the context of autonomous driving, a significant concern is ensuring that the learned policy of a trained navigation network will be safe. In some embodiments, constraints can be used to train the driving policy system so that the actions selected by the trained system can account for applicable safety constraints. Furthermore, in some embodiments, an additional layer of safety can be provided by subjecting the actions selected by the trained system to one or more hard constraints related to a specific sensed scenario in the host vehicle's environment. This approach can ensure that the actions taken by the host vehicle are limited to those that are confirmed to satisfy applicable safety constraints.

[0256] At its core, the navigation system may include a learning algorithm based on a policy function that maps observed states to one or more desired actions. In some embodiments, the learning algorithm is a deep learning algorithm. The desired action may include at least one action that is expected to maximize the expected reward for the vehicle. While in some cases, the actual action taken by the vehicle may correspond to one of the desired actions, in other cases, the actual action taken may be determined based on the observed state, the one or more desired actions, and non-learned hard constraints (e.g., safety constraints) imposed on the learning navigation engine. These constraints may include a no-driving zone around various types of detected objects (e.g., target vehicles, pedestrians, stationary objects on the side of the road or in the roadway, moving objects on the side of the road or in the roadway, guardrails, etc.). In some cases, the size of the zone may vary based on the detected motion (e.g., speed and / or direction) of the detected object. Other constraints may include a maximum travel speed when passing within the influence zone of a pedestrian, a maximum deceleration (to account for the distance between target vehicles behind the host vehicle), mandatory stops at sensed crosswalks or railroad crossings, and the like.

[0257] Hard constraints used in conjunction with systems trained via machine learning can provide a degree of safety in autonomous driving that may exceed that available based solely on the outputs of the trained system. For example, a machine learning system can be trained using a desired set of constraints as a training guide, and thus, the trained system can select actions in response to sensed navigation states that account for and adhere to the limitations of applicable navigation constraints. However, the trained system still has some flexibility in selecting navigation actions, and therefore, there may be at least some situations in which the actions selected by the trained system may not strictly adhere to the relevant navigation constraints. Therefore, in order to require that the selected actions strictly adhere to the relevant navigation constraints, the outputs of the trained system can be combined, compared, filtered, adjusted, modified, etc. using non-machine learning components outside of the learning / trained framework that ensure the strict application of the relevant navigation constraints.

[0258] The following discussion provides additional details about the trained system and the potential benefits (particularly from a safety perspective) that can be obtained by combining the trained system with algorithmic components outside the training / learning framework. As discussed, the reinforcement learning objective of the policy can be optimized by stochastic gradient ascent. The objective (e.g., expected reward) can be defined as

[0259] Goals involving expectations can be used in machine learning scenarios. However, unbounded by navigation constraints, such goals may not return actions strictly bounded by those constraints. For example, consider a reward function where, for a trajectory representing a rare "corner" event to be avoided (e.g., such as an accident), For the rest of the trajectories, One purpose of a learning system can be to learn to perform an overtaking maneuver. Typically, in an accident-free trajectory, rewards successful, smooth overtakes and penalizes remaining in the lane without completing the overtake—hence the range [-1,1]. represents an accident, then the reward -r should provide a high enough penalty to prevent this from happening. The question is what value of r should be to ensure accident-free driving.

[0260] Observed the accident The effect of is the additive term -pr, where p is the probability mass of the trajectory of the accident event. If this term is negligible, that is, Then the learning system may prefer a strategy that makes accidents (or generally adopts a reckless driving strategy) so that it can successfully perform overtaking maneuvers more often than a strategy that is more defensive but at the expense of some overtaking maneuvers not being successfully completed. In other words, if the probability of an accident will be at most p, then r must be set so that It is desirable to make p very small (e.g., approximately p = 10 -9 ). Therefore, r should be large. In terms of policy gradient, it can be estimated that The following lemma shows that the random variable The variance of pr 2 Growth, for For , this variance is larger than r. Therefore, estimating the purpose can be difficult, and estimating its gradient can be even more difficult.

[0261] Lemma: Let π o Let p and r be scalars, so that with probability p, we get And with probability 1-p, we obtain but,

[0262]

[0263] where the last approximation applies to the case

[0264] This discussion shows that the form The objection may not ensure functional safety without causing variance problems. The baseline subtraction method for variance reduction may not provide an adequate remedy for this problem because the problem will The high variance of translates into an equally high variance of the baseline constant whose estimate will also suffer from numerical instability. Moreover, if the probability of an accident is p, then on average one should have at least This means that the purpose is to Minimization of learning algorithms The solution to this problem can be found in the architectural design described in this paper, rather than through numerical tuning techniques. The approach here is based on the concept that hard constraints should be injected outside the learning framework. In other words, the policy function can be decomposed into a learnable part and a non-learnable part. Formally, the policy function can be structured as in Map the (agnostic) state space to a set of desires (e.g., desired navigation goals, etc.), and π (T) Maps expectations to trajectories (which determine how the car should move in the short term). Function Responsible for driving comfort and making strategic decisions such as which other cars should be overtaken or yielded to and the desired position of the host vehicle in its lane. The mapping from the sensed navigation state to the desired state is the policy This policy can be learned from experience by maximizing the expected reward. The generated expectation can be transformed into a cost function on the driving trajectory. Function π (T) (Not a learned function) can be implemented by finding a trajectory that minimizes the cost under hard constraints on functional safety. This decomposition can ensure functional safety while providing comfortable driving.

[0265] like Figure 11DAs depicted, a double merge navigation scenario provides an example to further illustrate these concepts. In a double merge, vehicles approach the merging area 1130 from both the left and right sides. And vehicles such as vehicle 1133 or vehicle 1135 can decide from each side whether to merge into the lane on the other side of the merging area 1130. Successfully executing a double merge in busy traffic can require considerable negotiation skill and experience, and can be difficult to perform with a heuristic or brute force approach by enumerating all possible trajectories that can be taken by all agents in the scene. In this double merge example, a set of expectations appropriate for the double merge maneuver can be defined. Can be the Cartesian product of the following sets: where [0,v 最大 ] is the desired target speed of the host vehicle, L = {1, 1.5, 2, 2.5, 3, 3.5, 4} is the desired lateral position in lane units, where integers denote lane centers and decimals denote lane edges, and {g, t, o} is a classification label assigned to each of the n other vehicles. The other vehicles may be assigned “g” if the host vehicle is to yield to the other vehicle, “t” if the host vehicle is to occupy the lane relative to the other vehicle, or “o” if the host vehicle is to maintain an offset distance relative to the other vehicle.

[0266] Here is how to set a set of expectations Transformed into a description of the cost function on the driving trajectory. The driving trajectory can be represented by (x1, y1),…, (x k ,y k ) means, where (x i ,y i ) is in time The (lateral, longitudinal) position of the host vehicle (in the egocentric unit) when . In some experiments, seconds and k = 10. Of course, other values ​​may be chosen. The cost assigned to the trajectory may comprise a weighted sum of the individual costs assigned to: the desired speed, the lateral position, and the labels assigned to each of the other n vehicles.

[0267] Given the expected velocity v∈[0,v 最大 ], the cost of the trajectory associated with the velocity is

[0268]

[0269] Given a desired lateral position l∈L, the cost associated with the desired lateral position is

[0270]

[0271] The distance (x i ,yi ,l) is the distance from point (x,y) to lane position l. Regarding the cost due to other vehicles, for any other vehicle, (x′1,y1′),…,(x′ k ,y′ k ) can represent another vehicle in the egocentric unit of the host vehicle, and i can be that there exists j such that (x i ,y i ) and (x j ′,y j ′ If there is no such point, then i can be set to i=∞. If another car is classified as "give way", then it can be expected that This means that the host vehicle will reach the track intersection point at least 0.5 seconds after the other vehicle reaches the same point. A possible formula for converting the above constraints into costs is

[0272] Likewise, if another car is classified as "blocking the road," one can expect This translates into costs If the other car is classified as "offset", then we can expect i = ∞, meaning that the trajectory of the host vehicle and the trajectory of the offset car do not intersect. This condition can be converted into a cost by penalizing the distance between the trajectories.

[0273] Assigning weights to each of these costs provides a single objective function π for the trajectory planner (T) A cost that encourages smooth driving can be added to the objective. Moreover, to ensure the functional safety of the trajectory, hard constraints can be added to the objective. For example, if |ij| is small, then for any trajectory point (x j ′ ,y j ′ ), you can prohibit (x i ,y i ) leave the roadway and may not allow (x i ,y i )Close to (x j ′ ,y j ′ ).

[0274] In summary, strategy π θ can be decomposed into a mapping from unrelated states to a set of desired states and a mapping from desired states to actual trajectories. The latter mapping is not based on learning and can be implemented by solving an optimization problem whose cost depends on the desired states and whose hard constraints guarantee the functional safety of the policy.

[0275] The following discussion describes the mapping from irrelevant states to this set of expectations. As described above, a system that relies solely on reinforcement learning to comply with functional safety may suffer from the reward This result can be avoided by using policy gradients to iteratively decompose the problem into a mapping from an (unrelated) state space to a set of desired trajectories, and then to actual trajectories without involving a machine learning-based training system.

[0276] For various reasons, decision making may be further decomposed into semantically meaningful components. For example, The size of may be large and even continuous. Figure 11D In the double merge scenario described, Additionally, the gradient estimator can involve the term In this expression, the variance can increase over the time range T. In some cases, the value of 'T' can be approximately 250, which may be high enough to produce significant variance. If the sampling rate is in the range of 10 Hz and the merging area 1130 is 100 meters, then preparation for the merge can begin approximately 300 meters before the merging area. If the host vehicle is traveling at 16 meters per second (approximately 60 kilometers per hour), the value of T for the round can be approximately 250.

[0277] Back to the idea of ​​the option map, Figure 11E It is shown that it can be expressed Figure 11D Option graph for a double merge scenario depicted. As previously discussed, an option graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There can be a special node in the graph called a "root" node 1140, which can be the only node with no incoming edges (e.g., decision lines). The decision process can traverse the graph starting from the root node until the decision process reaches a "leaf" node, i.e., a node with no outgoing edges. Each internal node can implement a policy function that selects a child from among its available children. There can be a set of desired choices traversed from a set on the option graph to the set of desired choices. In other words, traversals on the option graph can be automatically transformed into Given a node v in the graph, the parameter vector θ v A strategy for selecting children of v can be specified. If θ is all θ v cascade, then It can be defined as follows: traverse from the root of the graph to the leaves, while at each node v using the v Defines the strategy to select child nodes.

[0278] exist Figure 11EIn the dual merging option diagram 1139, the root node 1140 may first determine whether the host vehicle is in the merging area (e.g., Figure 11D 1130) or if the host vehicle is instead approaching a merging area and needs to prepare for a possible merge. In both situations, the host vehicle may need to decide whether to change lanes (e.g., to the left or to the right) or whether to remain in the current lane. If the host vehicle has already decided to change lanes, the host vehicle may need to decide whether conditions are suitable to proceed and perform a lane change maneuver (e.g., at "go" node 1142). If a lane change is not possible, the host vehicle may (e.g., at node 1144, as part of negotiating with the vehicle in the desired lane) attempt to "advance" toward the desired lane with the goal of being on the lane markings. Alternatively, the host vehicle may choose to "remain" in the same lane (e.g., at node 1146). This process may determine the lateral position of the host vehicle in a natural manner. For example,

[0279] This can enable the desired lateral position to be determined in a natural way. For example, if the host vehicle changes lanes from lane 2 to lane 3, the "pass" node can set the desired lateral position to 3, the "hold" node can set the desired lateral position to 2, and the "advance" node can set the desired lateral position to 2.5. Next, the host vehicle can decide whether to maintain the "same" speed (node ​​1148), "speed up" (node ​​1150), or "slow down" (node ​​1152). Next, the host vehicle can enter a "chain" structure 1154 that passes through other vehicles and sets their semantic meaning to values ​​in the set {g, t, o}. This process can set expectations relative to other vehicles. The parameters of all nodes in this chain can be shared (similar to a recurrent neural network).

[0280] A potential benefit of this option is the interpretability of the results. Another potential benefit is the ability to rely on the set The decomposable structure of , and therefore, the policy at each node can be selected from a small number of possibilities. In addition, this structure can allow reducing the variance of the policy gradient estimator.

[0281] As discussed above, the length of a turn in a double merge scenario can be approximately T = 250 steps. This value (or any other suitable value depending on the specific navigation scenario) can provide sufficient time to see the consequences of the host vehicle's actions (e.g., if the host vehicle decides to change lanes in preparation for a merge, the host vehicle will only see the benefits after successfully completing the merge). On the other hand, due to the dynamic nature of driving, the host vehicle must reach decisions at a sufficiently high frequency (e.g., 10 Hz in the above scenario).

[0282] The option graph can achieve a reduction in the effective value of T in at least two ways. First, given higher-level decisions, rewards can be defined for lower-level decisions while taking into account shorter episodes. For example, when the host vehicle has selected the "Lane Change" and "Pass" nodes, a strategy for assigning semantic meaning to the vehicle can be learned by observing 2 to 3 second episodes (meaning T becomes 20 to 30 instead of 250). Second, for high-level decisions (such as whether to change lanes or stay in the same lane), the host vehicle may not need to reach a decision every 0.1 seconds. Instead, the host vehicle may be able to reach a decision less frequently (e.g., once per second) or implement an "option expiration" function and then calculate the gradient only after each option is terminated. In both cases, the effective value of T may be an order of magnitude smaller than its original value. In summary, the estimator at each node can depend on a value of T that is an order of magnitude smaller than the original 250 steps, which may immediately shift to smaller variance.

[0283] As discussed above, hard constraints can promote safer driving, and there can be several different types of constraints. For example, static hard constraints can be defined directly from the sensed states. These constraints can include deceleration bumps, speed limits, road curvatures, intersections, etc. within the host vehicle's environment, which can involve one or more constraints on vehicle speed, heading, acceleration, braking (deceleration), etc. Static hard constraints can also include semantic free space, where, for example, the host vehicle is prohibited from passing outside of free space and navigating too close to physical barriers. Static hard constraints can also limit (e.g., prohibit) maneuvers that are inconsistent with various aspects of the vehicle's kinematic motion. For example, static hard constraints can be used to prohibit maneuvers that may cause the host vehicle to overturn, slide, or otherwise lose control.

[0284] Hard constraints can also be associated with the vehicle. For example, a constraint can be employed that requires the vehicle to maintain a longitudinal distance of at least one meter from other vehicles and a lateral distance of at least 0.5 meters from other vehicles. Constraints can also be applied such that the host vehicle avoids maintaining a collision course with one or more other vehicles. For example, time τ can be a time measure based on a particular scenario. The predicted trajectories of the host vehicle and one or more other vehicles from the current time to time τ can be considered. In the event that two trajectories intersect, We can denote the arrival and departure times of vehicle i at an intersection. That is, each car arrives at the intersection when the first part of the car passes through the intersection, and it takes a certain amount of time before the last part of the car passes through the intersection. This amount of time separates the arrival time from the departure time. Assume (i.e., the arrival time of vehicle 1 is less than the arrival time of vehicle 2), we will need to ensure that vehicle 1 has left the intersection before vehicle 2 arrives. Otherwise, a collision will result. Therefore, a hard constraint can be implemented so that Furthermore, to ensure that vehicle 1 and vehicle 2 do not miss each other by a minimum amount, an additional safety margin can be obtained by including a buffer time into the constraint (e.g., 0.5 seconds or another suitable value). The hard constraints associated with the predicted crossing trajectories of the two vehicles can be expressed as

[0285] The amount of time τ within which the trajectories of the host vehicle and one or more other vehicles are tracked can vary. However, in intersection scenarios where speeds may be low, τ can be longer, and τ can be defined so that the host vehicle will enter and exit the intersection in less than τ seconds.

[0286] Of course, applying hard constraints to vehicle trajectories requires predicting the trajectories of these vehicles. Trajectory prediction can be relatively simple for the host vehicle, as the host vehicle typically already understands and is actually planning its expected trajectory at any given time. Relative to other vehicles, predicting their trajectories may be less straightforward. For other vehicles, a baseline calculation for determining the predicted trajectory can rely on the current speed and heading of the other vehicles, such as determined based on, for example, analysis of image streams captured by one or more cameras and / or other sensors (radar, lidar, acoustic sensors, etc.) on the host vehicle.

[0287] However, there may be some exceptions that may simplify the problem or at least provide additional confidence in the trajectory predicted for another vehicle. For example, for structured roads where there is an indication of lanes and where yield rules may exist, the trajectory of the other vehicle may be based at least in part on the position of the other vehicle relative to the lanes and on the applicable yield rules. Thus, in some cases, when there is observed lane structure, it may be assumed that the next-lane vehicle will respect the lane boundaries. That is, the host vehicle may assume that the next-lane vehicle will remain in its lane unless there is observed evidence (e.g., a signal light, strong lateral movement, movement across a lane boundary) that indicates that the next-lane vehicle will cut into the host vehicle's lane.

[0288] Other situations can also provide clues about the intended trajectory of other vehicles. For example, at stop signs, traffic lights, roundabouts, and so on, where the host vehicle may have the right of way, it can be assumed that other vehicles will respect this right of way. Therefore, unless there is observed evidence of rule-breaking, it can be assumed that other vehicles follow trajectories that respect the right of way held by the host vehicle.

[0289] Hard constraints can also be applied to pedestrians in the host vehicle's environment. For example, a buffer distance can be established for pedestrians, such that the host vehicle is prohibited from navigating closer than a specified buffer distance relative to any observed pedestrian. The pedestrian buffer distance can be any suitable distance. In some embodiments, the buffer distance can be at least one meter relative to the observed pedestrian.

[0290] Similar to the case of vehicles, hard constraints can also be applied to the relative motion between pedestrians and the host vehicle. For example, the trajectory of a pedestrian can be monitored relative to the projected trajectory of the host vehicle (based on heading direction and speed). Given a particular pedestrian trajectory, for each point p on the trajectory, t(p) can represent the time required for the pedestrian to reach point p. In order to maintain the required buffer distance of at least 1 meter from the pedestrian, t(p) must be greater than the time the host vehicle will reach point p (with enough time difference so that the host vehicle passes in front of the pedestrian at a distance of at least one meter) or t(p) must be less than the time the host vehicle will reach point p (for example, if the host vehicle brakes to yield to the pedestrian). In the latter example, the hard constraint can still require the host vehicle to arrive at point p enough later than the pedestrian so that the host vehicle can pass behind the pedestrian and maintain the required buffer distance of at least one meter. Of course, there can be exceptions to the hard constraints for pedestrians. For example, the hard pedestrian constraint may be relaxed (e.g., to a smaller buffer of at least 0.75 meters or 0.50 meters) where the host vehicle has the right of way or is moving very slowly and there is no observed evidence that the pedestrian will refuse to yield to the host vehicle or will otherwise walk toward the host vehicle.

[0291] In some examples, if it is determined that not all constraints can be met, the constraints can be relaxed. For example, if the road is too narrow to leave a desired spacing (e.g., 0.5 meters) between two curbs or between a curb and a parked vehicle, one or more constraints can be relaxed if there is a mitigating factor. For example, if there are no pedestrians (or other objects) on the sidewalk, you can slow down to 0.1 meters from the curb. In some embodiments, if doing so improves the user experience, the constraints can be relaxed. For example, to avoid potholes, constraints can be relaxed to allow the vehicle to navigate closer to the edge of the lane, curb, or pedestrians than is normally allowed. In addition, when determining which constraints to relax, in some embodiments, the one or more constraints selected for relaxation are those that are considered to have the least effective negative impact on safety. For example, before relaxing the constraints on proximity to other vehicles, constraints related to how close a vehicle can travel to a curb or how close a concrete barrier can be relaxed. In some embodiments, the pedestrian constraint may be the last constraint to be relaxed, or in some cases may never be relaxed.

[0292] Figure 12Examples of scenes that may be captured and analyzed during navigation of a host vehicle are shown. For example, the host vehicle may include a navigation system as described above (e.g., system 100), which may receive a plurality of images representing the host vehicle's environment from cameras associated with the host vehicle (e.g., at least one of image capture device 122, image capture device 124, and image capture device 126). Figure 12 The scene shown is an example of one of the images that may be captured at time t from the environment of a host vehicle traveling in lane 1210 along a predicted trajectory 1212. The navigation system may include at least one processing device (e.g., including an EyeQ processor or any of the other devices described above) that is specifically programmed to receive multiple images and analyze the images to determine an action in response to the scene. Specifically, the at least one processing device may implement a sensing module 801, a driving strategy module 803, and a control module 805, such as Figure 8 As shown. The sensing module 801 may be responsible for collecting and outputting image information collected from the camera, and providing this information in the form of a recognized navigation state to the driving strategy module 803. The driving strategy module may constitute a trained navigation system that has been trained through machine training techniques (such as supervised learning, reinforcement learning, etc.). Based on the navigation state information provided by the sensing module 801 to the driving strategy module 803, the driving strategy module 803 (for example, by implementing the option map method described above) may generate a desired navigation action for the host vehicle to perform in response to the recognized navigation state.

[0293] In some embodiments, at least one processing device may use, for example, a control module 805 to directly convert the desired navigation action into a navigation command. However, in other embodiments, hard constraints may be applied to test the desired navigation action provided by the driving strategy module 803 against various predetermined navigation constraints that may be involved in the scenario and the desired navigation action. For example, in the case where the driving strategy module 803 outputs a desired navigation action that will cause the host vehicle to follow trajectory 1212, this navigation action may be tested against one or more hard constraints associated with various aspects of the host vehicle's environment. For example, the captured image 1201 may reveal a curb 1213, a pedestrian 1215, a target vehicle 1217, and a stationary object (e.g., an overturned box) present in the scene. Each of these may be associated with one or more hard constraints. For example, the curb 1213 may be associated with a static constraint that prohibits the host vehicle from navigating into or past the curb and onto the sidewalk 1214. Curb 1213 can also be associated with a road barrier envelope that defines a distance away from (e.g., 0.1 m, 0.25 m, 0.5 m, 1 m, etc.) the curb and extending along the curb (e.g., a buffer zone) that defines a non-navigation zone for the host vehicle. Of course, static constraints can also be associated with other types of roadside boundaries (e.g., guardrails, concrete bollards, traffic cones, pylons, or any other type of roadside barrier).

[0294] It should be noted that distance and ranging can be determined by any suitable method. For example, in some embodiments, distance information can be provided by an onboard radar and / or lidar system. Alternatively or additionally, distance information can be derived from an analysis of one or more images captured from the host vehicle's environment. For example, the number of pixels representing the identified object in the image can be determined and compared to the known field of view and focal length geometry of the image capture device to determine scale and distance. Speed ​​and acceleration can be determined, for example, by observing the change in scale between objects between images over a known time interval. This analysis can indicate the direction of movement toward or away from the host vehicle and how quickly the object is moving away from or toward the host vehicle. Crossing speed can be determined by analyzing the change in the X-coordinate position of the object from one image to another over a known time period.

[0295] Pedestrian 1215 may be associated with a pedestrian envelope that defines a buffer zone 1216. In some cases, the imposed hard constraints may prohibit the host vehicle from navigating within a distance of 1 meter from pedestrian 1215 (in any direction relative to the pedestrian). Pedestrian 1215 may also define the location of a pedestrian influence zone 1220. This influence zone may be associated with constraints that limit the host vehicle's speed within the influence zone. The influence zone may extend 5 meters, 10 meters, 20 meters, and so on, from pedestrian 1215. Each scale of the influence zone may be associated with a different speed limit. For example, within a zone from 1 to 5 meters from pedestrian 1215, the host vehicle may be limited to a first speed (e.g., 10 mph, 20 mph, etc.), which may be less than the speed limit in the pedestrian influence zone extending from 5 meters to 10 meters. Any scale may be used for various levels of the influence zone. In some embodiments, the first level may be narrower than from 1 to 5 meters and may extend only from 1 to 2 meters. In other embodiments, a first level of the influence zone may extend from 1 meter (the boundary of the non-navigation zone around the pedestrian) to a distance of at least 10 meters. A second level, in turn, may extend from 10 meters to at least about 20 meters. The second level may be associated with a maximum travel speed of the host vehicle that is greater than the maximum travel speed associated with the first level of the pedestrian influence zone.

[0296] One or more stationary object constraints may also be associated with a detected scene in the host vehicle's environment. For example, in image 1201, at least one processing device may detect a stationary object, such as a box 1219, located in the roadway. The detected stationary object may include various objects, such as at least one of a tree, a pole, a road sign, or an object in the roadway. One or more predefined navigation constraints may be associated with the detected stationary object. For example, such constraints may include a stationary object envelope, wherein the stationary object envelope defines a buffer zone around the object within which navigation of the host vehicle may be prohibited. At least a portion of the buffer zone may extend a predetermined distance from the edge of the detected stationary object. For example, in the scene represented by image 1201, a buffer zone of at least 0.1 meters, 0.25 meters, 0.5 meters, or more may be associated with box 1219, such that the host vehicle will pass to the right or left of the box at least a certain distance (e.g., a buffer zone distance) to avoid collision with the detected stationary object.

[0297] Predefined hard constraints may also include one or more target vehicle constraints. For example, a target vehicle 1217 may be detected in image 1201. To ensure that the host vehicle does not collide with target vehicle 1217, one or more hard constraints may be employed. In some cases, the target vehicle envelope may be associated with a single buffer zone distance. For example, a buffer zone may be defined by a 1-meter distance around the target vehicle in all directions. The buffer zone may define an area extending at least one meter from the target vehicle, into which the host vehicle is prohibited from navigating.

[0298] However, the envelope around target vehicle 1217 need not be defined by a fixed buffer distance. In some cases, the predefined hard constraints associated with the target vehicle (or any other movable object detected in the host vehicle's environment) can depend on the host vehicle's orientation relative to the detected target vehicle. For example, in some cases, the longitudinal buffer distance (e.g., the distance extending from the target vehicle toward the front or rear of the host vehicle—such as when the host vehicle is driving toward the target vehicle) can be at least one meter. The lateral buffer distance (e.g., the distance extending from the target vehicle toward either side of the host vehicle—such as when the host vehicle is traveling in the same or opposite direction as the target vehicle so that one side of the host vehicle will pass adjacent to a side of the target vehicle) can be at least 0.5 meters.

[0299] As described above, other constraints may also be involved in the detection of target vehicles or pedestrians in the host vehicle's environment. For example, the predicted trajectories of the host vehicle and the target vehicle 1217 may be considered, and in the event that the two trajectories intersect (e.g., at intersection 1230), a hard constraint may require that or Where the host vehicle is vehicle 1 and the target vehicle 1217 is vehicle 2. Similarly, the trajectory of the pedestrian 1215 (based on heading direction and speed) can be monitored relative to the expected trajectory of the host vehicle. Given a particular pedestrian trajectory, for each point p on the trajectory, t(p) will represent the pedestrian's arrival at point p (i.e., Figure 12 In order to maintain the required buffer distance of at least 1 meter from the pedestrian, t(p) must be greater than the time the host vehicle will arrive at point p (with enough time difference so that the host vehicle passes in front of the pedestrian by at least one meter) or t(p) must be less than the time the host vehicle will arrive at point p (for example, if the host vehicle brakes to yield to the pedestrian). In the latter example, the hard constraint still requires the host vehicle to arrive at point p sufficiently after the pedestrian so that the host vehicle can pass behind the pedestrian and maintain the required buffer distance of at least one meter.

[0300] Other hard constraints may also be employed. For example, a maximum deceleration rate for the host vehicle may be employed in at least some circumstances. This maximum deceleration rate may be determined based on a detected distance to a target vehicle following the host vehicle (e.g., using images collected from a rear-facing camera). Hard constraints may include mandatory stops at sensed crosswalks or railroad crossings, or other applicable constraints.

[0301] In the event that analysis of the scene in the host vehicle's environment indicates that one or more predefined navigation constraints may be involved, these constraints can be imposed relative to one or more planned navigation actions for the host vehicle. For example, in the event that analysis of the scene causes the driving strategy module 803 to return a desired navigation action, this desired navigation action can be tested against one or more of the involved constraints. If the desired navigation action is determined to violate any aspect of the involved constraints (e.g., if the desired navigation action would bring the host vehicle within a distance of 0.7 meters of a pedestrian 1215, where a predefined hard constraint requires the host vehicle to maintain a distance of at least 1.0 meters from the pedestrian 1215), at least one modification can be made to the desired navigation action based on the one or more predefined navigation constraints. Adjusting the desired navigation action in this manner can provide an actual navigation action for the host vehicle that complies with the constraints involved in the particular scene detected in the host vehicle's environment.

[0302] After determining an actual navigational maneuver for the host vehicle, the navigational maneuver may be implemented by causing at least one adjustment of a navigational actuator of the host vehicle in response to the determined actual navigational maneuver for the host vehicle. Such a navigational actuator may include at least one of a steering mechanism, a brake, or an accelerator of the host vehicle.

[0303] Prioritized constraints

[0304] As described above, various hard constraints can be applied to the navigation system to ensure safe operation of the host vehicle. Constraints can include a minimum safe driving distance relative to pedestrians, target vehicles, road barriers, or detected objects, a maximum driving speed when passing within the influence zone of a detected pedestrian, or a maximum deceleration rate for the host vehicle. These constraints can be imposed on trained systems trained based on machine learning (supervised, reinforcement, or a combination), but they can also be useful for untrained systems (e.g., those that employ algorithms to directly address anticipated situations arising from scenarios in the host vehicle environment).

[0305] In either case, there can be a hierarchy of constraints. In other words, some navigation constraints can take precedence over others. Therefore, if a situation arises where a navigation action that satisfies all involved constraints is unavailable, the navigation system can determine which available navigation action to implement first, the highest-priority constraint. For example, the system can cause the vehicle to avoid a pedestrian first, even if navigating to avoid the pedestrian would result in a collision with another vehicle or object detected on the road. In another example, the system can cause the vehicle to drive onto a curb to avoid the pedestrian.

[0306] Figure 13A flowchart illustrating an algorithm for implementing a hierarchy of constraints determined based on an analysis of a scene in the host vehicle's environment is provided. For example, at step 1301, at least one processing device associated with a navigation system (e.g., an EyeQ processor, etc.) may receive a plurality of images representing the host vehicle's environment from a camera mounted on the host vehicle. By analyzing the one or more images representing the scene in the host vehicle's environment at step 1303, a navigation state associated with the host vehicle may be identified. For example, the navigation state may indicate that the host vehicle is traveling along a path such as Figure 12 12. The host vehicle is traveling on a two-lane road 1210, a target vehicle 1217 is moving through an intersection in front of the host vehicle, a pedestrian 1215 is waiting to cross the road on which the host vehicle is traveling, an object 1219 is present ahead in the host vehicle's lane, and various other attributes of the scene.

[0307] At step 1305, one or more navigation constraints associated with the navigation state of the host vehicle may be determined. For example, after analyzing a scene represented by one or more captured images in the environment of the host vehicle, at least one processing device may determine one or more navigation constraints associated with objects, vehicles, pedestrians, etc. identified through image analysis of the captured images. In some embodiments, the at least one processing device may determine at least a first predefined navigation constraint and a second predefined navigation constraint associated with the navigation state, and the first predefined navigation constraint may be different from the second predefined navigation constraint. For example, the first navigation constraint may be associated with one or more target vehicles detected in the environment of the host vehicle, and the second navigation constraint may be associated with pedestrians detected in the environment of the host vehicle.

[0308] At step 1307, at least one processing device may determine a priority associated with the constraints identified in step 1305. In the depicted example, a second predefined navigation constraint associated with a pedestrian may have a higher priority than a first predefined navigation constraint associated with a target vehicle. While the priority associated with a navigation constraint may be determined or assigned based on a variety of factors, in some embodiments, the priority of a navigation constraint may be related to its relative importance from a safety perspective. For example, while it may be important to comply with or satisfy all implemented navigation constraints in as many situations as possible, some constraints may be associated with greater safety risks than others and, therefore, may be assigned a higher priority. For example, a navigation constraint requiring the host vehicle to maintain a separation distance of at least 1 meter from a pedestrian may have a higher priority than a constraint requiring the host vehicle to maintain a separation distance of at least 1 meter from a target vehicle. This may be because a collision with a pedestrian may have more severe consequences than a collision with another vehicle. Similarly, maintaining space between the host vehicle and the target vehicle may have a higher priority than a constraint requiring the host vehicle to avoid a box in the road, drive over a speed bump at less than a certain speed, or expose the host vehicle occupants to no more than a maximum acceleration level.

[0309] While the driving strategy module 803 is designed to maximize safety by satisfying navigation constraints associated with a particular scenario or navigation state, in some cases, satisfying each of the associated constraints may not be physically possible. In such cases, the priority of each of the associated constraints may be used to determine which of the associated constraints should be satisfied first, as shown in step 1309. Continuing with the above example, in a situation where it is not possible to satisfy both the pedestrian interval constraint and the target vehicle interval constraint, but only one of the constraints can be satisfied, the higher priority of the pedestrian interval constraint may result in satisfying this constraint before attempting to maintain a gap with the target vehicle. Thus, under normal circumstances, at least one processing device may determine a first navigation action for the host vehicle that satisfies both a first predefined navigation constraint and a second predefined navigation constraint based on the identified navigation state of the host vehicle, wherein both the first predefined navigation constraint and the second predefined navigation constraint may be satisfied, as shown in step 1311. However, in other situations where all involved constraints cannot be satisfied, at least one processing device may determine, based on the identified navigation state, a second navigation action for the main vehicle that satisfies a second predefined navigation constraint (i.e., a higher priority constraint) but does not satisfy a first predefined navigation constraint (having a lower priority than the second navigation constraint), wherein both the first predefined navigation constraint and the second predefined navigation constraint cannot be satisfied, as shown in step 1313.

[0310] Next, at step 1315, to implement the determined navigation action for the host vehicle, the at least one processing device may cause at least one adjustment of a navigation actuator of the host vehicle in response to the determined first navigation action or the determined second navigation action for the host vehicle. As with the previous example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0311] Constraint relaxation

[0312] As discussed above, navigation constraints can be imposed for safety purposes. Constraints may include a minimum safe driving distance relative to a pedestrian, target vehicle, road barrier, or detected object, a maximum travel speed when passing within the influence zone of a detected pedestrian, or a maximum deceleration rate for the host vehicle. These constraints can be imposed in either learning or non-learning navigation systems. In certain circumstances, these constraints can be relaxed. For example, if the host vehicle slows down or stops near a pedestrian and then slowly advances to communicate its intention to pass by the pedestrian, the pedestrian's response can be detected from the captured imagery. If the pedestrian responds by remaining still or stopping (and / or if eye contact with the pedestrian is sensed), it can be understood that the pedestrian recognized the navigation system's intention to pass by the pedestrian. In such cases, the system can relax one or more predefined constraints and implement less stringent constraints (e.g., allowing the vehicle to navigate within 0.5 meters of the pedestrian rather than within the more stringent 1-meter boundary).

[0313] Figure 14 A flow chart for implementing control of a host vehicle based on the relaxation of one or more navigation constraints is provided. At step 1401, at least one processing device may receive a plurality of images representing an environment of the host vehicle from a camera associated with the host vehicle. At step 1403, analysis of the images may enable identification of a navigation state associated with the host vehicle. At step 1405, at least one processor may determine a navigation constraint associated with the navigation state of the host vehicle. The navigation constraint may include a first predefined navigation constraint related to at least one aspect of the navigation state. At step 1407, analysis of the plurality of images may reveal the presence of at least one navigation constraint relaxation factor.

[0314] Navigation constraint relaxation factors may include any suitable indication that one or more navigation constraints may be suspended, modified, or otherwise relaxed in at least one aspect. In some embodiments, at least one navigation constraint relaxation factor may include a determination (based on image analysis) that the pedestrian's eyes are looking in the direction of the host vehicle. In such circumstances, it may be safer to assume that the pedestrian is aware of the host vehicle. Consequently, the confidence level that the pedestrian will not make an unintended action that would cause the pedestrian to move into the path of the host vehicle may be higher. Other constraint relaxation factors may also be used. For example, at least one navigation constraint relaxation factor may include: a pedestrian determined to be non-moving (e.g., a pedestrian presumed to be unlikely to enter the path of the host vehicle); or a pedestrian whose movement is determined to be slowing down. Navigation constraint relaxation factors may also include more complex actions, such as a pedestrian determined to be non-moving after the host vehicle has stopped and then resumed movement. In such cases, it may be assumed that the pedestrian understands that the host vehicle has the right-of-way, and that the pedestrian's stopping may indicate the pedestrian's intention to yield to the host vehicle. Other situations that may result in relaxing one or more constraints include the type of curb (e.g., a low curb or a curb with a gentle slope may allow for a relaxed distance constraint), the lack of pedestrians or other objects on the sidewalk, vehicles whose engines are not running may have a relaxed distance, or situations where pedestrians are facing away from and / or moving away from the area toward which the host vehicle is heading.

[0315] In the event that the presence of a navigation constraint relaxation factor is identified (e.g., at step 1407), a second navigation constraint may be determined or developed in response to detecting the constraint relaxation factor. This second navigation constraint may be different from the first navigation constraint and may include at least one characteristic that is relaxed relative to the first navigation constraint. The second navigation constraint may include a newly generated constraint based on the first constraint, wherein the newly generated constraint includes at least one modification that relaxes the first constraint in at least one aspect. Alternatively, the second constraint may constitute a predetermined constraint that is less stringent than the first navigation constraint in at least one aspect. In some embodiments, such a second constraint may be reserved for use only in situations where a constraint relaxation factor is identified in the environment of the host vehicle. Regardless of whether the second constraint is newly generated or selected from a set of fully or partially available predetermined constraints, applying the second navigation constraint in place of the more stringent first navigation constraint (which may be applied in the absence of a detection of a related navigation constraint relaxation factor) may be referred to as constraint relaxation and may be completed in step 1409.

[0316] If at least one constraint relaxation factor is detected at step 1407 and at least one constraint has been relaxed at step 1409, a navigation action for the host vehicle may be determined at step 1411. The navigation action for the host vehicle may be based on the identified navigation state and may satisfy the second navigation constraint. The navigation action may be implemented at step 1413 by causing at least one adjustment to a navigation actuator of the host vehicle in response to the determined navigation action.

[0317] As discussed above, the use of navigation constraints and relaxed navigation constraints can be employed with either trained (e.g., through machine learning) or untrained navigation systems (e.g., systems programmed to respond with predetermined actions in response to specific navigation conditions). In the case of using a trained navigation system, the availability of relaxed navigation constraints for certain navigation situations can represent a mode switch from trained system responses to untrained system responses. For example, a trained navigation network can determine an original navigation action for a host vehicle based on a first navigation constraint. However, the action taken by the vehicle can be an action different from the navigation action that satisfies the first navigation constraint. Instead, the action taken can satisfy a second, more relaxed navigation constraint and can be an action developed by an untrained system (e.g., as a response to detecting a specific condition in the host vehicle's environment, such as the presence of a navigation constraint relaxation factor).

[0318] There are many examples of navigation constraints that can be relaxed in response to a constraint relaxation factor being detected in the host vehicle's environment. For example, where a predefined navigation constraint includes a buffer zone associated with a detected pedestrian, and at least a portion of the buffer zone extends a distance from the detected pedestrian, the relaxed navigation constraint (newly developed, recalled from memory from a predetermined set, or generated as a relaxation of a pre-existing constraint) can include a different or modified buffer zone. For example, the different or modified buffer zone can have a distance relative to the pedestrian that is less than the distance of the original or unmodified buffer zone relative to the detected pedestrian. Thus, in light of the relaxed constraint, the host vehicle can be permitted to navigate closer to the detected pedestrian, where an appropriate constraint relaxation factor is detected in the host vehicle's environment.

[0319] The relaxed characteristics of the navigation constraints may include a reduced width of the buffer associated with at least one pedestrian, as noted above. However, the relaxed characteristics may also include a reduced width of the buffer associated with a target vehicle, a detected object, a roadside barrier, or any other object detected in the host vehicle's environment.

[0320] At least one relaxed characteristic may also include other types of modifications to the navigation constraint characteristic. For example, the relaxed characteristic may include an increase in speed associated with at least one predefined navigation constraint. The relaxed characteristic may also include an increase in the maximum allowable deceleration / acceleration associated with at least one predefined navigation constraint.

[0321] Although as described above, constraints can be relaxed in some cases, in other cases, navigation constraints can be enhanced. For example, in some cases, the navigation system can determine that conditions warrant enhancing a normal set of navigation constraints. Such enhancements can include adding new constraints to a predefined set of constraints or adjusting one or more aspects of the predefined constraints. The additions or adjustments can result in more conservative navigation relative to the predefined set of constraints that apply under normal driving conditions. Conditions that may warrant constraint enhancement can include sensor failures, adverse environmental conditions (rain, snow, fog, or other conditions associated with reduced visibility or reduced vehicle traction), etc.

[0322] Figure 15 A flow chart for implementing control of a host vehicle based on augmentation of one or more navigation constraints is provided. At step 1501, at least one processing device may receive a plurality of images representing an environment of the host vehicle from a camera associated with the host vehicle. At step 1503, analysis of the images may enable identification of a navigation state associated with the host vehicle. At step 1505, at least one processor may determine a navigation constraint associated with the navigation state of the host vehicle. The navigation constraint may include a first predefined navigation constraint related to at least one aspect of the navigation state. At step 1507, analysis of the plurality of images may reveal the presence of at least one navigation constraint augmenting factor.

[0323] The navigation constraints involved may include the above (e.g., relative to Figure 12) or any other suitable navigation constraints. A navigation constraint enhancement factor may include any indication that one or more navigation constraints may be supplemented / enhanced in at least one aspect. Supplementation or enhancement of navigation constraints may be performed on a per-set basis (e.g., by adding a new navigation constraint to a predetermined set of constraints) or on a per-constraint basis (e.g., by modifying a particular constraint so that the modified constraint is more restrictive than the original constraint, or by adding a new constraint corresponding to a predetermined constraint, where the new constraint is more restrictive in at least one aspect than the corresponding constraint). Additionally or alternatively, supplementation or enhancement of navigation constraints may refer to selection from a set of predetermined constraints based on a hierarchical structure. For example, a set of enhanced constraints may be available for selection based on whether a navigation enhancement factor is detected in or relative to the host vehicle's environment. Under normal conditions where no enhancement factor is detected, the navigation constraints involved may be drawn from the constraints applicable under normal conditions. On the other hand, if one or more constraint enhancement factors are detected, the constraints involved may be drawn from enhanced constraints generated or predefined relative to the one or more enhancement factors. The enhanced constraints may be more restrictive in at least one aspect than the corresponding constraints applicable under normal conditions.

[0324] In some embodiments, at least one navigation constraint enhancing factor may include detecting (e.g., based on image analysis) the presence of ice, snow, or water on the surface of a road in the host vehicle's environment. This determination may be based on, for example, detecting: areas of reflectivity higher than expected for a dry roadway (e.g., indicating ice or water on the roadway); white areas on the roadway indicating the presence of snow; shadows on the roadway consistent with the presence of longitudinal grooves on the roadway (e.g., tire tracks in the snow); water droplets or ice / snow particles on the host vehicle's windshield; or any other suitable indication of the presence of water or ice / snow on the surface of the roadway.

[0325] At least one navigation constraint enhancing factor may also include detection of particles on an exterior surface of a windshield of the host vehicle. Such particles may impair image quality of one or more image capture devices associated with the host vehicle. While described with respect to the windshield of the host vehicle (as associated with a camera mounted behind the windshield of the host vehicle), detection of particles on other surfaces (e.g., a camera lens or lens cover, a headlight lens, a rear windshield, a taillight lens, or any other surface of the host vehicle that is visible to (or detected by a sensor of) an image capture device associated with the host vehicle) may also indicate the presence of a navigation constraint enhancing factor.

[0326] A navigation constraint enhancing factor may also be detected as a property of one or more image acquisition devices. For example, a detected degradation in image quality of one or more images captured by an image capture device (e.g., a camera) associated with the host vehicle may also constitute a navigation constraint enhancing factor. Image quality degradation may be associated with a hardware failure or partial hardware failure associated with the image capture device or a component associated with the image capture device. Such image quality degradation may also be caused by environmental conditions. For example, the presence of smoke, fog, rain, snow, etc. in the air surrounding the host vehicle may also result in reduced image quality of roads, pedestrians, target vehicles, etc. that may be present in the environment relative to the host vehicle.

[0327] Navigation constraint enhancing factors may also relate to other aspects of the host vehicle. For example, in some cases, a navigation constraint enhancing factor may include a detected failure or partial failure of a system or sensor associated with the host vehicle. Such enhancing factors may include, for example, a detected failure or partial failure of a speed sensor, a GPS receiver, an accelerometer, a camera, a radar, a lidar, brakes, tires, or any other system associated with the host vehicle that may affect the host vehicle's ability to navigate relative to the navigation constraints associated with the host vehicle's navigational state.

[0328] In the event that the presence of a navigation constraint enhancing factor is identified (e.g., at step 1507), a second navigation constraint may be determined or developed in response to the detection of the constraint enhancing factor. This second navigation constraint may be different from the first navigation constraint and may include at least one characteristic that is enhanced relative to the first navigation constraint. The second navigation constraint may be more restrictive than the first navigation constraint because the detection of a constraint enhancing factor in the host vehicle's environment or associated with the host vehicle may indicate that the host vehicle may have at least one navigation capability that is reduced relative to normal operating conditions. Such reduced capability may include: reduced road traction (e.g., ice, snow, or water on the roadway; reduced tire pressure; etc.); impaired vision (e.g., rain, snow, dust, smoke, fog, etc. that may reduce the quality of captured images); impaired detection capability (e.g., sensor failure or partial failure, reduced sensor performance, etc.); or any other reduction in the host vehicle's ability to navigate in response to the detected navigation condition.

[0329] If at least one constraint enhancing factor is detected at step 1507 and at least one constraint has been enhanced at step 1509, a navigation action for the host vehicle may be determined at step 1511. The navigation action for the host vehicle may be based on the identified navigation state and may satisfy the second navigation (i.e., enhanced) constraint. The navigation action may be implemented at step 1513 by causing at least one adjustment to a navigation actuator of the host vehicle in response to the determined navigation action.

[0330] As discussed, the use of navigation constraints and enhanced navigation constraints can be employed with either trained (e.g., through machine learning) or untrained navigation systems (e.g., systems programmed to respond with predetermined actions in response to specific navigation conditions). In the case of using a trained navigation system, the availability of enhanced navigation constraints for certain navigation situations can represent a mode switch from trained system responses to untrained system responses. For example, a trained navigation network can determine an original navigation action for a host vehicle based on a first navigation constraint. However, the action taken by the vehicle can be an action different from the navigation action that satisfies the first navigation constraint. Instead, the action taken can satisfy an enhanced second navigation constraint and can be an action developed by an untrained system (e.g., as a response to detecting a specific condition in the host vehicle's environment, such as the presence of a factor that enhances the navigation constraint).

[0331] There are many examples of navigation constraints that can be generated, supplemented, or enhanced in response to the detection of a constraint-enhancing factor in the host vehicle's environment. For example, where a predefined navigation constraint includes a buffer zone associated with a detected pedestrian, object, vehicle, etc., and at least a portion of the buffer zone extends a certain distance from the detected pedestrian / object / vehicle, an enhanced navigation constraint (newly developed, recalled from memory from a predetermined set, or generated as an enhancement of a pre-existing constraint) can include a different or modified buffer zone. For example, the different or modified buffer zone can have a distance relative to the pedestrian / object / vehicle that is greater than the distance of the original or unmodified buffer zone relative to the detected pedestrian / object / vehicle. Thus, in light of the enhanced constraint, the host vehicle can be forced to navigate further away from the detected pedestrian / object / vehicle, where appropriate constraint-enhancing factors are detected in or relative to the host vehicle's environment.

[0332] The at least one enhanced characteristic may also include other types of modifications to the navigation constraint characteristic. For example, the enhanced characteristic may include a speed reduction associated with at least one predefined navigation constraint. The enhanced characteristic may also include a reduction in the maximum allowable deceleration / acceleration associated with at least one predefined navigation constraint.

[0333] Navigation based on long-range planning

[0334] In some embodiments, the disclosed navigation system may not only respond to a detected navigation state in the host vehicle's environment but may also determine one or more navigation actions based on long-range planning. For example, the system may consider the potential impact of one or more navigation actions available as options for navigating relative to the detected navigation state on the future navigation state. Considering the impact of available actions on the future state enables the navigation system to determine navigation actions based not only on the current detected navigation state but also on long-range planning. Navigation using long-range planning techniques may be particularly suitable for situations where the navigation system employs one or more reward functions as a technique for selecting navigation actions from available options. Potential rewards may be analyzed relative to available navigation actions that can be taken in response to the host vehicle's detected current navigation state. However, potential rewards may also be analyzed relative to actions that can be taken in response to future navigation states predicted to result from available actions in response to the current navigation state. Thus, in some situations, the disclosed navigation system may select a navigation action in response to a detected navigation state, even if the selected navigation action may not yield the highest reward among the available actions that can be taken in response to the current navigation state. This is particularly true when the system determines that the selected action may result in a future navigation state that produces one or more potential navigation actions that provide a higher reward than the selected action or, in some cases, any action available relative to the current navigation state. The principle can be more simply expressed as taking a less favorable action now in order to produce a higher reward option in the future. Therefore, the disclosed navigation system capable of long-term planning can select a suboptimal short-term action where the long-term prediction indicates that a short-term reward loss may result in a long-term reward gain.

[0335] Generally speaking, autonomous driving applications can involve a series of planning problems, in which the navigation system can make decisions on immediate actions in order to optimize long-term goals. For example, when a vehicle faces a merging situation at a roundabout, the navigation system can make decisions on immediate acceleration or braking commands in order to initiate navigation into the roundabout. Although the immediate action to the detected navigation state at the roundabout may involve acceleration or braking commands in response to the detected state, the long-term goal is to successfully merge, and the long-term effect of the selected command is the success / failure of the merge. The planning problem can be solved by breaking the problem into two stages. First, supervised learning can be applied to predict the near future based on the present (assuming that the predictor will be differentiable with respect to the present representation). Second, a recursive neural network can be used to model the complete trajectory of the agent, where unexplained factors are modeled as (additional) input nodes. This can allow the use of supervised learning techniques and direct optimization of the recursive neural network to determine the solution to the long-term planning problem. This approach can also enable the learning of robust policies by incorporating adversarial elements into the environment.

[0336] Two of the most fundamental elements of autonomous driving systems are sensing and planning. Sensing involves finding a compact representation of the current state of the environment, while planning involves deciding which actions to take to optimize future goals. Supervised machine learning techniques are useful for solving sensing problems. Machine learning algorithmic frameworks can also be used for planning, particularly reinforcement learning (RL) frameworks such as those described above.

[0337] RL can be performed in a series of consecutive rounds. At round t, the planner (also known as the agent or driving policy module 803) can observe the state s t ∈S, the state represents the agent and the environment. The planner should then decide on an action a t ∈A. After taking this action, the agent receives an immediate reward and go to the new state s t+1 As an example, the host vehicle may include an adaptive cruise control (ACC) system, where the vehicle should autonomously implement acceleration / braking in order to maintain a sufficient distance from the vehicle ahead while maintaining a smooth ride. The state can be modeled as where x t is the distance to the vehicle ahead, and v t is the speed of the host vehicle relative to the speed of the vehicle ahead. will be the acceleration command (if a t <0 if the host vehicle slows down). The reward can be determined by |a t |(Reflects the smoothness of driving) and depends on s t (reflecting the host vehicle maintaining a safe distance from the vehicle in front). The planner aims to maximize the cumulative reward (which may be determined by the time horizon or the discounted sum of future rewards). To do this, the planner can rely on a policy π:S→A, which maps states to actions.

[0338] Supervised learning (SL) can be viewed as a special case of RL, where s t is sampled from some distribution on S, and the reward function can have the form r t =-l(a t ,y t ), where l is the loss function and the learner observes y t The value of the view state s t There may be several differences between general RL models and the special case of SL, and these differences may make the general RL problem more challenging.

[0339] In some SL situations, the actions (or predictions) taken by the learner may have no effect on the environment. In other words, st+1 and a t are independent. This may have two important implications. First, in SL, samples (s1, y1),…, (s m ,y m ), and only then can we start searching for a policy (or predictor) that will have good accuracy with respect to the sample. In contrast, in RL, the state s t+1 Typically depends on the action taken (and also on the previous state), which in turn depends on the policy used to generate the action. This connects the data generation process with the policy learning process. Second, since actions do not affect the environment in SL, the effect of a t The choice of a contributes locally to the performance of π. Specifically, a t Only affects the value of the immediate reward. In contrast, in RL, the action taken at episode t may have long-term effects on the value of rewards in future episodes.

[0340] In SL, knowledge of the “correct” answer t And the shape of the reward t =-l(a t ,y t ) can provide a t Full knowledge of the rewards of all possible choices of , which can be achieved by calculating the reward relative to a t In contrast, in RL, the "one-time" value of the reward can be the full value that can be observed for a particular choice of action taken. This can be called "bandit" feedback. This is one of the most important reasons why "exploration" is needed as part of long-term navigation planning, because in RL-based systems, if only "bandit" feedback is available, the system may not always understand whether the action taken is the best action to take.

[0341] Many RL algorithms rely, at least in part, on the mathematically elegant model of the Markov decision process (MDP). The Markov assumption is that, given s t and a t In the case of t+1 The distribution of is completely deterministic. This yields a closed-form expression for the cumulative reward of a given policy in terms of a stable distribution over the states of the MDP. The stable distribution of a policy can be expressed as a solution to a linear programming problem. This gives rise to two families of algorithms: 1) optimization on the primal problem, which can be called policy search, and 2) optimization on the dual problem, whose variable is called the value function V. π If the MDP starts from an initial state s and the actions are chosen from there on according to π, then the value function determines the expected cumulative reward. The relevant number is the state-action-value function Q π(s, a), which determines the cumulative reward assuming starting from state s, taking action a at that moment, and the actions chosen from that point on according to π. The Q-function can produce a characterization of the optimal policy (using the Bellman equation). In particular, the Q-function can show that the optimal policy is a deterministic function from S to A (in fact, the optimal policy can be characterized as a "greedy" policy with respect to the optimal Q-function).

[0342] One potential advantage of the MDP model is that it allows the future to be coupled to the present using Q functions. For example, given that the host vehicle is now in state s, Q π The value of (s,a) can indicate the impact of taking action a on the future. Therefore, the Q function can provide a local measure of the quality of action a, making the RL problem more similar to the SL scenario.

[0343] Many RL algorithms approximate the V-function or the Q-function in one way or another. Value iteration algorithms, such as Q-learning, can rely on the fact that the V-function and Q-function of the optimal policy can be some operators derived from the Bellman equation. Actor-critic policy iteration algorithms aim to learn a policy in an iterative manner, where at iteration t, the "critic" estimates Q πt , and based on this estimate, the “actor” improves the policy.

[0344] While MDPs are mathematically elegant and switching to a Q-function representation is convenient, this approach can have several limitations. For example, in some cases, an approximate notion of a Markovian behavior state may be all that can be discovered. Furthermore, transitions between states may depend not only on the agent’s actions, but also on the actions of other actors in the environment. For example, in the ACC example mentioned above, while the dynamics of the autonomous vehicle may be Markovian, the next state may depend on the behavior of the driver of another car, which is not necessarily Markovian. One possible solution to this problem is to use a partially observed MDP, where a Markovian state is assumed to exist, but observations based on the hidden state distribution are what can be seen.

[0345] A more straightforward approach might consider game-theoretic generalizations of MDPs (e.g., the stochastic game framework). Indeed, algorithms for MDPs can be generalized to multi-agent games (e.g., minimax-Q learning or Nash-Q learning). Other approaches might include explicit modeling of other players and regret-vanishing learning algorithms. Learning in a multi-agent setting can be more complex than in a single-agent setting.

[0346] A second limitation of the Q-function representation may arise from a departure from the tabular setting. The tabular setting is when the number of states and actions is small, and therefore, Q can be expressed as a table with |S| rows and |A| columns. However, if the natural representation of S and A includes Euclidean space, and the state space and action space are discretized, the number of states / actions can be exponential in dimension. In such cases, it may not be practical to adopt a tabular setting. Instead, the Q-function can be approximated by a function from a class of parameter assumptions (e.g., a neural network of a certain architecture). For example, a deep Q-network (DQN) learning algorithm can be used. In DQN, the state space can be continuous, but the action space may still be a small discrete set. There may be methods for dealing with continuous action spaces, but these methods may rely on approximating the Q-function. In any case, the Q-function may be complex and sensitive to noise, and therefore may be challenging to learn.

[0347] A different approach could be to use recurrent neural networks (RNNs) to solve RL problems. In some cases, RNNs can be combined with concepts from game theory about multi-agent games and robustness to adversarial environments. Furthermore, this approach can be one that does not explicitly rely on any Markov assumptions.

[0348] The method for navigating by prediction-based planning is described in more detail below. In this method, it can be assumed that the state space S is A subset of , and the action space A is In many applications, this can be a natural representation. As noted above, there may be two key differences between RL and SL: (1) because past actions affect future rewards, information from the future may need to be backpropagated into the past; and (2) the “bandit” nature of rewards may obscure the correlation between (state, action) and reward, which may complicate the learning process.

[0349] As a first step in this approach, the following observation can be made: there are interesting problems where the bandit nature of the reward is not a problem. For example, the reward value for ACC applications (as discussed in more detail below) can be differentiable with respect to the current state and action. In fact, even if the reward is given in a "bandit" fashion, learning a differentiable function Make The problem can also be a relatively simple SL problem (e.g., a one-dimensional regression problem). Therefore, the first step of the method can be to define the reward as a function The function is differentiable with respect to s and a, or a regression learning algorithm is used to learn a differentiable function. This differentiable function transforms the instance vector into and the target scalar is r tIn some cases, an exploration element can be used to create a training set.

[0350] To solve the relationship between the past and the future, similar ideas can be used. For example, if we can learn differentiable functions Make Learning such a function can be formulated as a SL problem. can be viewed as a predictor for the near future. Next, we can use the parameter function π θ :S→A to describe the strategy of mapping from S to A. θ Representing it as a neural network can be achieved using a recurrent neural network (RNN) to represent the rounds of running the agent for T rounds, where the next state is defined as here, Can be defined by the environment and can express unpredictable aspects of the near future. t+1 Depends on s in a differentiable way t and a t The fact that can realize the connection between future reward value and past actions. The parameter vector π of the policy function θ It can be learned by back propagation on the resulting RNN. Note that there is no need to t Explicit probabilistic assumptions are imposed. In particular, no requirement for a Markov relation is required. Instead, recurrent networks can be relied upon to propagate “enough” information between the past and the future. Intuitively, can describe the predictable part of the near future, while v t can express unpredictable aspects that may arise due to the behavior of other actors in the environment. The learning system should learn policies that will be robust to the behavior of other actors. If ||v t If || is large, the relationship between past actions and future reward values ​​may be too noisy for learning meaningful policies. Explicitly expressing the dynamics of the system in a transparent way can make it easier to incorporate prior knowledge. For example, prior knowledge can simplify the definition of problem.

[0351] As discussed above, the learning system may benefit from robustness against an adversarial environment (such as the host vehicle's environment), which may include multiple other drivers who may behave in unexpected ways. t In the model that imposes the probability assumption, we can consider the case where v is selected in an adversarial manner. t In some cases, μ t Impose constraints, otherwise the adversary can make the planning problem difficult or even impossible. A natural constraint might be to require ||μt ||Bounded by a constant.

[0352] Robustness against adversarial environments can be useful in autonomous driving applications. t It can even speed up the learning process because it can focus the learning system on a robust optimal strategy. This idea can be illustrated using a simple game. The state is Action And the instantaneous loss function is 0.1|a t |+[|s t |-2]+, where [x] + = Maximum {x, 0} is the ReLU (rectified linear unit) function. The next state is s t+1 =s t +a t +v t , where v is selected against the environment in an adversarial manner t ∈[-0.5,0.5]. Here, the optimal strategy can be written as a two-layer network with ReLU: t =-[s t -1.5] + +[-s t -1.5] + . Observe that when |s t |∈(1.5,2], the best action has a larger immediate loss than action a=0. Therefore, the system can plan for the future and can not only rely on the immediate loss. Observe that the loss is relative to a t The derivative is 0.1sign(a t ), and relative to s t The derivative of is 1[|s t |>2]sign(s t ). t ∈(1.5,2], v t The adversarial choice will be to set v t = 0.5, and therefore, whenever a t >1.5-s t , there may be non-zero loss in round t+1. In such cases, the derivative of the loss can be directly back-propagated to a t Therefore, in a t In the case where the choice of is suboptimal, v t Adversarial selection of can help the navigation system obtain non-zero backpropagation messages. This relationship can help the navigation system choose the current action based on the expectation that this current action (even if it results in a suboptimal reward or even a loss) will provide opportunities to obtain more optimal actions with higher rewards in the future.

[0353] This approach can be applied to virtually any navigation situation that may arise. Below is a description of the approach applied to one example: Adaptive Cruise Control (ACC). In the ACC problem, the host vehicle may attempt to maintain a sufficient distance from a target vehicle ahead (e.g., 1.5 seconds from the target car). Another goal may be to drive as smoothly as possible while maintaining the desired separation. A model representing this situation can be defined as follows. The state space is And the action space is The first coordinate of the state is the speed of the target car, the second coordinate is the speed of the host vehicle, and the last coordinate is the distance between the host vehicle and the target vehicle (e.g., the position of the host vehicle minus the position of the target along the bend in the road). The action to be taken by the host vehicle is to accelerate, and can be recorded as a t .number Can refer to the time difference between consecutive rounds. can be set to any suitable number, but in one example, Can be 0.1 seconds. Position s t It can be recorded as And the (unknown) acceleration of the target vehicle can be written as

[0354] The complete dynamics of the system can be described as:

[0355]

[0356] This can be described as the sum of two vectors:

[0357]

[0358] The first vector is the predictable part, and the second vector is the unpredictable part. The reward for round t is defined as follows: in

[0359] The first term may result in a penalty for non-zero acceleration, thus encouraging smooth driving. The second term depends on the distance x to the target car. t Distance from expectations The policy is defined as the maximum between a distance of 1 meter and a braking distance of 1.5 seconds. In some cases, this ratio can be exactly 1, but as long as the ratio is within [0.7, 1.3], the policy can forgo any penalty, which can allow the host vehicle some slack in navigation—a property that can be important in achieving smooth driving.

[0360] Implementing the methods outlined above, the navigation system of the host vehicle (e.g., through operation of the driving strategy module 803 within the processing unit 110 of the navigation system) can select an action in response to the observed state. The selected action can be based not only on an analysis of rewards associated with available responsive actions relative to the sensed navigation state, but also on consideration and analysis of future states, potential actions in response to the future states, and rewards associated with the potential actions.

[0361] Figure 16 An algorithmic approach to navigation based on detection and long-range planning is presented. For example, at step 1601, at least one processing device 110 of a navigation system of a host vehicle may receive a plurality of images. These images may capture a scene representing the host vehicle's environment and may be provided by any of the image capture devices described above (e.g., cameras, sensors, etc.). Analysis of one or more of these images at step 1603 may enable the at least one processing device 110 to identify a current navigation state associated with the host vehicle (as described above).

[0362] Various potential navigation actions responsive to the sensed navigation state may be determined at steps 1605, 1607, and 1609. These potential navigation actions (e.g., a first navigation action through an Nth available navigation action) may be determined based on the sensed state of the navigation system and a long-range objective (e.g., completing a merge, smoothly following a lead vehicle, overtaking a target vehicle, avoiding an object in the roadway, slowing down for a detected stop sign, avoiding an oncoming target vehicle, or any other navigation action that may advance the navigation objective of the system).

[0363] For each of the potential navigation actions determined, the system can determine an expected reward. The expected reward can be determined according to any of the techniques described above and can include an analysis of the specific potential action relative to one or more reward functions. Expected rewards 1606, 1608, and 1610 can be determined for each of the potential navigation actions (e.g., the first, second, and Nth) determined in steps 1605, 1607, and 1609, respectively.

[0364] In some cases, the host vehicle's navigation system may select from among the available potential actions based on the values ​​(or any other type of indication of expected reward) associated with the expected rewards 1606, 1608, and 1610. For example, in some cases, the action that yields the highest expected reward may be selected.

[0365] In other situations, particularly where the navigation system performs long-range planning to determine navigation actions for the host vehicle, the system may not select the potential action that yields the highest expected reward. Instead, the system may look into the future to analyze whether there is an opportunity to achieve a higher reward later if a lower reward action is selected in response to the current navigation state. For example, a future state may be determined for any or all of the potential actions determined at steps 1605, 1607, and 1609. Each future state determined at steps 1613, 1615, and 1617 may represent a future navigation state that is expected to result based on the current navigation state as modified by the corresponding potential action (e.g., the potential action determined at steps 1605, 1607, and 1609).

[0366] For each of the future states predicted at steps 1613, 1615, and 1617, one or more future actions (as navigation options available in response to the determined future state) may be determined and evaluated. At steps 1619, 1621, and 1623, for example, a value or any other type of indication of an expected reward associated with one or more of the future actions may be developed (e.g., based on one or more reward functions). The expected reward associated with one or more future actions may be evaluated by comparing the value of the reward function associated with each future action or by comparing any other indication associated with the expected reward.

[0367] At step 1625, the navigation system of the host vehicle may select a navigation action for the host vehicle based on a comparison of expected rewards, not just based on the potential actions identified relative to the current navigation state (e.g., at steps 1605, 1607, and 1609), but also based on expected rewards determined due to potential future actions available in response to the predicted future state (e.g., determined at steps 1613, 1615, and 1617). The selection at step 1625 may be based on the option and reward analysis performed at steps 1619, 1621, and 1623.

[0368] The selection of a navigation action at step 1625 may be based solely on a comparison of the expected rewards associated with future action options. In this case, the navigation system may select an action for the current state based solely on a comparison of the expected rewards resulting from actions for potential future navigation states. For example, the system may select the potential action identified at steps 1605, 1607, or 1609 that is associated with the highest future reward value as determined by the analysis at steps 1619, 1621, and 1623.

[0369] The selection of a navigation action at step 1625 may also be based solely on a comparison of current action options (as noted above). In this case, the navigation system may select the potential action associated with the highest expected reward 1606, 1608, or 1610 identified at steps 1605, 1607, or 1609. This selection may be made with little or no consideration of future navigation states or future expected rewards for navigation actions available in response to the expected future navigation states.

[0370] On the other hand, in some cases, the selection of navigation action at step 1625 can be based on a comparison of the expected rewards associated with both the future action options and the current action options. In fact, this can be one of the navigation principles based on long-term planning. For example, the expected rewards for future actions can be analyzed to determine whether there is anyone who can guarantee to select a lower reward action in response to the current navigation state, so as to reach a potential higher reward in response to subsequent navigation actions, which are expected to be available in response to the future navigation state. As an example, the value or other indications of expected reward 1606 can indicate the highest expected reward among rewards 1606, 1608 and 1610. On the other hand, expected reward 1608 can indicate the lowest expected reward among rewards 1606, 1608 and 1610. Rather than simply selecting the potential action determined in step 1605 (i.e., the action that produces the highest expected reward 1606), the analysis of future state, potential future action and future reward can be used to select navigation action at step 1625. In one example, it may be determined that the reward identified at step 1621 (in response to at least one future action for a future state determined at step 1615 based on the second potential action determined at step 1607) may be higher than the expected reward 1606. Based on this comparison, the second potential action determined at step 1607 may be selected over the first potential action determined at step 1605, even though the expected reward 1606 is higher than the expected reward 1608. In one example, the potential navigation action determined at step 1605 may include merging in front of a detected target vehicle, while the potential navigation action determined at step 1607 may include merging behind the target vehicle. While the expected reward 1606 for merging in front of the target vehicle may be higher than the expected reward 1608 associated with merging behind the target vehicle, it may be determined that merging behind the target vehicle may result in a future state for which an action option may exist that yields an even higher potential reward than the expected rewards 1606, 1608, or other rewards based on actions available in response to the current sensed navigation state.

[0371] The selection from among the potential actions at step 1625 can be based on any suitable comparison of expected rewards (or any other measure or indication of the benefit associated with one potential action over another potential action). In some cases, as described above, if the second potential action is expected to provide at least one future action associated with an expected reward that is higher than the reward associated with the first potential action, then the second potential action can be selected instead of the first potential action. In other cases, more complex comparisons can be employed. For example, a reward associated with an action option responsive to a projected future state can be compared to more than one expected reward associated with a determined potential action.

[0372] In some scenarios, if at least one of the expected future actions is expected to produce a reward that is higher than any reward expected due to potential actions for the current state (e.g., expected rewards 1606, 1608, 1610, etc.), then the action based on the expected future state and the expected reward may influence the selection of the potential action for the current state. In some cases, the future action option that produces the highest expected reward (e.g., from among the expected rewards associated with potential actions for the sensed current state and from among the expected rewards associated with potential future action options relative to potential future navigation states) can be used as a guide for selecting the potential action for the current navigation state. That is, after identifying the future action option that produces the highest expected reward (or a reward above a predetermined threshold, etc.), the potential action that will result in the future state associated with the identified future action that produces the highest expected reward can be selected at step 1625.

[0373] In other cases, the selection of available actions may be based on the determined difference between the expected rewards. For example, if the difference between the expected reward associated with the future action determined at step 1621 and the expected reward 1606 is greater than the difference between the expected reward 1608 and the expected reward 1606 (assuming the sign of the difference), then the second potential action determined at step 1607 may be selected. In another example, if the difference between the expected reward associated with the future action determined at step 1621 and the expected reward associated with the future action determined at step 1619 is greater than the difference between the expected reward 1608 and the expected reward 1606, then the second potential action determined at step 1607 may be selected.

[0374] Several examples have been described for selecting from among potential actions for the current navigation state. However, any other suitable comparison technique or criterion may be used to select available actions through long-range planning based on action and reward analysis extending to projected future states. Figure 16While two layers of long-range planning analysis are shown (e.g., a first layer that considers rewards resulting from potential actions for the current state, and a second layer that considers rewards resulting from future action options in response to projected future states), analysis based on more layers may also be possible. For example, rather than basing long-range planning analysis on one or two layers, three, four, or more layers of analysis may be used to select from among available potential actions in response to the current navigation state.

[0375] After selecting from among the potential actions responsive to the sensed navigation state, at step 1627, at least one processor may cause at least one adjustment of a navigation actuator of the host vehicle responsive to the selected potential navigation action. The navigation actuator may include any suitable device for controlling at least one aspect of the host vehicle. For example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0376] Navigation based on the inferred aggressiveness of others

[0377] The target vehicle can be monitored by analyzing the captured image stream to determine an indication of driving aggressiveness. Aggressiveness is described herein as a qualitative or quantitative parameter, but other characteristics may also be used, such as the perceived level of attention (potential damage to the driver, distraction—cell phone, falling asleep, etc.). In some cases, the target vehicle may be considered to have a defensive posture, while in other cases, the target vehicle may be determined to have a more aggressive posture. Navigation actions can be selected or developed based on the aggressiveness indication. For example, in some cases, the relative speed, relative acceleration, increase in relative acceleration, following distance, etc. relative to the host vehicle can be tracked to determine whether the target vehicle is aggressive or defensive. For example, if the target vehicle is determined to have an aggressiveness level exceeding a threshold, the host vehicle may be inclined to yield to the target vehicle. The target vehicle's aggressiveness level can also be discerned based on the target vehicle's determined behavior relative to one or more obstacles in the target vehicle's path or in the vicinity (e.g., a leading vehicle, an obstacle in the road, a traffic light, etc.).

[0378] As an introduction to this concept, an example experiment will be described regarding a host vehicle merging into a roundabout, where the navigation objective is to navigate through and exit the roundabout. The situation may begin with the host vehicle approaching the entrance to the roundabout and may end with the host vehicle reaching the exit (e.g., the second exit) of the roundabout. Success may be measured based on whether the host vehicle maintains a safe distance from all other vehicles at all times, whether the host vehicle completes the route as quickly as possible, and whether the host vehicle follows a smooth acceleration strategy. In this illustration, N TTarget vehicles are randomly placed on the roundabout. To model a mixture of adversarial and typical behaviors, with probability p, the target vehicle can be modeled with an "aggressive" driving strategy, causing it to accelerate when the host vehicle attempts to merge in front of the aggressive target vehicle. With probability 1-p, the target vehicle can be modeled with a "defensive" driving strategy, causing it to slow down and allow the host vehicle to merge. In this experiment, p = 0.5, and the host vehicle's navigation system may not be provided with information about the other driver's type. The other driver's type can be randomly selected at the beginning of the round.

[0379] The navigation state can be represented as the velocity and position of the host vehicle (agent), and the position, velocity, and acceleration of the target vehicles. In order to distinguish between aggressive and defensive drivers based on the current state, it may be important to maintain an observation of the target acceleration. All target vehicles can move on a one-dimensional curve that outlines the path of the roundabout. The host vehicle can move on its own one-dimensional curve that intersects the target vehicle's curve at the merge point, and this point is the origin of the two curves. In order to model reasonable driving, the absolute value of the acceleration of all vehicles can be upper bounded by a constant. Velocity can also be passed through ReLU because backward driving is not allowed. Note that by not allowing backward driving, long-term planning can become essential because the agent will not have regrets about its past actions.

[0380] As described above, the next state s t+1 Can be broken down into predictable parts and the unpredictable part v t The sum of . The expression can represent the dynamics of the vehicle position and velocity (which can be well-defined in a differentiable way), while v t It can represent the acceleration of the target vehicle. It can be verified that It can be expressed as a combination of ReLU functions on an affine transformation, so it is relative to s t and a t is differentiable. The vector v t It can be defined by a simulator in a non-differentiable way and can perform aggressive actions on some targets and defensive actions on other targets. Two frameworks of such simulators are Figure 17A and 17B In this example experiment, the host vehicle 1701 learns to slow down as it approaches the entrance to the roundabout. The host vehicle also learns to yield to aggressive vehicles (e.g., vehicles 1703 and 1705) and to continue safely when merging in front of defensive vehicles (e.g., vehicles 1706, 1708, and 1710). Figure 17A and 17BIn the example shown, the type of target vehicle is not provided to the navigation system of the host vehicle 1701. Instead, whether a particular vehicle is determined to be aggressive or defensive is determined by inference based on, for example, the observed position and acceleration of the target vehicle. Figure 17A In FIG, based on position, velocity, and / or relative acceleration, host vehicle 1701 may determine that vehicle 1703 has an aggressive tendency, and therefore, host vehicle 1701 may stop and wait for target vehicle 1703 to pass rather than attempting to merge in front of target vehicle 1703. However, in Figure 17B , target vehicle 1701 (again based on the observed position, velocity and / or relative acceleration of vehicle 1710) recognizes that target vehicle 1710 traveling behind vehicle 1703 is exhibiting a defensive tendency, and therefore completes a successful merge in front of target vehicle 1710 and behind target vehicle 1703.

[0381] Figure 18 A flow chart representing an example algorithm for navigating a host vehicle based on the predicted aggressiveness of other vehicles is provided. Figure 18 In an example, a level of aggressiveness associated with at least one target vehicle can be inferred based on the target vehicle's observed behavior relative to objects in the target vehicle's environment. For example, at step 1801, at least one processing device (e.g., processing device 110) of a host vehicle navigation system can receive a plurality of images representing the host vehicle's environment from a camera associated with the host vehicle. At step 1803, analysis of one or more of the received images can enable the at least one processing device to identify a target vehicle (e.g., vehicle 1703) in the host vehicle's environment. At step 1805, analysis of one or more of the received images can enable the at least one processing device to identify at least one obstacle to the target vehicle in the host vehicle's environment. This object can include debris in the roadway, a stop light / traffic light, a pedestrian, another vehicle (e.g., a vehicle traveling in front of the target vehicle, a parked vehicle, etc.), a box in the roadway, a road barrier, a curb, or any other type of object that may be encountered in the host vehicle's environment. At step 1807, analysis of one or more of the received images can enable the at least one processing device to determine at least one navigation characteristic of the target vehicle relative to the at least one identified obstacle to the target vehicle.

[0382] Various navigational characteristics can be used to infer the aggressiveness level of a detected target vehicle in order to develop an appropriate navigational response to the target vehicle. For example, such navigational characteristics can include the relative acceleration between the target vehicle and at least one identified obstacle, the distance of the target vehicle from the obstacle (e.g., the following distance of the target vehicle behind another vehicle), and / or the relative speed between the target vehicle and the obstacle.

[0383] In some embodiments, the navigation characteristics of the target vehicle can be determined based on outputs from sensors associated with the host vehicle (e.g., radar, speed sensors, GPS, etc.). However, in some cases, the navigation characteristics of the target vehicle can be determined in part or in whole based on an analysis of images of the host vehicle's environment. For example, the image analysis techniques described above and in, for example, U.S. Patent No. 9,168,868 (which is incorporated herein by reference) can be used to identify target vehicles within the host vehicle's environment. Furthermore, monitoring the position of the target vehicle in captured images over time and / or monitoring the position of one or more features associated with the target vehicle (e.g., taillights, headlights, bumpers, wheels, etc.) in captured images can enable determination of the relative distance, speed, and / or acceleration between the target vehicle and the host vehicle or between the target vehicle and one or more other objects in the host vehicle's environment.

[0384] The aggressiveness level of an identified target vehicle can be inferred from any suitable observed navigational characteristic of the target vehicle, or any combination of observed navigational characteristics. For example, a determination of aggressiveness can be made based on any observed characteristic and one or more predetermined threshold levels, or any other suitable qualitative or quantitative analysis. In some embodiments, a target vehicle may be considered aggressive if it is observed following the host vehicle or another vehicle at a distance less than a predetermined aggressiveness distance threshold. On the other hand, a target vehicle may be considered defensive if it is observed following the host vehicle or another vehicle at a distance greater than a predetermined defensiveness distance threshold. The predetermined aggressiveness distance threshold need not be the same as the predetermined defensiveness distance threshold. Furthermore, either or both the predetermined aggressiveness distance threshold and the predetermined defensiveness distance threshold may include a range of values ​​rather than a single-line value. Furthermore, neither the predetermined aggressiveness distance threshold nor the predetermined defensiveness distance threshold need be fixed. Rather, these values ​​or ranges of values ​​may vary over time, and different thresholds / threshold ranges may be applied based on the observed characteristics of the target vehicle. For example, the applied threshold may depend on one or more other characteristics of the target vehicle. Higher observed relative speeds and / or accelerations may warrant the application of larger thresholds / ranges. Conversely, lower relative speeds and / or accelerations, including zero relative speed and / or acceleration, may warrant applying a smaller distance threshold / range when making offensive / defensive inferences.

[0385] Aggressiveness / defensiveness inferences can also be based on relative speed and / or relative acceleration thresholds. A target vehicle can be considered aggressive if its observed relative speed and / or relative acceleration relative to another vehicle exceeds a predetermined level or range. A target vehicle can be considered defensive if its observed relative speed and / or relative acceleration relative to another vehicle falls below a predetermined level or range.

[0386] While an aggressive / defensive determination can be made based solely on any observed navigation characteristic, the determination can also depend on any combination of observed characteristics. For example, as noted above, in some cases, a target vehicle may be deemed aggressive based solely on the observation that the target vehicle is following another vehicle at a distance below a certain threshold or range. However, in other cases, a target vehicle may be deemed aggressive if it is both following another vehicle by less than a predetermined amount (which may be the same or different threshold applied when the determination is based solely on distance) and has a relative speed and / or relative acceleration greater than a predetermined amount or range. Similarly, a target vehicle may be deemed defensive based solely on the observation that the target vehicle is following another vehicle at a distance greater than a certain threshold or range. However, in other cases, a target vehicle may be deemed defensive if it is both following another vehicle by more than a predetermined amount (which may be the same or different threshold applied when the determination is based solely on distance) and has a relative speed and / or relative acceleration less than a predetermined amount or range. System 100 can act aggressively / defensively in situations such as: the vehicle exceeds 0.5G acceleration or deceleration (e.g., jerk 5m / s3), the vehicle has a lateral acceleration of 0.5G when changing lanes or on a curve, the vehicle causes another vehicle to do any of the above, the vehicle changes lanes and causes another vehicle to yield at a deceleration exceeding 0.3G or a jerk of 3m / s3, and / or the vehicle changes two lanes without stopping.

[0387] It should be understood that a reference to a number that exceeds a range may indicate that the number exceeds all values ​​associated with the range or falls within the range. Similarly, a reference to a number that falls below a range may indicate that the number falls below all values ​​associated with the range or falls within the range. In addition, although the examples described for making aggressive / defensive inferences are described with respect to distance, relative acceleration, and relative speed, any other suitable number may be used. For example, any indirect indication of the time to collision or the distance, acceleration, and / or speed of the target vehicle may be used for calculation. It should also be noted that although the above examples focus on a target vehicle relative to other vehicles, an aggressive / defensive inference may be made by observing the navigation characteristics of the target vehicle relative to any other type of obstacle (e.g., pedestrians, road barriers, traffic lights, debris, etc.).

[0388] Return to Figure 17A and 17BIn the illustrated example, when host vehicle 1701 approaches a roundabout, a navigation system, including at least one processing device thereof, may receive an image stream from a camera associated with the host vehicle. Based on analysis of one or more of the received images, any of target vehicles 1703, 1705, 1706, 1708, and 1710 may be identified. Furthermore, the navigation system may analyze navigational characteristics of one or more of the identified target vehicles. The navigation system may recognize that the gap between target vehicles 1703 and 1705 represents a potential first opportunity to merge into the roundabout. The navigation system may analyze target vehicle 1703 to determine an aggressiveness indicator associated with target vehicle 1703. If target vehicle 1703 is deemed aggressive, the host vehicle's navigation system may choose to yield to vehicle 1703 rather than merge in front of it. On the other hand, if target vehicle 1703 is deemed defensive, the host vehicle's navigation system may attempt to complete the merge ahead of vehicle 1703.

[0389] As host vehicle 1701 approaches a roundabout, at least one processing device of the navigation system may analyze captured images to determine navigational characteristics associated with target vehicle 1703. For example, based on the images, it may be determined that vehicle 1703 is following vehicle 1705 at a distance that provides sufficient clearance for host vehicle 1701 to safely enter. In fact, it may be determined that vehicle 1703 is following vehicle 1705 at a distance exceeding an aggressive distance threshold, and therefore, based on this information, the host vehicle navigation system may be inclined to identify target vehicle 1703 as being defensive. However, in some cases, as discussed above, more than one navigational characteristic of the target vehicle may be analyzed when making an aggressive / defensive determination. Further analysis may determine that, while target vehicle 1703 is following behind target vehicle 1705 at a non-aggressive distance, vehicle 1703 has a relative speed and / or relative acceleration relative to vehicle 1705 that exceeds one or more thresholds associated with aggressive behavior. In practice, host vehicle 1701 may determine that target vehicle 1703 is accelerating relative to vehicle 1705 and closing the gap between them. Based on further analysis of relative speed, acceleration, and distance (and even the rate at which the gap between vehicles 1703 and 1705 is closing), host vehicle 1701 may determine that target vehicle 1703 is behaving aggressively. Thus, while there may be sufficient separation for the host vehicle to safely navigate into, host vehicle 1701 may anticipate that merging in front of target vehicle 1703 will result in the aggressively navigating vehicle being positioned directly behind the host vehicle. Furthermore, based on observed behavior through image analysis or other sensor output, host vehicle 1701 may anticipate that target vehicle 1703 will continue to accelerate toward host vehicle 1701 or continue to travel at a non-zero relative speed toward host vehicle 1701 if host vehicle 1701 were to merge in front of it. This scenario may be undesirable from a safety perspective and may also be uncomfortable for the host vehicle's passengers. For such reasons, host vehicle 1701 may choose to yield to vehicle 1703, such as Figure 17B As shown, merging into the roundabout behind vehicle 1703 and in front of vehicle 1710 (which is deemed to be defensive based on an analysis of one or more of its navigation characteristics).

[0390] Return to Figure 18At step 1809, at least one processing device of the navigation system of the host vehicle may determine a navigation action for the host vehicle (e.g., merging in front of vehicle 1710 and behind vehicle 1703) based on the at least one identified navigation characteristic of the target vehicle relative to the identified obstacle. To implement the navigation action (at step 1811), the at least one processing device may cause at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action. For example, the brakes may be applied to adjust the Figure 17A The vehicle 1703 in the middle gives way and the accelerator and steering of the host vehicle's wheels can be applied to make the host vehicle enter the roundabout behind the vehicle 1703, as shown in FIG. Figure 17B shown.

[0391] As described in the examples above, the navigation of the host vehicle may be based on the navigation characteristics of the target vehicle relative to another vehicle or object. Alternatively, the navigation of the host vehicle may be based solely on the navigation characteristics of the target vehicle without specific reference to another vehicle or object. For example, in Figure 18 At step 1807, analysis of the plurality of images captured from the host vehicle's environment may enable determination of at least one navigational characteristic of the identified target vehicle, the at least one navigational characteristic being indicative of a level of aggressiveness associated with the target vehicle. Navigational characteristics may include speed, acceleration, and the like that do not require reference to another object or target vehicle in order to make an aggressiveness / defensiveness determination. For example, an observed acceleration and / or speed associated with the target vehicle that exceeds a predetermined threshold, or falls within or exceeds a range of values, may indicate aggressive behavior. Conversely, an observed acceleration and / or speed associated with the target vehicle that falls below a predetermined threshold, or falls within or exceeds a range of values, may indicate defensive behavior.

[0392] Of course, in some cases, observed navigational characteristics (e.g., position, distance, acceleration, etc.) may be referenced relative to the host vehicle in order to make an aggressiveness / defensiveness determination. For example, observed navigational characteristics of a target vehicle that indicate a level of aggressiveness associated with the target vehicle may include an increase in relative acceleration between the target vehicle and the host vehicle, a following distance of the target vehicle behind the host vehicle, a relative speed between the target vehicle and the host vehicle, etc.

[0393] Navigation based on accident responsibility constraints

[0394] As described in the above sections, planned navigation actions can be tested against predetermined constraints to ensure that certain rules are met. In some embodiments, this concept can be extended to considerations of potential accident liability. As discussed below, a primary purpose of autonomous navigation is safety. Since absolute safety may not be possible (e.g., at least because a particular host vehicle under autonomous control cannot control other vehicles in its surroundings - the particular host vehicle can only control its own actions), using potential accident liability as a consideration in autonomous navigation and, in fact, as a constraint on planned actions can help ensure that a particular autonomous vehicle does not take any actions that are considered unsafe, such as those for which potential accident liability could be attributed to the host vehicle. A desired level of accident avoidance (e.g., less than 10 accidents per driving hour) can be achieved if the host vehicle only takes actions that are safe and that are determined not to cause accidents for which the host vehicle is itself at fault or responsible. -9 ).

[0395] Challenges posed by most current approaches to autonomous driving include a lack of safety guarantees (or at least an inability to provide the desired level of safety) and a lack of scalability. Consider the problem of ensuring safe driving by multiple agents. Since society is unlikely to tolerate road fatalities caused by machines, an acceptable level of safety is crucial for the acceptance of autonomous vehicles. While the goal could be to provide zero accidents, this may not be possible because multiple agents are often involved in an accident and one might imagine situations where an accident occurs only because of the attribution of blame to other agents. For example, Figure 19 As shown, host vehicle 1901 is driving on a multi-lane highway, and while host vehicle 1901 can control its own movements relative to target vehicles 1903, 1905, 1907, and 1909, it cannot control the movements of the target vehicles around it. Therefore, if vehicle 1905, for example, suddenly cuts into the host vehicle's lane on a collision course with the host vehicle, host vehicle 1901 may be unable to avoid an accident with at least one of the target vehicles. To address this challenge, autonomous vehicle practitioners typically resort to a statistically driven approach, where safety checks become more rigorous as more miles of data are collected.

[0396] However, to understand the nature of the problem with data-driven safety approaches, first consider that the probability of a fatality resulting from an accident for every hour of (human) driving is known to be 10 -6 It is reasonable to assume that for society to accept machines replacing humans in driving tasks, the death rate should be reduced by three orders of magnitude, to 10 per hour. -9 This estimate is similar to the airbag mortality rate assumption based on aviation standards. For example, 10 -9is the probability that the wing will spontaneously detach from the aircraft in mid-air. However, it is not practical to try to guarantee safety using data-driven statistical methods (which provide additional confidence by accumulating miles driven). -9 The amount of data required to determine the probability of mortality is proportional to the inverse of that probability (i.e., 10 9 hours of data), which is roughly about thirty billion miles. Furthermore, multi-agent systems interact with their environment and may not be validated offline (unless a realistic simulator is available that mimics real human driving with its richness and complexity, such as reckless driving, but the problem of validating a simulator will be even more difficult than creating a safe autonomous vehicle agent). Any change to the planning and control software will require the same amount of new data collection, which is obviously unwieldy and impractical. Furthermore, evolving a system through data always suffers from a lack of explainability and accountability of the actions taken - if an autonomous vehicle (AV) gets into an accident resulting in a fatality, we need to understand why. Therefore, a model-based safety approach is needed, but existing "functional safety" and ASIL requirements in the automotive industry were not designed to cope with multi-agent environments.

[0397] The second major challenge in developing safe driving models for autonomous vehicles is the need for scalability. The underlying premise of AVs isn't simply "building a better world," but rather that mobility without a driver can be supported at a lower cost than with a driver. This premise is always coupled with the concept of scalability—in the sense of supporting mass production of AVs (in the millions), and more importantly, in the sense of supporting negligible incremental costs to enable driving in new cities. So, while the cost of computing and sensing is certainly important, the cost of calibration and the ability to drive "everywhere" rather than in a select few cities are also necessary requirements to support the business if AVs are to be manufactured in large quantities.

[0398] The problem with most current approaches is a “brute force” mentality along three axes: (i) the required “computational density”, (ii) the way HD maps are defined and created, and (iii) the required specifications for sensors. The brute force approach defies scalability and shifts the focus to a future where unlimited in-vehicle computing is ubiquitous, the cost of constructing and maintaining HD maps becomes negligible and scalable, and exotic ultra-advanced sensors are developed and productized at automotive grade and at negligible cost. A future where any of the above is achieved is indeed plausible, but making all of the above applicable is likely a low probability event. Therefore, there is a need to provide formal models that integrate safety and scalability into a socially acceptable AV program and that are scalable in the sense of supporting millions of cars driving anywhere in the developed world.

[0399] The disclosed embodiments represent solutions that can provide the target safety level (or can even exceed the safety goal) and can also be scaled to systems consisting of millions (or more) of autonomous vehicles. On the safety front, a model called "Responsibility Sensitive Safety" (RSS) is introduced that formalizes the concept of "accident attribution", is explainable and accountable, and incorporates a sense of "responsibility" into the actions of robotic agents. The definition of RSS is independent of the way in which RSS is implemented - a key feature that enables the purpose of creating a convincing global safety model. RSS is motivated by the observation that (e.g. Figure 19 (as shown) agents play asymmetric roles in an accident, where typically only one of the agents is responsible and therefore liable for the accident. The RSS model also includes a formal treatment of "cautious driving" under limited sensing conditions, where not all agents are always visible (e.g., due to occlusions). A major purpose of the RSS model is to guarantee that an agent never creates an accident that is "attributed" to it or for which it is responsible. The model can only be useful if it has an efficient policy that complies with RSS (e.g., a function that maps "sensory states" to actions). For example, an action that seems innocent at the current moment may lead to a catastrophic event in the distant future (a "butterfly effect"). RSS can be useful for constructing a set of local constraints on the short-term future that can guarantee (or at least virtually guarantee) that no accidents will occur in the future due to the actions of the host vehicle.

[0400] Another contribution revolves around introducing a "semantic" language consisting of units, measurements, and action spaces, along with specifications for how to incorporate them into the planning, sensing, and actuation of an AV. To clarify the semantics, in this context, consider how someone taking a driving lesson might be instructed to think about a "driving strategy." These instructions are not geometric—the instructions don't take the form of "drive at your current speed for 13.7 meters, then at 0.8 m / s." 2 Instead, instructions are semantic in nature—“follow the car in front of you” or “pass the car on your left.” The typical language of human driving strategies is in terms of longitudinal and lateral objectives, not acceleration vectors in geometric units. Formal semantic languages ​​can be useful on several fronts related to: the computational complexity of planning (which does not scale exponentially with time and the number of agents), the way safety and comfort interact, the way computations on sensing are defined, and the specification of sensor modalities and how they interact in fusion methods. Fusion methods (based on semantic languages) can ensure that RSS models achieve the required 10 km / h per hour driven. -9 The probability of death, while only about 10 5 The dataset of 10 hours of driving data was used for offline verification.

[0401] For example, in a reinforcement learning setting, a Q-function (e.g., a function that evaluates the long-term quality of taking an action a∈A when the agent is in state s∈S; given such a Q-function, a natural choice of action would be to pick the action with the highest quality π(s) = argmax a Q(s,a)) can be defined on a semantic space where the number of trajectories to be reviewed at any given time is quantized in 10 4 is bounded, regardless of the time horizon used for planning. The signal-to-noise ratio in this space can be high, allowing efficient machine learning methods to successfully model the Q-function. In the case of computations on sensing, semantics can allow distinguishing errors that affect safety from errors that affect driving comfort. We define a PAC model for sensing (Probably Approximately Correct (PAC)), borrowing Valiant's terminology for PAC learning) that is connected to the Q-function, and show how measurement errors can be incorporated into planning in a way that conforms to RSS but allows for optimization of driving comfort. The semantic language may be important to the success of certain aspects of this model, as other standard error measures (such as error relative to a global coordinate system) may not conform to the PAC sensing model. Furthermore, the semantic language may be an important enabler for defining HD maps that can be built using low-bandwidth sensory data and therefore can be built through crowdsourcing and support scalability.

[0402] In summary, the disclosed embodiments may include a formalized model covering the following key components of an AV: sensing, planning, and action. This model can help ensure that, from a planning perspective, no accidents occur for which the AV itself is responsible. Furthermore, through the PAC sensing model, the described fusion approach can only require a very reasonable amount of offline data collection to comply with the described safety model, even in the presence of sensing errors. Furthermore, the model can tie safety and scalability together through a semantic language, providing a complete approach for safe and scalable AVs. Finally, it is worth noting that developing an accepted safety model that will be adopted by industry and regulators may be a necessary condition for the success of AVs.

[0403] The RSS model can generally follow the classic sense-plan-act robotic control approach. The sensing system can be responsible for understanding the current state of the host vehicle's environment. The planning part (which can be called a "driving policy" and can be implemented by a set of hard-coded instructions, by a trained system (e.g., a neural network), or by a combination) can be responsible for determining what the best next action is (e.g., how to move from the left lane to the right lane to exit the highway) given the available options for completing the driving goal. The action part is responsible for implementing the plan (e.g., a system of actuators and one or more controllers for steering, accelerating and / or braking the vehicle, etc., in order to implement the selected navigation action). The embodiments described below mainly focus on the sensing part and the planning part.

[0404] Accidents can result from sensing errors or planning errors. Planning is a difficult task for multi-agents because there are other road users (humans and machines) that may react to the actions of the AV. The RSS model described is designed to address the safety issues of the planning part, among others. This can be called multi-agent safety. In statistical methods, the estimation of the probability of planning errors can be done "online". That is, after each software update, billions of miles would have to be driven with the new version to provide an acceptable level of estimate of the frequency of planning errors. This is obviously not feasible. As an alternative, the RSS model can provide 100% guarantee (or almost 100% guarantee) that the planning module will not make mistakes that are attributable to the AV (the concept of "attribution" is formally defined). The RSS model can also provide an efficient way to verify it without relying on online testing.

[0405] Errors in the sensing system may be easier to verify because sensing can be independent of vehicle motion, and thus we can use “offline” data to verify the probability of severe sensing errors. However, even if more than 10 9 Offline data from 10 hours of driving is also challenging. As part of the description of the disclosed sensing system, a fusion method is described that can be verified using significantly smaller amounts of data.

[0406] The described RSS system is also scalable to millions of cars. For example, the described semantic driving policy and applied safety constraints can be consistent with sensing and mapping requirements that can scale to millions of cars even with today's technology.

[0407] The fundamental building block of such a system is a thorough safety definition—the minimum standard to which an AV system may be expected to conform. As shown in the following technical lemma, statistical methods for verifying AV systems are shown to be infeasible, even for simple statements such as "the system experiences N incidents per hour." This means that a model-based safety definition is the only viable tool for verifying AV systems.

[0408] Lemma 1 Let X be a probability space and let A be an event, where Pr(A) = p1 < 0.1. Suppose we sample from X iid samples, and let but

[0409] Pr(Z=0)≥e -2 .

[0410] Prove that we use the inequality 1-x ≥ e -2x (Completeness is proved in Appendix A.1), we get

[0411]

[0412] Corollary 1: Assume that AV system AV1 has an accident with a small but insufficient probability p1. Any deterministic calibration procedure given 1 / p1 samples will (with constant probability) fail to distinguish between AV1 and a different AV system AV0 that never has an accident.

[0413] To get an idea of ​​typical values ​​for these probabilities, suppose we expect an accident probability of 10 per hour. -9 , while one AV system provides only 10 -8 Even if the system gets 10 8 There is also a constant probability that the calibration process will not indicate that the system is dangerous.

[0414] Finally, it's important to note that this challenge is designed to disable a single, specific, dangerous AV system. A complete solution cannot be considered a single system, as new versions, bug fixes, and updates will be necessary. From the verifier's perspective, every change, even to a single line of code, creates a new system. Therefore, a statistically verified solution must be verified online on new samples after each small fix or change to account for the shift in the state distribution observed and achieved by the new system. It's infeasible to repeatedly and systematically obtain such a large number of samples (and even then, fail to verify the system with constant probability).

[0415] Furthermore, any statistical claim must be formalized in order to be measurable. Making a statistical statement about the number of accidents a system has had is significantly weaker than stating that it drives safely. In order to make this statement, one must formally define what safety is.

[0416] Absolute security is impossible

[0417] If no accidents will occur after an action a taken by car c at some future time, then the action can be considered absolutely safe. Figure 19It is not possible to achieve absolute safety without the use of a vehicle 1901 as depicted. From the perspective of vehicle 1901, no action can ensure that surrounding cars will not crash into it. It is also not possible to solve this problem by prohibiting autonomous cars from being in such situations. Since every highway with more than two lanes will lead to this situation at some point, prohibiting this scenario is equivalent to a requirement to stay in the garage. At first glance, the consequences may seem disappointing. Nothing is absolutely safe. However, as defined above, such a requirement for absolute safety may be too demanding, as evidenced by the fact that human drivers will not follow the requirement for absolute safety. Instead, humans act according to a concept of safety based on responsibility.

[0418] Responsible for Sensitive Security (RSS)

[0419] An important aspect missing from the concept of absolute safety is the asymmetric nature of most accidents – usually one of the drivers is responsible for the crash and is therefore blamed. Figure 19 In the example, if left-hand vehicle 1909 suddenly steers toward center vehicle 1901, the center vehicle is not at fault. To formalize this fact, AV 1901's behavior of remaining in its lane can be considered safe, given its lack of responsibility. To this end, a formal concept of "accident attribution" or accident responsibility is described, which can serve as a prerequisite for a safe driving method.

[0420] As an example, consider the simple case of two cars c f 、c r A car is driving at the same speed along a straight road, behind another car. Assume that c f (The car in front) brakes suddenly because of an obstacle on the road and tries to avoid it. Unfortunately, c r No with c f Keep enough distance, fail to react in time, and hit c f On the back side. Obviously, the blame should be placed on c r The car behind is responsible for maintaining a safe distance from the car in front and for being prepared for unexpected but reasonable braking.

[0421] Next, consider a broader set of scenarios: driving on a multi-lane road, where cars can freely change lanes, cut into the paths of other cars, drive at varying speeds, and so on. To simplify the following discussion, assume a straight road on a flat surface, with the lateral and longitudinal axes being the x- and y-axes, respectively. Under mild conditions, this can be achieved by defining a homomorphism between actual curved roads and straight roads. Furthermore, consider discrete time space. This definition helps distinguish between two intuitively distinct groups of situations: simple situations, where no significant lateral maneuvers are performed, and more complex situations involving lateral movement.

[0422] Definition 1 (Car Corridor): The corridor of car c is the range [c x,左 ,c x,右 ]×[±∞], where c x,左 ,c x,右 The positions of the leftmost and rightmost corners of c.

[0423] Definition 2 (cut-in) car c1 (e.g., Figure 20A and 20B The car 2003 in the example will be switched to the car c0 at time t (for example, Figure 20A and 20B 2001), provided that the former does not intersect with the corridor of c0 at time t-1 but intersects with the corridor of the latter at time t.

[0424] A further distinction can be made between the front and rear of the corridor. The term "direction of approach" can describe movement in the direction of the relevant corridor boundary. These definitions can be used to define situations with lateral movement. For simple situations where this does not exist, such as a car following another car, a safe longitudinal distance is defined:

[0425] Definition 3 (Safe Longitudinal Distance) Car c r (Car 2103) and another car c f (Car 2105)(which is located in c r The longitudinal distance between the front of the corridor is 2101 ( Figure 21 ) is safe with respect to the response time p, provided that for f Any braking command a, |a| 最大,致动 , if c r will apply its maximum brake from time p until it comes to a complete stop, then the car will not f collision.

[0426] Lemma 2 below will calculate d according to: c r 、c f speed, response time ρ, and maximum acceleration a 最大,致动 ​ρ and a 最大,致动 Both are constants that should be adjusted to some reasonable values. In another example, the response time ρ and the maximum acceleration a 最大,致动 Any of these may be set for a particular vehicle or vehicle type, or may be adapted / adjusted based on measurements or otherwise input parameters regarding vehicle conditions, road conditions, user (e.g., driver or passenger) preferences, etc.

[0427] Lemma 2 Let c r The longitudinal axis is c f The vehicle behind . Let a 最大,致动 、a 最大,加速 is the maximum braking and acceleration command, and let ρ be c r The response time of r 、υ f is the longitudinal velocity of the car, and l f 、l r is the length of the car. Define υ p,最大 =υ r +ρ·a 最大,加速 , and define as well as Let L = (l r +l f ) / 2. Then, c r The minimum safe longitudinal distance is:

[0428]

[0429] Prove that d t is the distance at time t. To prevent accidents, for each t, ​​there must be d t >L. In order to construct d 最小 , we need to find the tightest required lower bound on d0. Obviously, d0 must be at least L. As long as both cars have not stopped after T ≥ ρ seconds, the speed of the leading car will be υ f –Ta 最大,致动 , and c r The upper bound of the velocity will be υ ρ,最大 –(T–ρ)a 最大,加速 . Therefore, the lower bound on the distance between cars after T seconds will be:

[0430]

[0431] Please note that T r It is c r The time to reach a complete stop (speed is 0), and T f is the time it takes for the other vehicle to come to a complete stop. 最大,致动 (T r –Tf ) = υ ρ,最大 –υ f + ρa 最大,致动 , so if T r ≤ T f , it is sufficient to require d0 > L. If T r > T f , then

[0432]

[0433] require d Tr > L and rearranging the terms gives the proof.

[0434] Finally, define a comparison operator that allows comparison with a "margin" concept: when comparing lengths, speeds, etc., it is necessary to accept very similar numbers as "equal".

[0435] Definition 4 (μ - comparison) The μ - comparison of two numbers a, b is a > μ b when a > b + μ, a < μ b when a < b - μ, and a = μ b when |a - b| ≤ μ.

[0436] The following comparisons (argmin, argmax, etc.) are μ - comparisons for some suitable μ. Suppose an accident occurs between cars c1 and c2. To consider who is responsible for the accident, define the relevant time points to be checked. This is a time point before the accident, and intuitively, it is the "point of no return"; after this point, no measures can be taken to prevent the accident.

[0437] Definition 5 (Responsible time) The responsible time of an accident is the earliest time before the accident, where:

[0438] · There is an intersection between the corridors of one car and the other car, and

[0439] · The longitudinal distance is unsafe.

[0440] Obviously, such a time exists because at the moment of the accident, both of these conditions are satisfied. The responsible time can be split into two separate categories:

[0441] · The responsible time with incursion, i.e., these responsible times are the first moments when one car crosses the corridor of the other car and is at a non - safe distance.

[0442] · The responsible time without incursion, i.e., it has already crossed the corridor, is at a safe longitudinal distance,

[0443] and this distance changes to unsafe at this responsible time.

[0444] Definition 6 (Lateral Velocity μ Failure): Assume that a cut-in occurs between cars c1 and c2. We say that c1 suffers a lateral velocity μ failure if its lateral velocity relative to the cut-in direction is μ higher than that of c2.

[0445] It should be noted that the direction of the velocity is important: for example, a velocity of –1,1 (both cars hit each other) is a draw, but if the velocity is 1,1+μ / 2, then the car that has a positive direction towards the other car is to be attributed. Intuitively, this definition would allow us to attribute the car that is driving very fast sideways towards the other car.

[0446] Definition 7 (Gain of Lateral Position (μ1, μ2)) Assume that a cut-in occurs between cars c1 and c2. We say that c1 has a gain of lateral position (μ1, μ2) if its lateral position relative to the center of the cut-in lane (closest to the center of the corridor involved in the cut-in) is less than μ1 (in absolute value) and less than the lateral position of c2 by μ2.

[0447] Intuitively, no blame would be placed on a car that is very close to the lane center (μ1) and is even closer than another car (by μ2).

[0448] Definition 8 (Attribution) Attribution or liability for an accident between cars c1 and c2 varies with their status at the time of attribution and is defined as follows:

[0449] If the responsible time is not the cut-in time, the responsibility lies with the car behind.

[0450] If the imputation time is also the cut-in time, then both cars are imputed unless, without loss of generality, for one of the cars, c1, for some predefined μ, the following two conditions hold:

[0451] The car will not lose ground due to lateral speed,

[0452] The car benefits from its lateral position.

[0453] In this case, c1 would survive. In other words, if an unsafe cut-in occurs, both cars are blamed unless one of the cars is not (significantly) faster laterally and (significantly) closer to the center of the lane. This captures the desired behavior: if following a car, maintain a safe distance; and if cutting into a corridor with a car driving only in its own lane, do so only at a safe distance. An automated controller-based system designed to adhere to the above safety guidelines should not lead to overly defensive driving, as discussed further below.

[0454] Coping with limited sensing

[0455] After considering the highway example, the second example addresses the problem of limited sensing. A very common human response when being blamed for an accident is of the form “but I couldn’t see him.” A lot of the time, this is true. Human sensing capabilities are limited, sometimes because of unconscious decisions to focus on different parts of the road, sometimes because of carelessness, and sometimes because of physical limitations—it’s impossible to see a pedestrian hidden behind a parked car. Of these human limitations, advanced automated sensing systems may only be affected by the latter: a 360° view of the road, and the fact that computers are never careless, allow advanced automated sensing systems to surpass human sensing capabilities. Returning to the “but I couldn’t see him” example, the appropriate answer is “well, you should have been more careful.” To formalize what to be careful about with regard to limited sensing, consider Figure 22 The depicted scenario. Car 2201 (c0) is attempting to exit a parking lot and merge onto a (presumably) busy road, but cannot see if there are any cars on the street because its view is obscured by buildings 2203. Assume this is a narrow city street with a speed limit of 30 km / h. A human driver's behavior is to slowly merge onto the road, gaining more and more field of view until the sensing limitation is removed. A significant moment should be defined: the first moment an occluded object is revealed to us; after the occluded object is revealed, it is treated like any other sensed object.

[0456] Definition 9 (Exposure Time) The exposure time of an object is the first time we see it.

[0457] Definition 10 (Liability due to unreasonable speed) Assume that at or after the exposure time, car c1 (car 2205) moves at a speed υ>υ 极限 driving, and c0 did not do so. Then, only c1 is to blame. We say that c1 is to blame for the unreasonable speed.

[0458] This extension allows c0 to exit the parking lot safely. Using our previous definition of responsible sensitive security, and dynamic υ 极限 definitions (which use road conditions and speed limits, plus a reasonable margin), the only thing required is to check that in the worst case, as shown, the cut-in is within a safe longitudinal distance, assuming that c1 does not exceed υ 极限 Intuitively, this would encourage c0 to drive slower and farther away from the obstruction, thereby slowly increasing its field of view and allowing it to merge safely into the street later.

[0459] After extending the accident liability definition to the basic case of limited sensing, a series of extensions can address similar situations. There are questions about what might be obscured (a fast car might not be obscured between two closely parked cars, but a pedestrian might be) and what the worst-case maneuver a car can make (a pedestrian's 极限 The simple assumption that the vehicle is much smaller than the car implies a limitation on driving: one must be prepared for the worst-case scenario and be able to react when the moment of exposure suddenly arrives. A more complex example can be obtained in an urban scenario where pedestrians may be obscured by parked cars. Accident liability for pedestrian accidents can be defined as:

[0460] Definition 11 (Attribution of Liability for Pedestrian Accidents) Liability for pedestrian accidents always lies with the vehicle unless one of the following three conditions is true:

[0461] The car hits the pedestrian with its side and the lateral velocity of the car relative to the direction of impact is less than μ.

[0462] The pedestrian's speed at or after the exposure time is greater than υ 极限 .

[0463] The car comes to a complete stop.

[0464] Informally, the car is not at fault only if the pedestrian hits the side of the car and the car is moving towards the pedestrian no faster than μ, or if the car is stopped, or if the pedestrian is running at superhuman speed in a direction (not necessarily the direction of impact).

[0465] While the system described may not guarantee absolute safety, it can lead to a scenario where accidents between autonomous vehicles are rare (if any). For example, if all cars (and other road users) could successfully verify that they cannot be held responsible for an accident caused by an action ...

Claims

1. A system for navigating a host vehicle, the system comprising: at least one processor comprising circuitry and memory, wherein the memory comprises instructions executable by the circuitry to cause the at least one processor to: receiving at least one image acquired by an image capture device, the at least one image representing an environment of the host vehicle; analyzing the at least one image to identify at least one characteristic associated with the environment of the host vehicle; A navigation action for the host vehicle is determined based on: the at least one characteristic associated with the environment of the host vehicle, and a steering limit corresponding to a maximum allowable lateral acceleration for the host vehicle; and One or more actuators associated with the host vehicle are caused to perform a determined navigation action. 2 . The system of claim 1 , wherein the at least one characteristic associated with the environment of the host vehicle comprises a characteristic associated with a target vehicle in the environment of the host vehicle. 3 . The system of claim 2 , wherein the characteristics associated with the target vehicle include a vehicle type. 4 . The system of claim 2 , wherein the characteristic associated with the target vehicle includes vehicle size. 5 . The system of claim 2 , wherein the characteristics associated with the target vehicle include a vehicle model.

6. The system of claim 2, wherein the characteristic associated with the target vehicle is determined based on an analysis of the at least one image.

7. The system of claim 2, wherein the characteristic associated with the target vehicle is determined based on at least one of a lidar output or a radar output. 8 . The system of claim 1 , wherein the at least one characteristic associated with the environment of the host vehicle comprises a characteristic of an object in the environment of the host vehicle.

9. The system of claim 8, wherein the characteristic of the object is determined based on an analysis of the at least one image.

10. The system of claim 8, wherein the characteristic of the object is determined based on at least one of a lidar output or a radar output.

11. The system of claim 1, wherein the at least one characteristic associated with the environment of the host vehicle comprises a characteristic of a road surface in the environment of the host vehicle.

12. The system of claim 11, wherein the characteristic of the road surface is determined based on an analysis of the at least one image.

13. The system of claim 11, wherein the characteristic of the road surface is determined based on at least one of a lidar output or a radar output.

14. The system of claim 1, wherein the at least one characteristic comprises a curvature of a roadway in the environment of the host vehicle.

15. The system of claim 1, wherein the maximum allowable lateral acceleration is determined based on the at least one characteristic associated with the environment of the host vehicle.

16. The system of claim 1, wherein the maximum allowable lateral acceleration is determined based on a user preference.

17. The system of claim 1, wherein the steering limit is based on at least one of a wheelbase associated with the host vehicle or a turning radius associated with the host vehicle.

18. The system of claim 1, wherein the steering limit is based on an operating parameter associated with the host vehicle.

19. The system of claim 18, wherein the operating parameter includes at least one of a laden weight of the host vehicle, a number of passengers in the host vehicle, or a weight distribution within the host vehicle.

20. The system of claim 1, wherein the steering limit is based on an indication of wheel slip experienced by the host vehicle.

21. The system of claim 20, wherein the indication of the wheel slip is obtained based on an analysis of the at least one image.

22. The system of claim 1, wherein the determined navigation action comprises at least one of a lane change maneuver, a merging maneuver, or an overtaking maneuver.

23. A system according to claim 1, wherein determining the navigation action of the main vehicle includes determining a plurality of navigation actions based at least on a long-term purpose, a driving strategy, and a safety strategy of the main vehicle, wherein the plurality of navigation actions are optimized using the driving strategy to achieve the long-term purpose, and wherein the plurality of navigation actions are selected based on the safety strategy to ensure that the plurality of navigation actions are safe.

24. The system of claim 23, wherein selecting the plurality of navigation actions according to the safety policy to ensure that the plurality of navigation actions are safe includes verifying that the plurality of navigation actions allow the host vehicle to maintain a minimum safe distance between the host vehicle and at least one object.

25. The system of claim 23, wherein the steering constraints are configured such that the plurality of navigation actions do not invoke a steering maneuver that causes the host vehicle to exceed the maximum allowable lateral acceleration.

26. The system of claim 1, wherein the determined navigation action comprises a steering maneuver, and wherein determining the navigation action for the host vehicle comprises: determining, based on the characteristic associated with the environment of the host vehicle, whether implementing the steering maneuver consistent with the steering constraint will result in the host vehicle maintaining a safe distance from at least one object in the environment of the host vehicle; and If performing the steering maneuver consistent with the steering constraint would not result in the host vehicle maintaining the safe distance from at least one object in the environment of the host vehicle, causing performance of the steering maneuver consistent with the steering constraint is foregone.

27. The system of claim 26, wherein the safety distance comprises at least one of a longitudinal safety distance or a lateral safety distance.

28. A non-transitory computer readable medium storing instructions executable by at least one processor to perform a method for navigating a host vehicle, the method comprising: receiving at least one image acquired by an image capture device, the at least one image representing an environment of the host vehicle; analyzing the at least one image to identify at least one characteristic associated with the environment of the host vehicle; A navigation action for the host vehicle is determined based on: the at least one characteristic associated with the environment, and a steering limit corresponding to a maximum allowable lateral acceleration for the host vehicle; and One or more actuators associated with the host vehicle are caused to perform a determined navigation action.

29. A method for navigating a host vehicle, the method comprising: receiving at least one image acquired by an image capture device, the at least one image representing an environment of the host vehicle; analyzing the at least one image to identify at least one characteristic associated with the environment of the host vehicle; A navigation action for the host vehicle is determined based on: the at least one characteristic associated with the environment, and a steering limit corresponding to a maximum allowable lateral acceleration for the host vehicle; and One or more actuators associated with the host vehicle are caused to perform a determined navigation action.

Images