NAVIGATION SYSTEM FOR HOST VEHICLE, NAVIGATION METHOD, AND PROGRAM - Patent application

Abstract

A navigation system for a host vehicle is provided. The system may include at least one processing device including a circuit and a memory, the memory including instructions that, when executed by the circuit, cause the at least one processing device to: receive a plurality of images acquired by a camera, the plurality of images representing an environment of a host vehicle; analyze the plurality of images to identify a navigation rule suspension condition existing within the environment of the host vehicle; temporarily suspend at least one navigation rule in response to identifying the navigation rule suspension condition; and cause at least one navigation change of the host vehicle that is not constrained by the at least one navigation rule that was temporarily suspended.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a continuation application, PCT / US2018 / 013391, filed on January 11, 2018, which claims the benefit of priority to U.S. Provisional Application No. 62 / 445,500, filed on January 12, 2017, U.S. Provisional Application No. 62 / 546,343, filed on August 16, 2017, U.S. Provisional Application No. 62 / 565,244, filed on September 29, 2017, and U.S. Provisional Application No. 62 / 582,687, filed on November 7, 2017. The above applications are incorporated herein by reference in their entireties.

[0002] background Technical Field

[0002] The present disclosure relates generally to autonomous vehicle navigation. In addition, the present disclosure relates to a system and method for navigating using reinforcement learning techniques. [Background technology]

[0003] Background information

[0003] As technology continues to evolve, the goal of a fully autonomous vehicle capable of navigating on the road becomes more realistic. An autonomous vehicle may need to consider various factors and, based on those factors, may make appropriate decisions to safely and accurately reach an intended destination. For example, an autonomous vehicle may need to process and interpret information from visual information (e.g., information captured from a camera), radar, or lidar, and may also use information obtained from other sources (e.g., GPS devices, speed sensors, accelerometers, suspension sensors, etc.). At the same time, to navigate to a destination, an autonomous vehicle may need to identify its position within a particular road (e.g., a particular lane within a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, observe traffic signals and signs, proceed from one road to another at the appropriate intersections or interchanges, and respond to any other conditions that occur or develop during the operation of the vehicle. Summary of the Invention [Means for solving the problem]

[0004] overview

[0004] Embodiments according to the present disclosure provide a system and method for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation capabilities. For example, according to embodiments of the present disclosure, the disclosed system may include one, two, or more cameras that monitor the vehicle's environment. The disclosed system may provide a navigation response, for example, based on an analysis of images captured by one or more of the cameras. The navigation response may also take into account other data, including, for example, global positioning (GPS) data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, radar, lidar, etc.), and / or other map data.

[0005]

[0005] In one embodiment, a navigation system for a host vehicle may include at least one processing device. The processing device may be programmed to receive a plurality of images from a camera representing an environment of the host vehicle. The processing device may also be programmed to analyze the plurality of images to identify at least one navigation state of the host vehicle. The processing device may also be programmed to identify a jurisdiction in which the host vehicle is traveling based on at least one indicator of a position of the host vehicle, the at least one indicator being based at least in part on the analysis of the plurality of images. The processing device may also be programmed to determine at least one navigation rule specific to the identified jurisdiction. The processing device may also be programmed to cause a navigation change of the host vehicle based on the identified navigation state of the host vehicle and based on the determined at least one navigation rule specific to the identified jurisdiction.

[0006]

[0006] In one embodiment, a method of navigation for a host vehicle may include receiving a plurality of images from a camera representative of an environment of the host vehicle. The method may further include analyzing the plurality of images to identify at least one navigation state of the host vehicle. The method may further include identifying a jurisdiction in which the host vehicle is traveling based on at least one indicator of a position of the host vehicle, the at least one indicator being based at least in part on the analysis of the plurality of images. The method may further include determining at least one navigation rule specific to the identified jurisdiction. The method may further include causing a navigation change of the host vehicle based on the identified navigation state of the host vehicle and based on the determined at least one navigation rule specific to the identified jurisdiction.

[0007]

[0007] In one embodiment, a navigation system for a host vehicle may include an autonomous vehicle, the vehicle including a body, at least one image capture device configured to capture at least one image representative of the vehicle's environment, and at least one processor. The processor may be programmed to receive a plurality of images representative of the host vehicle's environment from a camera. The processor may also be programmed to analyze the plurality of images to identify at least one navigation state of the host vehicle. The processor may also be programmed to identify a jurisdiction in which the host vehicle is traveling based on at least one indicator of the host vehicle's location, the at least one indicator being based at least in part on the analysis of the plurality of images. The processor may also be programmed to determine at least one navigation rule specific to the identified jurisdiction. The processor may also be programmed to cause a navigation change of the host vehicle based on the identified navigation state of the host vehicle and based on the determined at least one navigation rule specific to the identified jurisdiction.

[0008]

[0008] In one embodiment, a navigation system for a host vehicle may include at least one processing device. The at least one processing device may also be programmed to receive a plurality of images from a camera representing an environment of the host vehicle. The at least one processing device may also be programmed to analyze the plurality of images to identify at least one target vehicle in the environment of the host vehicle. The at least one processing device may also be programmed to analyze the plurality of images to identify at least one hostile characteristic of the target vehicle relative to the host vehicle. The at least one processing device may also be programmed to cause at least one navigation change of the host vehicle to initiate overtaking of the target vehicle after identifying the at least one characteristic of the target vehicle.

[0009] In one embodiment, a method of navigation for a host vehicle may include receiving a plurality of images from a camera representing an environment of the host vehicle. The method may also include analyzing the plurality of images to identify at least one target vehicle within the environment of the host vehicle. The method may also include analyzing the plurality of images to identify at least one characteristic of the target vehicle. The method may also include initiating at least one navigation change of the host vehicle to initiate overtaking of the target vehicle after identifying the at least one characteristic of the target vehicle.

[0010]

[0010] In one embodiment, a navigation system for a host vehicle may include an autonomous vehicle, the vehicle including a body, at least one image capture device configured to capture at least one image representative of the vehicle's environment, and at least one processor. The processor may be programmed to receive a plurality of images representative of the host vehicle's environment from a camera. The processor may also be programmed to analyze the plurality of images to identify at least one target vehicle in the host vehicle's environment. The processor may also be programmed to analyze the plurality of images to identify at least one characteristic of the target vehicle. The processor may also be programmed to cause at least one navigation change of the host vehicle to initiate overtaking of the target vehicle after identifying the at least one characteristic of the target vehicle.

[0011]

[0011] In one embodiment, a navigation system for a host vehicle may include at least one processing device programmed to receive a plurality of images from a camera representative of an environment of the host vehicle. The at least one processing device may also be programmed to analyze the plurality of images to identify at least one target vehicle within the environment of the host vehicle. The at least one processing device may also be programmed to determine, based at least in part on the analysis of the plurality of images, that the target vehicle and the host vehicle have similar navigational priorities resulting in a navigational tie situation. The at least one processing device may also be programmed to cause at least one action by the host vehicle to establish a navigational priority for the host vehicle or the target vehicle. The at least one processing device may also be programmed to cause control of at least one navigational actuator of the host vehicle according to the established navigational priority.

[0012]

[0012] In one embodiment, a method of navigation for a host vehicle may include receiving a plurality of images from a camera representing an environment of the host vehicle. The method may also include analyzing the plurality of images to identify at least one target vehicle within the environment of the host vehicle. The method may also include determining, based at least in part on the analysis of the plurality of images, that the target vehicle and the host vehicle have similar navigation priorities resulting in a navigational tie situation. The method may also include causing at least one action by the host vehicle to establish a navigation priority for the host vehicle or the target vehicle. The method may also include causing control of at least one navigation actuator of the host vehicle in accordance with the established navigation priority.

[0013]

[0013] In one embodiment, a navigation system for a host vehicle may include an autonomous vehicle, the vehicle including a body, at least one image capture device configured to acquire at least one image representative of the vehicle's environment, and at least one processor programmed to receive a plurality of images representative of the host vehicle's environment from a camera. The at least one processor may also be programmed to analyze the plurality of images to identify at least one target vehicle in the host vehicle's environment. The at least one processor may also be programmed to determine, based at least in part on the analysis of the plurality of images, that the target vehicle and the host vehicle have similar navigation priorities resulting in a navigational tie situation. The at least one processor may also be programmed to cause at least one action by the host vehicle to establish a navigation priority for the host vehicle or the target vehicle. The at least one processor may also be programmed to cause control of at least one navigation actuator of the host vehicle according to the established navigation priority.

[0014]

[0014] In one embodiment, a navigation system for a host vehicle may include at least one processing device programmed to receive a plurality of images from a camera representing an environment of the host vehicle. The at least one processing device may also be programmed to analyze the plurality of images to identify the presence of a navigation rule interruption condition within the environment of the host vehicle. The at least one processing device may also be programmed to temporarily suspend at least one navigation rule in response to identifying the navigation rule interruption condition. The at least one processing device may also be programmed to cause at least one navigation change of the host vehicle that is not constrained by the temporarily suspended at least one navigation rule.

[0015]

[0015] In one embodiment, a method of navigation for a host vehicle may include receiving a plurality of images from a camera representing an environment of the host vehicle. The method may also include analyzing the plurality of images to identify the presence of a navigation rule interruption condition within the environment of the host vehicle. The method may also include temporarily suspending at least one navigation rule in response to identifying the navigation rule interruption condition. The method may also include causing at least one navigation change of the host vehicle that is not constrained by the at least one navigation rule that is temporarily suspended.

[0016]

[0016] In one embodiment, the navigation system for the host vehicle may include an autonomous vehicle, the vehicle including a body, at least one image capture device configured to acquire at least one image representative of the environment of the vehicle, and at least one processor programmed to receive a plurality of images representative of the environment of the host vehicle from the camera. The at least one processor may also be programmed to analyze the plurality of images to identify the presence of a navigation rule interruption condition in the environment of the host vehicle. The at least one processor may also be programmed to temporarily suspend at least one navigation rule in response to the identification of the navigation rule interruption condition. The at least one processor may also be programmed to cause at least one navigation change of the host vehicle that is not constrained by the temporarily suspended at least one navigation rule.

[0017]

[0017] According to other disclosed embodiments, a non-transitory computer-readable storage medium may store program instructions that are executed by at least one processing device and perform any of the methods described herein.

[0018]

[0018] The foregoing general description and the following detailed description are merely exemplary and explanatory and are not intended to be limiting of the scope of the claims.

[0019] BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various disclosed embodiments. [Brief description of the drawings]

[0020] [Figure 1] 1 is a diagrammatic representation of an exemplary system according to the disclosed embodiments. [Figure 2A]

[0021] 1 is a side view representation of an exemplary vehicle including a system according to disclosed embodiments. [Figure 2B]

[0022] 2B is a top view representation of the vehicle and system shown in FIG. 2A according to a disclosed embodiment. [Figure 2C]

[0023] 1 is a top view representation of another embodiment of a vehicle including a system according to the disclosed embodiments. [Figure 2D]

[0024] 1 is a top view representation of yet another embodiment of a vehicle including a system according to disclosed embodiments. [Figure 2E]

[0025] 1 is a top view representation of yet another embodiment of a vehicle including a system according to disclosed embodiments. [Figure 2F]

[0026] 1 is a diagrammatic representation of an exemplary vehicle control system in accordance with a disclosed embodiment. [Figure 3A]

[0027] 1 is a diagrammatic representation of the interior of a vehicle including a rearview mirror and a user interface of a vehicle imaging system according to disclosed embodiments. [Figure 3B]

[0028] 1 is a diagram of an example camera mount configured to be positioned behind a rearview mirror, facing a vehicle windshield, in accordance with a disclosed embodiment. [Figure 3C]

[0029] 3C is a diagram of the camera mount shown in FIG. 3B from a different perspective, according to a disclosed embodiment. [Figure 3D]

[0030] 1 is a diagram of an example camera mount configured to be positioned behind a rearview mirror, facing a vehicle windshield, in accordance with a disclosed embodiment. [Figure 4]

[0031] FIG. 1 illustrates an example block diagram of a memory configured to store instructions for performing one or more operations in accordance with the disclosed embodiments. [Figure 5A]

[0032] 1 is a flowchart illustrating an example process for generating one or more navigational responses based on monocular image analysis, according to disclosed embodiments. [Figure 5B]

[0033] 1 is a flowchart illustrating an example process for detecting one or more vehicles and / or pedestrians in a set of images, according to disclosed embodiments. [Figure 5C]

[0034] 1 is a flowchart illustrating an example process for detecting road markings and / or lane geometry information in a set of images, according to disclosed embodiments. [Figure 5D]

[0035] 1 is a flowchart illustrating an example process for detecting traffic lights in a set of images, according to disclosed embodiments. [Figure 5E]

[0036] 1 is a flowchart of an example process for generating one or more navigational responses based on a vehicle path, according to disclosed embodiments. [Figure 5F]

[0037] 4 is a flowchart illustrating an example process for determining whether a leading vehicle is changing lanes, according to disclosed embodiments. [Figure 6]

[0038] 1 is a flowchart illustrating an example process for generating one or more navigational responses based on stereo image analysis, according to disclosed embodiments. [Figure 7]

[0039] 1 is a flowchart illustrating an example process for generating one or more navigational responses based on an analysis of three sets of images, according to disclosed embodiments. [Figure 8]

[0040] 1 is a block diagram representation of modules that may be implemented by one or more specifically programmed processing devices of a navigation system for an autonomous vehicle according to disclosed embodiments. [Figure 9]

[0041] 1 is a graph of navigation options in accordance with a disclosed embodiment; [Figure 10]

[0042] 1 is a graph of navigation options in accordance with a disclosed embodiment; [Figure 11A]

[0043] 1 illustrates a schematic diagram of navigation options for a host vehicle within a merge area according to a disclosed embodiment. [Figure 11B]

[0043] FIG. 1 shows a schematic diagram of navigation options for a host vehicle within a merge area according to a disclosed embodiment. [Figure 11C]

[0043] FIG. 1 shows a schematic diagram of navigation options for a host vehicle within a merge area according to a disclosed embodiment. [Figure 11D]

[0044] 1 shows a pictorial representation of a double merge scenario. [Figure 11E]

[0045] 1 shows a graph of potentially useful options in a double merge scenario. [Figure 12]

[0046] 1 illustrates an illustration of a representative image captured of a host vehicle's environment along with potential navigation constraints in accordance with a disclosed embodiment; [Figure 13]

[0047] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. [Figure 14]

[0048] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. [Figure 15]

[0049] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. [Figure 16]

[0050] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. [Figure 17A]

[0051] 1 illustrates a diagram of a host vehicle navigating into a roundabout in accordance with a disclosed embodiment; [Figure 17B]

[0051] FIG. 1 shows a diagram of a host vehicle navigating into a roundabout in accordance with a disclosed embodiment. [Figure 18]

[0052] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. [Figure 19]

[0053] FIG. 1 illustrates an example block diagram of a memory configured to store instructions for performing one or more operations in accordance with the disclosed embodiments. [Figure 20]

[0054] 1 illustrates a diagram of a representative environment of a host vehicle, in accordance with a disclosed embodiment; [Figure 21]

[0055] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. [Figure 22]

[0056] FIG. 1 illustrates an example block diagram of a memory configured to store instructions for performing one or more operations in accordance with the disclosed embodiments. [Figure 23]

[0057] 1 illustrates a diagram of a representative environment of a host vehicle, in accordance with a disclosed embodiment; [Figure 24]

[0058] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. [Diagram 25]

[0059] FIG. 1 illustrates an example block diagram of a memory configured to store instructions for performing one or more operations in accordance with the disclosed embodiments. [Figure 26A]

[0060] 1 illustrates a diagram of a representative environment of a host vehicle, in accordance with disclosed embodiments; [Figure 26B]

[0060] A diagram of a representative environment of a host vehicle is shown, in accordance with the disclosed embodiments. [Figure 27]

[0061] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. [Figure 28]

[0062] FIG. 1 illustrates an example block diagram of a memory configured to store instructions for performing one or more operations in accordance with the disclosed embodiments. [Figure 29]

[0063] 1 illustrates a diagram of a representative environment of a host vehicle, in accordance with disclosed embodiments; [Diagram 30]

[0064] 1 illustrates a diagram of a representative environment of a host vehicle, in accordance with disclosed embodiments; [Diagram 31]

[0065] 4 illustrates a flowchart of an algorithm for navigating a vehicle according to a disclosed embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Detailed Description

[0066] The following detailed description refers to the accompanying drawings. Whenever possible, the same reference numbers are used in the drawings and the following description to refer to the same or similar parts. Although several exemplary embodiments are described herein, modifications, adaptations, and other implementations are possible. For example, substitutions, additions, or modifications may be made to the components shown in the drawings, and the exemplary methods described herein may be modified by substituting, reordering, deleting, or adding steps of the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. Instead, the appropriate scope is defined by the appended claims.

[0022]

[0067] Autonomous Vehicle Overview

[0068] As used throughout this disclosure, the term "autonomous vehicle" refers to a vehicle that can perform at least one navigation change without driver input. A "navigation change" refers to one or more changes in steering, braking, or acceleration / deceleration of the vehicle. To be autonomous, a vehicle need not be fully automatic (e.g., fully operable without a driver or driver input). Rather, an autonomous vehicle includes a vehicle that can operate under driver control during certain periods of time and without driver control during other periods of time. An autonomous vehicle can also include a vehicle that controls only some aspects of vehicle navigation, such as steering (e.g., to maintain a vehicle course between vehicle lane constraints), or controls some steering actions under certain circumstances (but not under all circumstances), but leaves other aspects (e.g., braking or braking under certain circumstances) to the driver. In some cases, an autonomous vehicle may handle some or all aspects of braking, speed control, and / or steering of the vehicle.

[0023]

[0069] Because human drivers typically rely on visual cues and observations to control the vehicle, the transportation infrastructure is built accordingly, with lane markings, traffic signs, and traffic lights designed to provide visual information to the driver. In light of these design features of the transportation infrastructure, an autonomous vehicle may include a camera and a processing unit that analyzes visual information captured from the vehicle's environment. The visual information may include, for example, images depicting components of the transportation infrastructure (e.g., lane markings, traffic signs, traffic lights, etc.) and other obstacles (e.g., other vehicles, pedestrians, debris, etc.) observable by the driver. Additionally, an automated vehicle may also use stored information when navigating, such as information that provides a model of the vehicle's environment. For example, the vehicle may use GPS data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data to provide information related to the vehicle's environment while the vehicle is traveling, and the vehicle (and other vehicles) may use the information to locate itself in the model. Some vehicles may also be capable of vehicle-to-vehicle communication, sharing information, peer vehicle changes of hazards or changes in the vehicle's surroundings, etc.

[0024]

[0070] System Overview

[0071] FIG. 1 is a block diagram representation of a system 100 according to an exemplary embodiment disclosed. The system 100 may include various components depending on the requirements of a particular implementation. In some embodiments, the system 100 may include a processing unit 110, an image acquisition unit 120, a position sensor 130, one or more memory units 140, 150, a map database 160, a user interface 170, and a wireless transceiver 172. The processing unit 110 may include one or more processing devices. In some embodiments, the processing unit 110 may include an application processor 180, an image processor 190, or any other suitable processing device. Similarly, the image acquisition unit 120 may include any number of image acquisition devices and components depending on the requirements of a particular application. In some embodiments, the image acquisition unit 120 may include one or more image capture devices (e.g., cameras, CCDs, any other type of image sensor), such as an image capture device 122, an image capture device 124, an image capture device 126, etc. System 100 may also include a data interface 128 that communicatively connects processing unit 110 to image acquisition unit 120. For example, data interface 128 may include any wired and / or wireless link for transmitting image data acquired by image acquisition unit 120 to processing unit 110.

[0025]

[0072] Wireless transceiver 172 may include one or more devices configured to exchange transmissions with one or more networks (e.g., cellular, Internet, etc.) over a wireless interface through the use of radio frequencies, infrared frequencies, magnetic fields, or electric fields. Wireless transceiver 172 may transmit and / or receive data using any known standard (e.g., Wi-Fi, Bluetooth, Bluetooth Smart, 802.15.4, ZigBee, etc.). Such transmissions may include communications from the host vehicle to one or more remotely located servers. Such transmissions may include communications (one-way or two-way) between the host vehicle and one or more target vehicles in the host vehicle's environment (e.g., to facilitate coordinating the host vehicle's navigation in light of or with target vehicles in the host vehicle's environment), as well as broadcast transmissions to unspecified recipients in the vicinity of the transmitting vehicle.

[0026]

[0073] Both application processor 180 and image processor 190 may include various types of hardware-based processing devices. For example, either or both of application processor 180 and image processor 190 may include a microprocessor, a preprocessor (such as an image preprocessor), a graphics processor, a central processing unit (CPU), support circuits, a digital signal processor, an integrated circuit, memory, or any other type of device suitable for executing applications and processing and analyzing images. In some embodiments, application processor 180 and / or image processor 190 may include any type of single-core or multi-core processor, mobile device microcontroller, central processing unit, etc. A variety of processing devices may be used, including, for example, processors available from manufacturers such as Intel®, AMD®, etc., and may include a variety of architectures (e.g., x86 processor, ARM®, etc.).

[0027]

[0074] In some embodiments, application processor 180 and / or image processor 190 may include any of the EyeQ series of processors available from Mobileye®. These processor designs include multiple processing units, each with a local memory and instruction set. Such processors may include video inputs that receive image data from multiple image sensors, and may also include video output capabilities. In one example, the EyeQ2® uses 90 nm-micron technology operating at 332 MHz. The EyeQ2® architecture consists of two floating-point hyper-threaded 32-bit RISC CPUs (MIPS32® 34K® cores), five Vision Computation Engines (VCEs), three Vector Microcode Processors (VMPs®), a Denali 64-bit mobile DDR controller, a 128-bit internal acoustic interconnect, dual 16-bit video input and 18-bit video output controllers, a 16-channel DMA, and several peripherals. The MIPS34K CPU manages the five VCEs, three VMPs™ and DMA, a second MIPS34K CPU and multi-channel DMA, and other peripherals. The five VCEs, three VMPs, and the MIPS34K CPU can perform intensive vision calculations required by feature-rich bundle applications. In another example, the disclosed embodiments may use the EyeQ3, a third generation processor that is six times more powerful than the EyeQ2. In another example, the EyeQ4 and / or EyeQ5 may be used with the disclosed embodiments. Of course, newer or future EyeQ processing devices may be used with the disclosed embodiments.

[0028]

[0075] Any of the processing devices disclosed herein can be configured to perform a particular function. Configuring a processing device, such as any of the described EyeQ processors or other controllers or microprocessors, to perform a particular function may include programming computer-executable instructions and providing the processing device for execution during operation of the processing device. In some embodiments, configuring a processing device may include directly programming architectural instructions into the processing device. In other embodiments, configuring a processing device may include storing executable instructions on a memory accessible by the processing device during operation. For example, the processing device may access the memory during operation to retrieve and execute the stored instructions. In any event, a processing device configured to perform the sensing, image analysis, and / or navigation functions disclosed herein represents a dedicated hardware-based system that controls multiple hardware-based components of a host vehicle.

[0029]

[0076] 1 shows two separate processing devices included in processing unit 110, more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to accomplish the tasks of application processor 180 and image processor 190. In other embodiments, these tasks may be performed by three or more processing devices. Furthermore, in some embodiments, system 100 may include one or more of processing units 110 without including other components, such as image acquisition unit 120.

[0030]

[0077] The processing unit 110 may include various types of devices. For example, the processing unit 110 may include various devices such as a controller, an image preprocessor, a central processing unit (CPU), support circuits, a digital signal processor, an integrated circuit, a memory, or any other type of device that processes and analyzes images. The image preprocessor may include a video processor that captures, digitizes, and processes images from an image sensor. The CPU may include any number of microcontrollers or microprocessors. The support circuits may be any number of circuits commonly known in the art, including cache, power, clock, and input / output circuits. The memory may store software that, when executed by the processor, controls the operation of the system. The memory may include databases and image processing software. The memory may include any number of random access memories, read-only memories, flash memories, disk drives, optical storage devices, tape storage devices, removable storage devices, and other types of storage devices. In one example, the memory may be separate from the processing unit 110. In another example, the memory may be integrated into the processing unit 110.

[0031]

[0078] Each memory 140, 150 may contain software instructions that, when executed by a processor (e.g., application processor 180 and / or image processor 190), may control the operation of various aspects of system 100. These memory units may include various databases and image processing software, as well as trained systems, such as neural networks or deep neural networks. The memory units may include random access memory, read-only memory, flash memory, disk drives, optical storage devices, tape storage devices, removable storage devices, and / or any other type of storage device. In some embodiments, memory units 140, 150 may be separate from application processor 180 and / or image processor 190. In other embodiments, these memory units may be integrated into application processor 180 and / or image processor 190.

[0032]

[0079] Position sensor 130 may include any type of device suitable for determining a location associated with at least one component of system 100. In some embodiments, position sensor 130 may include a GPS receiver. Such a receiver may determine the location and velocity of a user by processing signals broadcast by Global Positioning System satellites. Position information from position sensor 130 may be provided to application processor 180 and / or image processor 190.

[0033]

[0080] In some embodiments, system 100 may include components such as a speed sensor (e.g., a speedometer) for measuring the speed of vehicle 200. System 100 may also include one or more accelerometers (single-axis or multi-axis) for measuring the acceleration of vehicle 200 along one or more axes.

[0034]

[0081] The memory units 140, 150 may contain a database or any other organized format of data indicating the locations of known landmarks. Sensor information of the environment (images, radar signals, depth information from lidar or stereo processing of two or more images, etc.) can be processed together with location information such as GPS coordinates and vehicle ego motion to determine the vehicle's current location relative to known landmarks and to refine the vehicle's location. Certain aspects of this technology are included in a location technology known as REM™, sold by the assignee of the present application.

[0035]

[0082] User interface 170 may include any device suitable for providing information or receiving input from one or more users of system 100. In some embodiments, user interface 170 may include user input devices including, for example, a touch screen, a microphone, a keyboard, a pointer device, a track wheel, a camera, a knob, buttons, etc. Using such input devices, a user may be able to provide information input or commands to system 100 by typing instructions or information, providing voice commands, selecting on-screen menu options using buttons, pointers, or eye tracking capabilities, or through any other suitable technique for communicating information to system 100.

[0036]

[0083] User interface 170 may comprise one or more processing devices configured to provide information to or receive information from a user and process that information, e.g., for use by application processor 180. In some embodiments, such processing devices may execute instructions to recognize and track eye movements, receive and interpret voice commands, recognize and interpret touches and / or gestures made on a touchscreen, respond to keyboard entries or menu selections, etc. In some embodiments, user interface 170 may include a display, a speaker, a haptic device, and / or any other device that provides output information to a user.

[0037]

[0084] Map database 160 may include any type of database that stores map data useful to system 100. In some embodiments, map database 160 may include data related to the locations in a reference coordinate system of various items, including roads, water features, geographic features, businesses, points of interest, restaurants, gas stations, etc. Map database 160 may store not only the locations of such items, but also descriptors related to those items, including, for example, names associated with any of the stored features. In some embodiments, map database 160 may be physically located with other components of system 100. Alternatively or additionally, map database 160 or portions thereof may be located remotely with respect to other components of system 100 (e.g., processing unit 110). In such embodiments, information from map database 160 may be downloaded to a network (e.g., via a cellular network and / or the Internet, etc.) via a wired or wireless data connection. In some cases, map database 160 may store a sparse data model that includes polynomial representations of specific road features (e.g., lane marks) or a target trajectory of the host vehicle. The map database 160 may also include stored representations of various recognized landmarks that may be used to determine or update the known position of the host vehicle relative to the target trajectory. The landmark representations may include data fields such as the landmark type and the landmark location, among other possible identifiers.

[0038]

[0085] Image capture devices 122, 124, and 126 may each include any type of device suitable for capturing at least one image from an environment. Furthermore, any number of image capture devices may be used to obtain images for input to the image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. Image capture devices 122, 124, and 126 are further described below with reference to Figures 2B-2E.

[0039]

[0086] One or more cameras (e.g., image capture devices 122, 124, and 126) may be part of a sensing block included on the vehicle. A variety of other sensors may be included in the sensing block, and any or all of the sensors may be utilized to create a sensed navigation state of the vehicle. In addition to cameras (forward facing, side facing, rear facing, etc.), other sensors may be included in the sensing block, such as radar, lidar, acoustic sensors, etc. Additionally, the sensing block may include one or more components configured to communicate, transmit, and receive information related to the vehicle's environment. For example, such components may include a wireless transceiver (RF, etc.) that may receive sensor-based information or any other type of information related to the host vehicle's environment from a source located remotely relative to the host vehicle. Such information may include sensor output information or related information received from a vehicle system other than the host vehicle. In some embodiments, such information may include information received from a remote computing device, a centralized server, etc. Additionally, the cameras may take many different configurations, such as a single camera unit, multiple cameras, a camera cluster, long FOV, short FOV, wide angle, fisheye, etc.

[0040]

[0087] System 100 or various components of system 100 may be incorporated into a variety of different platforms. In some embodiments, system 100 may be included in vehicle 200, as shown in FIG. 2A. For example, vehicle 200 may include processing unit 110 and any other components of system 100, as described above with respect to FIG. 1. In some embodiments, vehicle 200 may include only a single image capture device (e.g., a camera), while in other embodiments, such as those discussed in connection with FIGS. 2B-2E, multiple image capture devices may be used. For example, as shown in FIG. 2A, either of image capture devices 122 and 124 of vehicle 200 may be part of an ADAS (advanced driver assistance system) imaging set.

[0041]

[0088] The image capture device included in vehicle 200 as part of image acquisition unit 120 may be located in any suitable location. In some embodiments, as shown in Figures 2A-2E and 3A-3C, image capture device 122 may be located near the rearview mirror. This location may provide a line of sight similar to that of the driver of vehicle 200 and may assist in determining what the driver can and cannot see. Although image capture device 122 may be located anywhere near the rearview mirror, placing image capture device 122 on the driver's side of the mirror may further assist in capturing an image representative of the driver's field of view and / or line of sight.

[0042]

[0089] Other locations for the image capture devices of the image acquisition unit 120 may also be used. For example, the image capture device 124 may be located on or in the bumper of the vehicle 200. Such a location may be particularly suitable for an image capture device having a wide field of view. The line of sight of an image capture device located in the bumper may be different from the line of sight of the driver, and thus the bumper image capture device and the driver do not always see the same object. The image capture devices (e.g., image capture devices 122, 124, and 126) may also be located in other locations. For example, the image capture devices may be located on one or both side mirrors of the vehicle 200, on the roof of the vehicle 200, on the hood of the vehicle 200, on the trunk of the vehicle 200, on the sides of the vehicle 200, mounted on, positioned behind, or positioned in front of any window of the vehicle 200, mounted on or near the front and / or rear lights of the vehicle 200, etc.

[0043]

[0090] In addition to the image capture device, vehicle 200 may include various other components of system 100. For example, processing unit 110 may be integrated into the vehicle's engine control unit (ECU) or may be included in vehicle 200 separate from the ECU. Vehicle 200 may also be equipped with a position sensor 130, such as a GPS receiver, and vehicle 200 may also include a map database 160 and memory units 140 and 150.

[0044]

[0091] As described above, the wireless transceiver 172 may transmit and / or receive data via one or more networks (e.g., a cellular network, the Internet, etc.). For example, the wireless transceiver 172 may upload data collected by the system 100 to one or more servers and download data from one or more servers. Through the wireless transceiver 172, the system 100 may receive updates to the data stored in the map database 160, the memory 140, and / or the memory 150, for example, periodically or on demand. Similarly, the wireless transceiver 172 may upload any data from the system 100 (e.g., images captured by the image acquisition unit 120, data received by the position sensor 130, other sensors, or the vehicle control system, etc.) and / or any data processed by the processing unit 110 to one or more servers.

[0045]

[0092] System 100 may upload data to a server (e.g., the cloud) based on a privacy level setting. For example, system 100 may implement a privacy level setting that regulates or limits the types of data (including metadata) that may uniquely identify the vehicle and / or the driver / owner of the vehicle that are sent to the server. Such settings may be set by a user via wireless transceiver 172, may be initialized by factory default settings, or may be set by data received by wireless transceiver 172, for example.

[0046]

[0093] In some embodiments, system 100 may upload data according to a "high" privacy level, under which system 100 may transmit data (e.g., location information associated with a route, captured images, etc.) without any details about a particular vehicle and / or driver / owner. For example, when uploading data according to a "high" privacy level, system 100 may transmit data such as captured images and / or limited location information associated with a route that does not include the vehicle identification number (VIN) or the name of the driver or owner of the vehicle, but instead.

[0047]

[0094] Other privacy levels are contemplated. For example, system 100 may transmit data to the server according to a "medium" privacy level, which may include additional information not included under a "high" privacy level, such as the make and / or model of the vehicle and / or vehicle type (e.g., passenger car, sport utility vehicle, truck, etc.). In some embodiments, system 100 may upload data according to a "low" privacy level. Under a "low" privacy level setting, system 100 may upload and include sufficient data to uniquely identify a particular vehicle, owner / driver, and / or some or all of the route traveled by the vehicle. Such "low" privacy level data may include, for example, one or more of the VIN, driver / owner name, starting point of the vehicle prior to departure, intended destination of the vehicle, make and / or model of the vehicle, type of vehicle, etc.

[0048]

[0095] FIG 2A is a side view representation of an exemplary vehicle imaging system according to a disclosed embodiment. FIG 2B is a top view representation of the embodiment shown in FIG 2A. As shown in FIG 2B, the disclosed embodiment may show a vehicle 200 including a system 100 within the body having a first image capture device 122 positioned near a rearview mirror and / or near a driver of the vehicle 200, a second image capture device 124 positioned on or within a bumper area (e.g., one of the bumper areas 210) of the vehicle 200, and a processing unit 110.

[0049]

[0096] As shown in Figure 2C, both image capture devices 122 and 124 may be positioned near the rearview mirror and / or near the driver of vehicle 200. Additionally, while two image capture devices 122 and 124 are shown in Figures 2B and 2C, it should be understood that other embodiments may include three or more image capture devices. For example, in the embodiment shown in Figures 2D and 2E, a first image capture device 122, a second image capture device 124, and a third image capture device 126 are included in system 100 of vehicle 200.

[0050]

[0097] 2D, image capture device 122 may be positioned near a rearview mirror and / or near the driver of vehicle 200, and image capture devices 124 and 126 may be positioned on a bumper area (e.g., one of bumper areas 210) of vehicle 200. Also, as shown in FIG. 2E, image capture devices 122, 124, and 126 may be positioned near a rearview mirror and / or near the driver's seat of vehicle 200. The disclosed embodiments are not limited to any particular number and configuration of image capture devices, and image capture devices may be positioned in any suitable location within and / or on vehicle 200.

[0051]

[0098] It should be understood that the disclosed embodiments are not limited to vehicles and may be applicable in other contexts, and it should also be understood that the disclosed embodiments are not limited to a particular type of vehicle 200 and may be applicable to all types of vehicles, including cars, trucks, trailers, and other types of vehicles.

[0052]

[0099] The first image capture device 122 may include any suitable type of image capture device. The image capture device 122 may include an optical axis. In one example, the image capture device 122 may include an Aptina M9V024 WVGA sensor with a global shutter. In other embodiments, the image capture device 122 may provide a resolution of 1280×960 pixels and may include a rolling shutter. The image capture device 122 may include various optical elements. In some embodiments, one or more lenses may be included to provide, for example, a desired focal length and field of view for the image capture device. In some embodiments, a 6 mm lens or a 12 mm lens may be associated with the image capture device 122. In some embodiments, the image capture device 122 may be configured to capture an image having a desired field of view (FOV) 202, as shown in FIG. 2D. For example, image capture device 122 may be configured to have a regular FOV, such as in the range of 40 degrees to 56 degrees, including a 46 degree FOV, a 50 degree FOV, a 52 degree FOV, or degrees greater than 52 degrees. Alternatively, image capture device 122 may be configured to have a narrow FOV, such as in the range of 23 degrees to 40 degrees, such as a 28 degree FOV or a 36 degree FOV. In addition, image capture device 122 may be configured to have a wide FOV, such as in the range of 100 degrees to 180 degrees. In some embodiments, image capture device 122 may include a wide angle bumper camera or a bumper camera with up to 180 degrees FOV. In some embodiments, image capture device 122 may be a 7.2 Mpixel image capture device with an aspect ratio of about 2:1 (e.g., H×V=3800×1900 pixels) with a horizontal FOV of about 100 degrees. Such an image capture device may be used in place of a three-dimensional image capture device configuration. Due to large lens distortion, the vertical FOV of such an image capture device can be much less than 50 degrees in implementations in which the image capture device uses a radially symmetric lens, for example, such lenses are not radially symmetric, thereby allowing a vertical FOV greater than 50 degrees with a horizontal FOV of 100 degrees.

[0053]

[0100] The first image capture device 122 may acquire a plurality of first images of a scene associated with the vehicle 200. The plurality of first images may each be acquired as a series of image scan lines, which may be captured using a rolling shutter. Each scan line may include a plurality of pixels.

[0054]

[0101] The first image capture device 122 may have a scan rate associated with the acquisition of each of the first series of image scan lines. The scan rate may refer to the rate at which the image sensor can acquire image data associated with each pixel included in a particular scan line.

[0055]

[0102] Image capture devices 122, 124, and 126 may include any suitable type and number of image sensors, including, for example, CCD sensors or CMOS sensors. In one embodiment, a CMOS image sensor may be utilized with a rolling shutter whereby each pixel in a row is read one at a time, and the scanning of the rows proceeds row by row until the entire image frame is captured. In some embodiments, the rows may be captured sequentially from top to bottom for the frame.

[0056]

[0103] In some embodiments, one or more of the image capture devices disclosed herein (e.g., image capture devices 122, 124, and 126) may constitute a high-resolution imager and may have a resolution of greater than 5M pixels, greater than 7M pixels, greater than 10M pixels, or more.

[0057]

[0104] The use of a rolling shutter can result in pixels in different rows being exposed and captured at different times, which can cause skew and other image artifacts in the captured image frame. On the other hand, if the image capture device 122 is configured to operate with a global or synchronous shutter, all pixels can be exposed for the same amount of time during a common exposure period. As a result, image data in a frame collected from a system utilizing a global shutter represents a snapshot of the entire FOV (such as FOV 202) at a particular time. Conversely, when applying a rolling shutter, each row in a frame is exposed and data is captured at a different time. Thus, moving objects can appear distorted with an image capture device having a rolling shutter. This phenomenon is described in more detail below.

[0058]

[0105] The second image capture device 124 and the third image capture device 126 may be any type of image capture device. Like the first image capture device 122, each of the image capture devices 124 and 126 may include an optical axis. In one embodiment, each of the image capture devices 124 and 126 may include an Aptina M9V024 WVGA sensor with a global shutter. Alternatively, each of the image capture devices 124 and 126 may include a rolling shutter. Like the image capture device 122, the image capture devices 124 and 126 may be configured to include various lenses and optical elements. In some embodiments, the lenses associated with the image capture devices 124 and 126 may provide the same FOV (such as FOV 202) as the FOV associated with the image capture device 122, or a narrower FOV (such as FOVs 204 and 206). For example, image capture devices 124 and 126 may have an FOV of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less than 20 degrees.

[0059]

[0106] Image capture devices 124 and 126 may acquire multiple second and third images of a scene associated with vehicle 200. Each of the multiple second and third images may be acquired as a second and third series of image scan lines, which may be captured using a rolling shutter. Each scan line or row may have a number of pixels. Image capture devices 124 and 126 may have second and third scan rates associated with acquiring each image scan line included in the second and third series.

[0060]

[0107] Each image capture device 122, 124, and 126 may be positioned in any suitable location and in any suitable orientation relative to vehicle 200. The relative positions of image capture devices 122, 124, and 126 may be selected to assist in fusing together information obtained from the image capture devices. For example, in some embodiments, the FOV associated with image capture device 124 (FOV 204) may partially or completely overlap with the FOV associated with image capture device 122 (e.g., FOV 202) and the FOV associated with image capture device 126 (e.g., FOV 206).

[0061]

[0108] Image capture devices 122, 124, and 126 may be positioned on vehicle 200 at any suitable relative height. In one example, there may be a height difference between image capture devices 122, 124, and 126, which may provide sufficient parallax information to enable stereo analysis. For example, as shown in FIG. 2A, two image capture devices 122 and 124 are at different heights. There may also be a lateral displacement difference between image capture devices 122, 124, and 126, which may provide additional parallax information for stereo analysis by processing unit 110, for example. The lateral displacement difference may be denoted by dx, as shown in FIGS. 2C and 2D. In some embodiments, a forward or rearward displacement (e.g., range displacement) may exist between image capture devices 122, 124, and 126. For example, image capture device 122 may be positioned 0.5 to 2 meters or more behind image capture device 124 and / or image capture device 126. With this type of displacement, one of the image capture devices may be able to cover a potential blind spot of the other image capture device.

[0062]

[0109] Image capture device 122 may have any suitable resolution capability (e.g., the number of pixels associated with the image sensor), and the resolution of the image sensor associated with image capture device 122 may be higher, lower, or the same as the resolution of the image sensors associated with image capture devices 124 and 126. In some embodiments, the image sensors associated with image capture device 122 and / or image capture devices 124 and 126 may have a resolution of 640×480, 1024×768, 1280×960, or any other suitable resolution.

[0063]

[0110] The frame rate (e.g., the rate at which an image capture device acquires a set of pixel data for one image frame before moving on to capture pixel data associated with the next image frame) may be controllable. The frame rate associated with image capture device 122 may be higher, lower, or the same as the frame rate associated with image capture devices 124 and 126. The frame rates associated with image capture devices 122, 124, and 126 may depend on a variety of factors that may affect the timing of the frame rate. For example, one or more of image capture devices 122, 124, and 126 may include a selectable pixel delay period imposed before or after acquisition of image data associated with one or more pixels of an image sensor in image capture devices 122, 124, and / or 126. In general, image data corresponding to each pixel may be acquired according to the device's clock rate (e.g., one pixel per clock cycle). Additionally, in embodiments including a rolling shutter, one or more of image capture devices 122, 124, and 126 may include a selectable horizontal blanking period that is imposed before or after acquisition of image data associated with a row of pixels of an image sensor in image capture devices 122, 124, and / or 126. Additionally, one or more of image capture devices 122, 124, and / or 126 may include a selectable vertical blanking period that is imposed before or after acquisition of image data associated with an image frame of image capture devices 122, 124, and 126.

[0064]

[0111] These timing controls may enable the frame rates associated with image capture devices 122, 124, and 126 to be synchronized even if the line scan rate of each image capture device is different. Additionally, as discussed in more detail below, among other factors (e.g., image sensor resolution, maximum line scan rate, etc.), these selectable timing controls may enable the synchronization of image capture from areas where the FOV of image capture device 122 overlaps with one or more of the FOVs of image capture devices 124 and 126, even if the field of view of image capture device 122 differs from the FOV of image capture devices 124 and 126.

[0065]

[0112] Frame rate timing at image capture devices 122, 124, and 126 may depend on the resolution of the associated image sensor. For example, assuming both devices have similar line scan rates, if one device includes an image sensor with a resolution of 640×480 and the other device includes an image sensor with a resolution of 1280×960, capturing a frame of image data from the sensor with the higher resolution will require a longer period of time.

[0066]

[0113] Another factor that may affect the timing of image data acquisition at image capture devices 122, 124, and 126 is the maximum line scan rate. For example, acquisition of a row of image data from an image sensor included in image capture devices 122, 124, and 126 requires some minimum amount of time. Assuming that no pixel delay period is added, this minimum amount of time to acquire a row of image data will be related to the maximum line scan rate of a particular device. A device that offers a higher maximum line scan rate has the potential to provide a higher frame rate than a device that has a lower maximum line scan rate. In some embodiments, one or both of image capture devices 124 and 126 may have a maximum line scan rate that is higher than the maximum line scan rate associated with image capture device 122. In some embodiments, the maximum line scan rate of image capture devices 124 and / or 126 may be 1.25 times, 1.5 times, 1.75 times, or 2 times or more the maximum line scan rate of image capture device 122.

[0067]

[0114] In another embodiment, image capture devices 122, 124, and 126 may have the same maximum line scan rate, but image capture device 122 may operate at a scan rate that is equal to or less than its maximum scan rate. The system may be configured such that one or both of image capture devices 124 and 126 operate at a line scan rate that is equal to the line scan rate of image capture device 122. In other examples, the system may be configured such that the line scan rate of image capture devices 124 and / or 126 may be 1.25 times, 1.5 times, 1.75 times, or 2 times or more the line scan rate of image capture device 122.

[0068]

[0115] In some embodiments, image capture devices 122, 124, and 126 may be asymmetric. That is, the image capture devices may include cameras with different fields of view (FOV) and focal lengths. The fields of view of image capture devices 122, 124, and 126 may include any desired area of ​​the environment of vehicle 200, for example. In some embodiments, one or more of image capture devices 122, 124, and 126 may be configured to acquire image data from the environment in front of vehicle 200, the environment behind vehicle 200, the environment on either side of vehicle 200, or a combination thereof.

[0069]

[0116] Further, the focal length associated with each image capture device 122, 124, and / or 126 may be selectable (e.g., by inclusion of an appropriate lens, etc.) such that each device captures images of objects at a desired distance range from vehicle 200. For example, in some embodiments, image capture devices 122, 124, and 126 may capture images of close-by objects within a few meters of the vehicle. Image capture devices 122, 124, 126 may also be configured to capture images of objects at greater distances from the vehicle (e.g., 25 m, 50 m, 100 m, 150 m, or more). Further, the focal lengths of image capture devices 122, 124, and 126 may be selected such that one image capture device (e.g., image capture device 122) can capture images of objects relatively close to the vehicle (e.g., within 10 m or within 20 m), while the other image capture devices (e.g., image capture devices 124 and 126) can capture images of objects farther away from vehicle 200 (e.g., greater than 20 m, greater than 50 m, greater than 100 m, greater than 150 m, etc.).

[0070]

[0117] According to some embodiments, the FOV of one or more of image capture devices 122, 124, and 126 may have a wide angle. For example, it may be advantageous to have an FOV of 140 degrees, particularly for image capture devices 122, 124, and 126 that may be used to capture images of areas near vehicle 200. For example, image capture device 122 may be used to capture images of areas to the right or left of vehicle 200, and in such embodiments, it may be desirable for image capture device 122 to have a wide FOV (e.g., at least 140 degrees).

[0071]

[0118] The field of view associated with each of the image capture devices 122, 124, and 126 may depend on the respective focal lengths. For example, as the focal length increases, the corresponding field of view decreases.

[0072]

[0119] Image capture devices 122, 124, and 126 may be configured to have any suitable field of view. In one particular example, image capture device 122 may have a horizontal FOV of 46 degrees, image capture device 124 may have a horizontal FOV of 23 degrees, and image capture device 126 may have a horizontal FOV between 23 and 46 degrees. In another example, image capture device 122 may have a horizontal FOV of 52 degrees, image capture device 124 may have a horizontal FOV of 26 degrees, and image capture device 126 may have a horizontal FOV between 26 and 52 degrees. In some embodiments, the ratio of the FOV of image capture device 122 to the FOV of image capture device 124 and / or image capture device 126 may vary between 1.5 and 2.0. In other embodiments, the ratio may vary between 1.25 and 2.25.

[0073]

[0120] System 100 may be configured such that the field of view of image capture device 122 at least partially or completely overlaps the field of view of image capture device 124 and / or image capture device 126. In some embodiments, system 100 may be configured such that the fields of view of image capture devices 124 and 126, for example, fall within the field of view of image capture device 122 (e.g., are smaller than the field of view of image capture device 122) and share a common center with the field of view of image capture device 122. In other embodiments, image capture devices 122, 124, and 126 may capture adjacent FOVs or may have partially overlapping FOVs. In some embodiments, the fields of view of image capture devices 122, 124, and 126 may be aligned such that the center of image capture device 124 and / or 126, which has a narrower FOV, may be located in the bottom half of the field of view of device 122, which has a wider FOV.

[0074]

[0121] 2F is a diagrammatic representation of an exemplary vehicle control system according to disclosed embodiments. As shown in FIG. 2F, vehicle 200 may include a throttle system 220, a brake system 230, and a steering system 240. System 100 may provide inputs (e.g., control signals) to one or more of throttle system 220, brake system 230, and steering system 240 via one or more data links (e.g., any wired and / or wireless link or link that transmits data). For example, based on analysis of images acquired by image capture devices 122, 124, and / or 126, system 100 may provide control signals to one or more of throttle system 220, brake system 230, and steering system 240 to navigate vehicle 200 (e.g., by accelerating, turning, lane shifting, etc.). Additionally, system 100 may receive inputs indicative of operating conditions of vehicle 200 (e.g., speed, whether vehicle 200 is braking and / or turning, etc.) from one or more of throttle system 220, braking system 230, and steering system 24. Further details are provided below in conjunction with Figures 4-7.

[0075]

[0122] As shown in FIG. 3A , the vehicle 200 may also include a user interface 170 for interacting with a driver or passenger of the vehicle 200. For example, the user interface 170 in a vehicle application may include a touch screen 320, knobs 330, buttons 340, and a microphone 350. The driver or passenger of the vehicle 200 may also interact with the system 100 using a steering wheel (e.g., located on or near the steering column of the vehicle 200, including, for example, a turn signal handle), buttons (e.g., located on the steering wheel of the vehicle 200), and the like. In some embodiments, the microphone 350 may be positioned adjacent to the rearview mirror 310. Similarly, in some embodiments, the image capture device 122 may be located near the rearview mirror 310. In some embodiments, the user interface 170 may also include one or more speakers 360 (e.g., speakers of a vehicle audio system). For example, the system 100 may provide various notifications (e.g., alerts) via the speaker 360.

[0076]

[0123] 3B-3D are diagrams of an exemplary camera mount 370 configured to be positioned behind a rearview mirror (e.g., rearview mirror 310) and opposite a vehicle windshield, according to disclosed embodiments. As shown in FIG. 3B, camera mount 370 may include image capture devices 122, 124, and 126. Image capture devices 124 and 126 may be positioned behind glare shield 380, which may directly contact the windshield and may include a film and / or composition of anti-reflective materials. For example, glare shield 380 may be positioned to be aligned opposite a windshield having a matching slope. In some embodiments, each of image capture devices 122, 124, and 126 may be positioned behind glare shield 380, for example, as shown in FIG. 3D. The disclosed embodiments are not limited to any particular configuration of image capture devices 122, 124, and 126, camera mount 370, and glare shield 380. FIG. 3C is a view of the camera mount 370 shown in FIG. 3B from the front.

[0077]

[0124] As will be appreciated by one of ordinary skill in the art having the benefit of this disclosure, many variations and / or modifications may be made to the above disclosed embodiments. For example, not all components are essential to the operation of system 100. Furthermore, any component may be located in any suitable portion of system 100, and the components may be rearranged in various configurations while still providing the functionality of the disclosed embodiments. Thus, the above-described configurations are examples, and regardless of the above-described configuration, system 100 may provide a wide range of functionality for analyzing the surroundings of vehicle 200 and navigating vehicle 200 in response to the analysis.

[0078]

[0125] As discussed in more detail below, in accordance with various disclosed embodiments, system 100 may provide various features related to autonomous driving and / or driver assistance technologies. For example, system 100 may analyze image data, location data (e.g., GPS location information), map data, speed data, and / or data from sensors included in vehicle 200. System 100 may collect data for analysis, for example, from image capture unit 120, location sensor 130, and other sensors. Additionally, system 100 may analyze the collected data to identify whether vehicle 200 should take a particular action, and then automatically take the determined action without human intervention. For example, when vehicle 200 navigates without human participation, system 100 may automatically control braking, acceleration, and / or steering of vehicle 200 (e.g., by sending control signals to one or more of throttle system 220, braking system 230, and steering system 240). Additionally, system 100 may analyze the collected data and issue warnings and / or alerts to vehicle occupants based on the analysis of the collected data. Further details regarding various embodiments provided by system 100 are provided below.

[0079]

[0126] Forward-looking multi-imaging system

[0127] As described above, system 100 may provide driver assistance features using a multi-camera system. The multi-camera system may use one or more cameras facing toward the front of the vehicle. In other embodiments, the multi-camera system may include one or more cameras facing toward the side of the vehicle or toward the rear of the vehicle. In one embodiment, for example, system 100 may use a two-camera imaging system, where a first camera and a second camera (e.g., image capture devices 122 and 124) may be positioned at the front and / or side of the vehicle (e.g., vehicle 200). Other camera configurations are consistent with the disclosed embodiments, and the configurations disclosed herein are examples. For example, system 100 may include any number of camera configurations (e.g., one, two, three, four, five, six, seven, eight, etc.). Additionally, system 100 may include a "cluster" of cameras. For example, a cluster of cameras (including any suitable number of cameras, e.g., 1, 4, 8, etc.) can face forward relative to the vehicle, or can face in any other direction (e.g., rearward, sideways, diagonally, etc.). Thus, system 100 may include multiple clusters of cameras, with each cluster oriented in a particular direction to capture images from a particular region of the vehicle's environment.

[0080]

[0128] The first camera may have a field of view that is larger, smaller, or partially overlaps that of the second camera. Additionally, the first camera may be connected to a first image processor to perform monocular image analysis of the image provided by the first camera, and the second camera may be connected to a second image processor to perform monocular image analysis of the image provided by the second camera. The outputs (e.g., processed information) of the first and second image processors may be combined. In some embodiments, the second image processor may receive images from both the first and second cameras to perform stereo analysis. In another embodiment, the system 100 may use a three-camera imaging system, where each camera has a different field of view. Thus, such a system may make decisions based on information derived from objects at various distances both in front of and to the sides of the vehicle. References to monocular image analysis may refer to cases where image analysis is performed based on images captured from a single viewpoint (e.g., a single camera). Stereo image analysis may refer to cases where image analysis is performed based on two or more images captured with one or more changes in image capture parameters. For example, captured images suitable for performing stereoscopic image analysis may include images captured from two or more different positions, images captured from different fields of view, images captured using different focal lengths, images captured with parallax information, etc.

[0081]

[0129] For example, in one embodiment, system 100 may implement a three camera configuration using image capture devices 122-126. In such a configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees or other value selected from the range of about 20-45 degrees, etc.), image capture device 124 may provide a wide field of view (e.g., 150 degrees or other value selected from the range of about 100-180 degrees), and image capture device 126 may provide a medium field of view (e.g., 46 degrees or other value selected from the range of about 35-60 degrees). In some embodiments, image capture device 126 may operate as a main or primary camera. Image capture devices 122-126 may be positioned substantially side-by-side (e.g., 6 cm apart) behind rearview mirror 310. Additionally, in some embodiments, as described above, one or more of image capture devices 122-126 may be mounted behind glare shield 380 that is flush with the windshield of vehicle 200. Such a shield may operate to minimize the effect of any reflections from the interior of the vehicle on the image capture devices 122-126.

[0082]

[0130] 3B and 3C, the wide field of view camera (e.g., image capture device 124 in the example above) may be mounted lower than the narrow main field of view camera (e.g., image capture devices 122 and 126 in the example above). This configuration may provide a free line of sight from the wide field of view camera. To reduce reflections, the camera may be mounted near the windshield of vehicle 200 and may include a polarizer to attenuate reflected light.

[0083]

[0131] A three camera system may provide certain performance characteristics. For example, some embodiments may include the ability to verify the detection of an object by one camera based on the detection results from another camera. In the three camera configuration described above, processing unit 110 may include, for example, three processing devices (e.g., three EyeQ series processor chips as described above), each processing device dedicated to processing images captured by one or more of image capture devices 122-126.

[0084]

[0132] In a three-camera system, the first processing device may receive images from both the main camera and the narrow FOV camera and may perform vision processing of the narrow FOV camera to detect, for example, other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the first processing device may calculate pixel disparity between images from the main camera and the narrow camera to create a 3D reconstruction of the vehicle 200's environment. The first processing device may then combine the 3D reconstruction with 3D map data or 3D information calculated based on information from another camera.

[0085]

[0133] The second processing device may receive images from the main camera and perform vision processing to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the second processing device may calculate camera displacement, calculate pixel disparity between successive images based on the displacement, and create a 3D reconstruction (e.g., structure-from-motion) of the scene. The second processing device may send the structure-from-motion based on the 3D reconstruction to the first processing device and combine the structure-from-motion with the stereoscopic 3D image.

[0086]

[0134] The third processing device may receive images from the wide FOV camera and process the images to detect vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. The third processing device may further execute additional processing instructions to analyze the images and identify moving objects in the images, such as vehicles changing lanes, pedestrians, etc.

[0087]

[0135] In some embodiments, having image-based information streams captured and processed independently may provide an opportunity to provide redundancy in the system, such as by using a first image capture device and images processed therefrom to verify and / or supplement information obtained by capturing and processing image information with at least a second image capture device.

[0088]

[0136] In some embodiments, system 100 may use two image capture devices (e.g., image capture devices 122 and 124) in providing navigation assistance to vehicle 200, and may use a third image capture device (e.g., image capture device 126) to provide redundancy and verify analysis of data received from the other two image capture devices. For example, in such a configuration, image capture devices 122 and 124 may provide images for stereo analysis by system 100 for navigating vehicle 200, while image capture device 126 may provide images for monocular analysis by system 100 to provide redundancy and verification of information derived based on images captured from image capture devices 123 and / or image capture device 124. That is, image capture device 126 (and corresponding processing device) may be considered to provide a redundant subsystem that provides a check on the analysis derived from image capture devices 122 and 124 (e.g., to provide an automatic emergency braking (AEB) system). Additionally, in some embodiments, redundancy and validation of received data can be supplemented based on information received from one or more sensors (e.g., radar, lidar, acoustic sensors, information received from one or more transceivers outside the vehicle, etc.).

[0089]

[0137] Those skilled in the art will recognize that the above camera configurations, camera placements, camera numbers, camera locations, etc. are merely exemplary. These components, etc. described for an overall system, may be assembled and used in a variety of different configurations without departing from the scope of the disclosed embodiments. Further details regarding the use of multi-camera systems to provide driver assistance and / or autonomous vehicle functions follow below.

[0090]

[0138] 4 is an example functional block diagram of memory 140 and / or 150 that may be stored / programmed with instructions to perform one or more operations in accordance with the disclosed embodiments. Although reference is made below to memory 140, those skilled in the art will recognize that instructions may be stored in memory 140 and / or 150.

[0091]

[0139] As shown in FIG. 4, memory 140 may store monocular image analysis module 402, stereo image analysis module 404, velocity and acceleration module 406, and navigation response module 408. The disclosed embodiments are not limited to any particular configuration of memory 140. Furthermore, application processor 180 and / or image processor 190 may execute instructions stored in any of modules 402-408 included in memory 140. Those skilled in the art will appreciate that references to processing unit 110 in the following discussion may refer to application processor 180 and image processor 190 individually or collectively. Thus, any steps of the following processes may be performed by one or more processing devices.

[0092]

[0140] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis of a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensor information (e.g., information from radar) to perform the monocular image analysis. As described in connection with FIGS. 5A-5D below, the monocular image analysis module 402 may include instructions to detect a set of features in the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazards, and any other features associated with the vehicle's environment. Based on the analysis, the system 100 may cause one or more navigational responses in the vehicle 200 (e.g., via the processing unit 110), such as turns, lane shifts, and acceleration changes, as discussed below in connection with the navigation response module 408.

[0093]

[0141] In one embodiment, the monocular image analysis module 402 may store instructions (e.g., computer vision software) that, when executed by the processing unit 110, perform monocular image analysis of a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensor information (e.g., information from radar, lidar, etc.) to perform the monocular image analysis. As described below in connection with FIGS. 5A-5D, the monocular image analysis module 402 may include instructions for detecting a set of features in the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazards, and any other features associated with the vehicle's environment. Based on the analysis, the system 100 may cause one or more navigational responses in the vehicle 200, such as a turn, a lane shift, an acceleration change, etc., as discussed below in connection with determining a navigational response (e.g., by the processing unit 110).

[0094]

[0142] In one embodiment, stereo image analysis module 404 may store instructions (e.g., computer vision software) that, when executed by processing unit 110, perform stereo image analysis of a first and second set of images acquired by a combination of image capture devices selected from image capture devices 122, 124, and 126. In some embodiments, processing unit 110 may combine information from the first and second sets of images with additional sensor information (e.g., information from radar) to perform stereo image analysis. For example, stereo image analysis module 404 may include instructions to perform stereo image analysis based on a first set of images acquired by image capture device 124 and a second set of images acquired by image capture device 126. As described below in connection with FIG. 6, stereo image analysis module 404 may include instructions to detect a set of features in the first and second sets of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and hazards. Based on the analysis, processing unit 110 may cause one or more navigational responses in vehicle 200, such as turns, lane shifts, and acceleration changes, as described below in connection with navigation response module 408. Additionally, in some embodiments, stereo image analysis module 404 may implement techniques related to trained systems (such as neural networks or deep neural networks) or untrained systems.

[0095]

[0143] In one embodiment, the speed and acceleration module 406 may store software configured to analyze data received from one or more computational and electromechanical devices in the vehicle 200 configured to alter the speed and / or acceleration of the vehicle 200. For example, the processing unit 110 may execute instructions associated with the speed and acceleration module 406 to calculate a target speed of the vehicle 200 based on data derived from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include, for example, target position, speed, and / or acceleration, the position and / or speed of the vehicle 200 relative to nearby vehicles, pedestrians, or road objects, and position information of the vehicle 200 relative to lane markings on the road. Additionally, the processing unit 110 may calculate a target speed of the vehicle 200 based on sensor inputs (e.g., information from a radar) and inputs from other systems of the vehicle 200, such as the throttle system 220, the braking system 230, and / or the steering system 240 of the vehicle 200. Based on the calculated target speed, the processing unit 110 may send electronic signals to the throttle system 220, the braking system 230, and / or the steering system 240 of the vehicle 200 to trigger a change in speed and / or acceleration, for example, by physically reducing the brakes or reducing the accelerator of the vehicle 200.

[0096]

[0144] In one embodiment, the navigation response module 408 may store software executable by the processing unit 110 to determine a desired navigation response based on data derived from execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include position and speed information associated with nearby vehicles, pedestrians, and road objects, as well as target position information for the vehicle 200, and the like. Additionally, in some embodiments, the navigation response may be based (partially or fully) on map data, a predetermined position of the vehicle 200, and / or a relative speed or relative acceleration between the vehicle 200 and one or more objects detected from execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. The navigation response module 408 may also determine a desired navigation response based on sensor inputs (e.g., information from a radar) and inputs from other systems of the vehicle 200, such as the throttle system 220, the braking system 230, and the steering system 240 of the vehicle 200. Based on the desired navigational response, processing unit 110 may trigger the desired navigational response by sending electronic signals to throttle system 220, braking system 230, and steering system 240 of vehicle 200 to, for example, turn the steering wheel of vehicle 200 to achieve a predetermined angle of rotation. In some embodiments, processing unit 110 may use the output of navigational response module 408 (e.g., the desired navigational response) as an input to the execution of velocity and acceleration module 406 to calculate a speed change for vehicle 200.

[0097]

[0145] Additionally, any of the modules disclosed herein (e.g., modules 402, 404, and 406) may implement techniques related to trained systems (such as neural networks or deep neural networks) or untrained systems.

[0098]

[0146] FIG. 5A is a flow chart illustrating an example process 500A for generating one or more navigational responses based on monocular image analysis, according to disclosed embodiments. In step 510, processing unit 110 may receive a plurality of images via data interface 128 between processing unit 110 and image acquisition unit 120. For example, a camera (such as image capture device 122 having field of view 202) included in image acquisition unit 120 may capture a plurality of images of an area in front of vehicle 200 (or, for example, to the side or rear of the vehicle) and transmit them to processing unit 110 via a data connection (e.g., digital, wired, USB, wireless, Bluetooth, etc.). Processing unit 110 may execute monocular image analysis module 402 to analyze the plurality of images in step 520, as described in further detail below in connection with FIGS. 5B-5D. By performing the analysis, processing unit 110 may detect a set of features in the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, and traffic lights.

[0099]

[0147] Processing unit 110 may also execute monocular image analysis module 402 in step 520 to detect various road hazards, such as truck tire parts, fallen road signs, loose cargo, and small animals. Road hazards may vary in structure, shape, size, and color, making such hazards more difficult to detect. In some embodiments, processing unit 110 may execute monocular image analysis module 402 to perform multi-frame analysis on multiple images to detect road hazards. For example, processing unit 110 may estimate camera movement between consecutive image frames and calculate pixel disparity between frames to build a 3D map of the road. Processing unit 110 may then use the 3D map to detect the road surface and hazards present on the road surface.

[0100]

[0148] In step 530, processing unit 110 may execute navigation response module 408 to cause vehicle 200 to make one or more navigation responses based on the analysis performed in step 520 and the techniques described above in connection with FIG. 4. The navigation responses may include, for example, a turn, a lane shift, an acceleration change, and the like. In some embodiments, processing unit 110 may use data derived from execution of speed and acceleration module 406 to cause one or more navigation responses. Furthermore, multiple navigation responses may occur simultaneously, sequentially, or any combination thereof. For example, processing unit 110 may cause vehicle 200 to move one lane and then accelerate, for example, by sequentially sending control signals to steering system 240 and throttle system 220 of vehicle 200. Alternatively, processing unit 110 may cause vehicle 200 to brake and simultaneously shift lanes, for example, by simultaneously sending control signals to braking system 230 and steering system 240 of vehicle 200.

[0101]

[0149] FIG. 5B is a flow chart illustrating an example process 500B for detecting one or more vehicles and / or pedestrians in a set of images according to the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to perform process 500B. In step 540, processing unit 110 may identify a set of candidate objects representing possible vehicles and / or pedestrians. For example, processing unit 110 may scan one or more images, compare the images to one or more predefined patterns, and identify locations within each image that may contain a target object (e.g., a vehicle, a pedestrian, or a portion thereof). The predefined patterns may be specified to achieve a low rate of "false hits" and a low rate of "misses." For example, processing unit 110 may use a low similarity threshold to the predefined patterns to identify candidate objects as possible vehicles or pedestrians. By doing so, processing unit 110 may be able to reduce the probability of missing (e.g., not identifying) a candidate object representing a vehicle or pedestrian.

[0102]

[0150] At step 542, processing unit 110 may filter the set of candidate objects to eliminate certain candidates (e.g., irrelevant or less relevant objects) based on classification criteria. Such criteria may be derived from various characteristics associated with object types stored in a database (e.g., a database stored in memory 140). The characteristics may include the object's shape, size, texture, and location (e.g., relative to vehicle 200), etc. Thus, processing unit 110 may use one or more sets of criteria to reject false candidates from the set of candidate objects.

[0103]

[0151] At step 544, processing unit 110 may analyze multiple image frames to identify whether objects in the set of candidate images represent vehicles and / or pedestrians. For example, processing unit 110 may track detected candidate objects across successive frames and accumulate per-frame data associated with the detected objects (e.g., size, position relative to vehicle 200, etc.). Additionally, processing unit 110 may estimate parameters of the detected objects and compare the per-frame position data of the objects to predicted positions.

[0104]

[0152] In step 546, processing unit 110 may construct a set of measurements of the detected objects. Such measurements may include, for example, position, velocity, and acceleration values ​​(relative to vehicle 200) associated with the detected objects. In some embodiments, processing unit 110 may construct the measurements based on estimation techniques using a series of time-based observations, such as a Kalman filter or linear quadratic estimation (LQE), and / or modeling data available for different object types (e.g., cars, trucks, pedestrians, bicycles, road signs, etc.). A Kalman filter may be based on a measurement of the scale of the object, where the scale measurement is proportional to the time to impact (e.g., the amount of time it takes vehicle 200 to reach the object). Thus, by performing steps 540-546, processing unit 110 may identify vehicles and pedestrians appearing in the set of captured images and derive information (e.g., position, velocity, size) associated with the vehicles and pedestrians. Based on the identification and derived information, processing unit 110 may cause one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.

[0105]

[0153] In step 548, processing unit 110 may perform optical flow analysis of one or more images to reduce the probability of detecting "false hits" and the probability of missing candidate objects representing vehicles or pedestrians. Optical flow analysis may refer to, for example, analyzing movement patterns, separate from road surface movement, for vehicle 200 in one or more images associated with other vehicles and pedestrians. Processing unit 110 may calculate the movement of candidate objects by observing different positions of the object across multiple image frames captured at different times. Processing unit 110 may use the position and time values ​​as inputs to a mathematical model to calculate the movement of candidate objects. Thus, optical flow analysis may provide another way to detect vehicles and pedestrians in the vicinity of vehicle 200. Processing unit 110 may perform optical flow analysis in combination with steps 540-546 to provide redundancy in detecting vehicles and pedestrians and to increase the reliability of system 100.

[0106]

[0154] FIG. 5C is a flow chart illustrating an example process 500C for detecting road marks and / or lane geometry information in a set of images according to the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to perform process 500C. In step 550, processing unit 110 may detect a set of objects by scanning one or more images. To detect lane mark segments, lane geometry information, and other relevant road marks, processing unit 110 may filter the set of objects to exclude those determined to be irrelevant (e.g., small holes, small rocks, etc.). In step 552, processing unit 110 may group together the segments detected in step 550 that belong to the same road mark or lane mark. Based on the grouping, processing unit 110 may develop a model, such as a mathematical model, to represent the detected segments.

[0107]

[0155] In step 554, processing unit 110 may construct a set of measurements associated with the detected segment. In some embodiments, processing unit 110 may create a projection of the detected segment from the image plane to the real-world plane. The projection may be characterized using a third-order polynomial with coefficients corresponding to physical properties such as position, slope, curvature, and curvature derivatives of the detected road. In generating the projection, processing unit 110 may take into account road surface changes and pitch and roll rates associated with vehicle 200. In addition, processing unit 110 may model the road height by analyzing the position and motion cues present on the road surface. Furthermore, processing unit 110 may estimate pitch and roll rates associated with vehicle 200 by tracking a set of feature points in one or more images.

[0108]

[0156] In step 556, processing unit 110 may perform a multi-frame analysis, for example, by tracking the detected segment across successive image frames and accumulating frame-by-frame data associated with the detected segment. When processing unit 110 performs a multi-frame analysis, the set of measurements constructed in step 554 may become more reliable and may be associated with an increasingly higher degree of confidence. Thus, by performing steps 550-556, processing unit 110 may identify road marks appearing in the set of captured images and derive lane geometry information. Based on the identification and derived information, processing unit 110 may cause one or more navigational responses in vehicle 200, as described above in connection with FIG. 5A.

[0109]

[0157] In step 558, processing unit 110 may further develop a safety model of vehicle 200 in the vehicle's surroundings, taking into account additional information sources. Processing unit 110 may use the safety model to define situations in which system 100 may safely execute autonomous control of vehicle 200. To develop the safety model, in some embodiments, processing unit 110 may consider the positions and movements of other vehicles, detected road edges and barriers, and / or general road shape descriptions extracted from map data (such as data from map database 160). By taking into account additional information sources, processing unit 110 may provide redundancy in detecting road marks and lane geometry and increase the reliability of system 100.

[0110]

[0158] FIG. 5D is a flow chart illustrating an example process 500D for detecting traffic lights in a set of images according to disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to perform process 500D. In step 560, processing unit 110 may scan the set of images and identify objects that appear at locations in the images that are likely to contain traffic lights. For example, processing unit 110 may filter the identified objects to build a set of candidate objects that excludes objects that are unlikely to correspond to traffic lights. The filtering may be based on various characteristics associated with traffic lights, such as shape, size, texture, and location (e.g., relative to vehicle 200). Such characteristics may be based on many examples of traffic lights and traffic control signals and may be stored in a database. In some embodiments, processing unit 110 may perform a multi-frame analysis on the set of candidate objects reflecting possible traffic lights. For example, processing unit 110 may track the candidate objects across consecutive image frames, estimate the real-world positions of the candidate objects, and filter out objects that are moving (and therefore unlikely to be traffic lights). In some embodiments, processing unit 110 may perform color analysis on the candidate objects to identify the relative location of detected colors appearing within potential traffic lights.

[0111]

[0159] In step 562, processing unit 110 may analyze the geometry of the intersection. The analysis may be based on any combination of (i) the number of lanes detected on either side of vehicle 200, (ii) marks (such as arrow marks) detected on the road, and (iii) a description of the intersection extracted from map data (such as data from map database 160). Processing unit 110 may perform the analysis using information derived from execution of monocular analysis module 402. In addition, processing unit 110 may identify a correspondence between traffic lights detected in step 560 and lanes appearing in the vicinity of vehicle 200.

[0112]

[0160] As vehicle 200 approaches the intersection, in step 564, processing unit 110 may update a confidence associated with the analyzed intersection geometry and the detected traffic lights. For example, the number of traffic lights estimated to appear at the intersection compared to the number that actually appear at the intersection may affect the confidence. Thus, based on the confidence, processing unit 110 may delegate control to the driver of vehicle 200 to improve safety conditions. By performing steps 560-564, processing unit 110 may identify traffic lights appearing in the set of captured images and analyze the intersection geometry information. Based on the identification and analysis, processing unit 110 may cause one or more navigation responses in vehicle 200, as described above in connection with FIG. 5A.

[0113]

[0161] 5E is a flowchart of an example process 500E for generating one or more navigation responses in vehicle 200 based on a vehicle path, according to disclosed embodiments. In step 570, processing unit 110 may construct an initial vehicle path associated with vehicle 200. The vehicle path may be represented using a set of points represented by coordinates (x,y), with distances d between two points in the set of points being expressed as i may be in the range of 1 to 5 meters. In one embodiment, processing unit 110 may construct an initial vehicle path using two polynomials, such as left and right road polynomials. Processing unit 110 may calculate the geometric midpoint between the two polynomials and, if there is a predefined offset (offset 0 may correspond to driving in the center of the lane), offset each point in the resulting vehicle path by a predefined offset value (e.g., a smart lane offset). The offset may be in a direction perpendicular to the segment between any two points in the vehicle path. In another embodiment, processing unit 110 may use one polynomial and an estimated lane width to offset each point in the vehicle path by half the estimated lane width plus a predefined offset value (e.g., a smart lane offset).

[0114]

[0162] In step 572, processing unit 110 may update the vehicle path constructed in step 570. Processing unit 110 may update the vehicle path by calculating the distance d k is the distance d i A higher resolution may be used to reconstruct the vehicle path constructed in 570 such that the distance d k may be in the range of 0.1 to 0.3 meters. Processing unit 110 may reconstruct the vehicle path using a parabolic spline algorithm, which may result in a cumulative distance vector S corresponding to the total length of the vehicle path (i.e., based on the set of points representing the vehicle path).

[0115]

[0163] In step 574, the processing unit 110 calculates the look-ahead point ((x l ,z l ) in coordinates. The processing unit 110 may extract look-ahead points from the cumulative distance vector S, and the look-ahead points may be associated with a look-ahead distance and a look-ahead time. The look-ahead distance may have a lower bound range of 10-20 meters and may be calculated as the product of the speed of the vehicle 200 and the look-ahead time. For example, as the speed of the vehicle 200 decreases, the look-ahead distance may also decrease (e.g., until a lower bound is reached). The look-ahead time, which may range from 0.5 to 1.5 seconds, may be inversely proportional to the gain of one or more control loops associated with producing a navigation response in the vehicle 200, such as a heading error tracking control loop. For example, the gain of the heading error tracking control loop may depend on the bandwidth of the yaw rate loop, the steering actuator loop, and the vehicle lateral dynamics, etc. Thus, the higher the gain of the heading error tracking control loop, the shorter the look-ahead time.

[0116]

[0164] In step 576, processing unit 110 may determine a heading error and yaw rate command based on the look-ahead point identified in step 574. Processing unit 110 may calculate the arctangent of the look-ahead point, e.g., arctan(x l / z l) to determine the heading error. Processing unit 110 may determine the yaw rate command as the product of the heading error and a high-level control gain. The high-level control gain may be equal to (2 / look ahead time) if the look ahead distance is not at a lower limit. If the look ahead distance is at a lower limit, the high-level control gain may be equal to (2×velocity of vehicle 200 / look ahead distance).

[0117]

[0165] 5F is a flow chart illustrating an example process 500F for determining whether a leading vehicle is changing lanes, according to a disclosed embodiment. In step 580, processing unit 110 may determine navigation information associated with a leading vehicle (e.g., a vehicle traveling in front of vehicle 200). For example, processing unit 110 may determine a position, a velocity (e.g., direction and speed), and / or an acceleration of the leading vehicle using the techniques described above in connection with FIGs. 5A and 5B. Processing unit 110 may also determine one or more road polynomials, lookahead points (associated with vehicle 200), and / or a snail trail (e.g., a set of points describing a path taken by a leading vehicle) using the techniques described above in connection with FIGs. 5E.

[0118]

[0166] In step 582, processing unit 110 may analyze the navigation information identified in step 580. In one embodiment, processing unit 110 may calculate a distance (e.g., along the trail) between the snail trail and the road polynomial. If the difference in this distance along the trail exceeds a predetermined threshold (e.g., 0.1-0.2 meters for straight roads, 0.3-0.4 meters for gently curving roads, and 0.5-0.6 meters for sharply curving roads), processing unit 110 may determine that the leading vehicle is likely changing lanes. If multiple vehicles are detected to be traveling in front of vehicle 200, processing unit 110 may compare the snail trails associated with each vehicle. Based on the comparison, processing unit 110 may determine that a vehicle whose snail trail does not match the snail trails of the other vehicles is likely changing lanes. Processing unit 110 may further compare the curvature of the snail trail (associated with the leading vehicle) to the expected curvature of the road segment along which the leading vehicle is traveling. The expected curvature may be extracted from map data (e.g., data from map database 160), road polynomials, snail trails of other vehicles, prior knowledge about the road, etc. If the difference between the snail trail curvature and the expected curvature of the road segment exceeds a predetermined threshold, processing unit 110 may determine that the leading vehicle is likely changing lanes.

[0119]

[0167] In another embodiment, processing unit 110 may compare the leading vehicle's instantaneous position to the look-ahead point (associated with vehicle 200) over a particular time period (e.g., 0.5-1.5 seconds). If the difference in distance and the cumulative sum of discrepancies between the leading vehicle's instantaneous position and the look-ahead point during the particular time period exceeds a predetermined threshold (e.g., 0.3-0.4 meters for straight roads, 0.7-0.8 meters for gently curving roads, and 1.3-1.7 meters for sharply curving roads), processing unit 110 may determine that the leading vehicle is likely changing lanes. In another embodiment, processing unit 110 may analyze the geometry of the snail trail by comparing the lateral distance traveled along the trail to the expected curvature of the snail trail. The expected radius of curvature is calculated as: (δ z 2 +δ x 2 ) / 2 / (δ x ) where σ x represents the lateral movement distance, and σ z represents the longitudinal travel distance. If the difference between the lateral travel distance and the expected curvature exceeds a predetermined threshold (e.g., 500-700 meters), processing unit 110 may determine that the leading vehicle is likely changing lanes. In another embodiment, processing unit 110 may analyze the position of the leading vehicle. If the position of the leading vehicle obscures the road polynomial (e.g., the leading vehicle overlaps the road polynomial), processing unit 110 may determine that the leading vehicle is likely changing lanes. If the position of the leading vehicle is such that another vehicle is detected ahead of the leading vehicle and the snail trails of the two vehicles are not parallel, processing unit 110 may determine that the (closer) leading vehicle is likely changing lanes.

[0120]

[0168] In step 584, processing unit 110 may determine whether leading vehicle 200 is changing lanes based on the analysis performed in step 582. For example, processing unit 110 may make that determination based on a weighted average of the individual analyses performed in step 582. Under such a scheme, for example, a determination by processing unit 110 that the leading vehicle is likely changing lanes based on a particular type of analysis may be assigned a value of "1" (with "0" representing a determination that the leading vehicle is unlikely to be changing lanes). Different weights may be assigned to different analyses performed in step 582, and the disclosed embodiments are not limited to any particular combination of analyses and weights. Furthermore, in some embodiments, the analysis may utilize a trained system (e.g., a machine learning or deep learning system), which may, for example, estimate a future path beyond the vehicle's current location based on images captured at the current location.

[0121]

[0169] 6 is a flow chart illustrating an example process 600 for generating one or more navigational responses based on stereo image analysis, according to disclosed embodiments. In step 610, processing unit 110 may receive a first and second plurality of images via data interface 128. For example, a camera included in image acquisition unit 120 (such as image capture devices 122 and 124 having fields of view 202 and 204) may capture and transmit a first and second plurality of images of an area in front of vehicle 200 to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first and second plurality of images via two or more data interfaces. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0122]

[0170] In step 620, processing unit 110 may execute stereo image analysis module 404 to perform stereo image analysis of the first and second plurality of images to create a 3D map of the road in front of the vehicle and detect features in the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and road hazards. The stereo image analysis may be performed similar to the steps described above in connection with FIGS. 5A-5D. For example, processing unit 110 may execute stereo image analysis module 404 to detect candidate objects (e.g., vehicles, pedestrians, road marks, traffic lights, road hazards, etc.) in the first and second plurality of images, filter out a subset of the candidate objects based on various criteria, perform a multi-frame analysis, construct measurements, and identify confidence levels for the remaining candidate objects. In performing the above steps, processing unit 110 may consider information from both the first and second plurality of images, rather than information from only one set of images. For example, processing unit 110 may analyze differences in pixel-level data (or other subsets of data from the two streams of captured images) of a candidate object that appears in both the first and second multiple images. As another example, processing unit 110 may estimate the position and / or velocity (e.g., relative to vehicle 200) of a candidate object by observing that an object appears in one of the multiple images but not in the other, or other differences that may exist for an object that appears in the two image streams. For example, the position, velocity, and / or acceleration relative to vehicle 200 may be determined based on the trajectory, location, movement characteristics, etc. of features associated with an object that appears in one or both of the image streams.

[0123]

[0171] In step 630, processing unit 110 may execute navigation response module 408 to cause one or more navigational responses in vehicle 200 based on the analysis performed in step 620 and the techniques described above in connection with FIG. 4. The navigational responses may include, for example, turns, lane shifts, acceleration changes, speed changes, braking, etc. In some embodiments, processing unit 110 may use data derived from execution of speed and acceleration module 406 to cause one or more navigational responses. Furthermore, multiple navigational responses may be performed simultaneously, sequentially, or any combination thereof.

[0124]

[0172] FIG. 7 is a flow chart illustrating an example process 700 for generating one or more navigation responses based on an analysis of three sets of images, according to disclosed embodiments. In step 710, processing unit 110 may receive a first, second, and third plurality of images via data interface 128. For example, cameras included in image acquisition unit 120 (such as image capture devices 122, 124, and 126 having fields of view 202, 204, and 206) may capture and transmit a first, second, and third plurality of images of an area in front of and / or to the side of vehicle 200 to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first, second, and third plurality of images via three or more data interfaces. For example, each of image capture devices 122, 124, and 126 may have an associated data interface that communicates data to processing unit 110. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0125]

[0173] In step 720, processing unit 110 may analyze the first, second, and third pluralities of images to detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and road hazards. The analysis may be performed similar to the steps described above in connection with FIGS. 5A-5D and 6. For example, processing unit 110 may perform monocular image analysis on each of the first, second, and third pluralities of images (e.g., via execution of monocular image analysis module 402 and based on the steps described above in connection with FIGS. 5A-5D). Alternatively, processing unit 110 may perform stereo image analysis on the first and second pluralities of images, the second and third pluralities of images, and / or the first and third pluralities of images (e.g., via execution of stereo image analysis module 404 and based on the steps described above in connection with FIG. 6). Processed information corresponding to the analysis of the first, second, and / or third pluralities of images may be combined. In some embodiments, processing unit 110 may perform a combination of monocular image analysis and stereo image analysis. For example, processing unit 110 may perform monocular image analysis on the first plurality of images (e.g., via execution of monocular image analysis module 402) and stereo image analysis on the second and third plurality of images (e.g., via execution of stereo image analysis module 404). The configuration of image capture devices 122, 124, and 126 (including their respective positions and fields of view 202, 204, and 206) may affect the type of analysis performed on the first, second, and third plurality of images. The disclosed embodiments are not limited to a particular configuration of image capture devices 122, 124, and 126 or the type of analysis performed on the first, second, and third plurality of images.

[0126]

[0174] In some embodiments, processing unit 110 may perform tests on system 100 based on the images acquired and analyzed in steps 710 and 720. Such tests may provide an indicator of the overall performance of system 100 with a particular configuration of image capture devices 122, 124, and 126. For example, processing unit 110 may identify the rate of "false hits" (e.g., when system 100 incorrectly determines the presence of a vehicle or pedestrian) and "misses."

[0127]

[0175] At step 730, processing unit 110 may cause one or more navigation responses in vehicle 200 based on information derived from two of the first, second, and third pluralities of images. The selection of two of the first, second, and third pluralities of images may depend on various factors, such as, for example, the number, type, and size of objects detected in each of the plural images. Processing unit 110 may make the selection based on the quality and resolution of the images, the effective field of view reflected in the images, the number of captured frames, the degree to which one or more objects of interest actually appear in the frames (e.g., the percentage of frames in which the object appears, the proportion in which the object appears in each such frame, etc.), etc.

[0128]

[0176] In some embodiments, processing unit 110 may select information derived from two of the first, second, and third pluralities of images by identifying the extent to which information derived from one image source is consistent with information derived from the other image source. For example, processing unit 110 may combine processed information derived from each of image capture devices 122, 124, and 126 (whether monocular analysis, stereo analysis, or any combination of the two) to identify visual indicators (e.g., lane markings, detected vehicles and / or their positions and / or paths, detected traffic lights, etc.) that are consistent across the images captured from each of image capture devices 122, 124, and 126. Processing unit 110 may also filter out information that is not consistent across the captured images (e.g., vehicles changing lanes, lane models showing vehicles too close to vehicle 200, etc.). Thus, processing unit 110 may select information derived from two of the first, second, and third pluralities of images based on identifying consistent and non-consistent information.

[0129]

[0177] The navigational responses may include, for example, turns, lane shifts, and acceleration changes. Processing unit 110 may generate one or more navigational responses based on the analysis performed in step 720 and the techniques described above in connection with FIG. 4. Processing unit 110 may also generate one or more navigational responses using data derived from execution of velocity and acceleration module 406. In some embodiments, processing unit 110 may generate one or more navigational responses based on a relative position, relative velocity, and / or relative acceleration between vehicle 200 and an object detected in any of the first, second, and third multiple images. The multiple navigational responses may be performed simultaneously, sequentially, or any combination thereof.

[0130]

[0178] Reinforcement learning and trained navigation systems

[0179] The following sections discuss autonomous driving, along with systems and methods for achieving autonomous control of a vehicle, whether the autonomous control of the vehicle is fully autonomous (self-driving vehicle) or partially autonomous (e.g., one or more drivers assist the system or function). As shown in FIG. 8, the autonomous driving task can be divided into three modules, including, for example, a sensing module 801, a driving policy module 803, and a control module 805. In some embodiments, the modules 801, 803, and 805 can be stored in the memory unit 140 and / or the memory unit 150 of the system 100, and / or the modules 801, 803, and 805 (or portions thereof) can be stored remotely from the system 100 (e.g., stored in a server accessible to the system 100, for example, via the wireless transceiver 172). Furthermore, any of the modules disclosed herein (e.g., modules 801, 803, and 805) can implement techniques related to trained systems (such as neural networks or deep neural networks) or untrained systems.

[0131]

[0180] The sensing module 801, which may be implemented using the processing unit 110, may handle various tasks related to detections of navigation conditions in the host vehicle's environment. Such tasks may depend on inputs from various sensors and sensing systems associated with the host vehicle. Those inputs may include images or image streams from one or more on-board cameras, GPS location information, accelerometer output, user feedback, user input to one or more user interface devices, radar, lidar, etc. Detections, which may include data from cameras and / or any other available sensors along with map information, may be collected, analyzed, and organized into a "sensing state" that describes information extracted from a scene in the host vehicle's environment. The sensing state may include sensing information related to target vehicles, lane markings, pedestrians, traffic lights, road geometry, lane shape, obstacles, distance to other objects / vehicles, relative speed, relative acceleration, among other possible sensing information. Supervised machine learning may be implemented to generate a sensing state output based on the sensing data provided to the sensing module 801. The output of the sensing module may represent the sensed navigation “state” of the host vehicle, which may be sent to the driving policy module 803.

[0132]

[0181] Although the sensed states may be created based on image data received from one or more cameras or image sensors associated with the host vehicle, the sensed states used for navigation may be created using any suitable sensor or combination of sensors. In some embodiments, the sensed states may be created without using captured image data. Indeed, any of the navigation principles described herein may be applicable to sensed states created based on captured image data as well as sensed states created using other non-image-based sensors. The sensed states may also be determined by sources external to the host vehicle. For example, the sensed states may be created in whole or in part based on information received from a source remote from the host vehicle (e.g., based on sensor information, processed state information, etc. shared from other vehicles, shared from a central server, or any other source of information related to the host vehicle's navigation state).

[0133]

[0182] The driving policy module 803, which will be described in more detail below and may be implemented using the processing unit 110, may implement a desired driving policy for determining one or more navigation actions to be taken by the host vehicle in response to sensed navigation conditions. If there are no other agents (e.g., target vehicles or pedestrians) in the host vehicle's environment, the sensed conditions input to the driving policy module 803 may be processed in a relatively straightforward manner. If the sensed conditions require negotiation with one or more other agents, the task becomes more complex. Techniques used to generate the output of the driving policy module 803 may include reinforcement learning (discussed in more detail below). The output of the driving policy module 803 may include at least one navigation action for the host vehicle, and may include, among other possible desired navigation actions, a desired acceleration (which may lead to an updated speed of the host vehicle), a desired yaw rate of the host vehicle, and a desired trajectory.

[0134]

[0183] Based on the output from the driving policy module 803, the control module 805, which may also be implemented using the processing unit 110, may create control instructions for one or more actuators or controlled devices associated with the host vehicle. Such actuators and devices may include an accelerator, one or more steering controls, brakes, signal transmitters, displays, or any other actuators or devices that may be controlled as part of a navigation operation associated with the host vehicle. Aspects of control theory may be used to generate the output of the control module 805. To implement the desired navigation goals or requirements of the driving policy module 803, the control module 805 may be responsible for creating and outputting instructions to the controllable components of the host vehicle.

[0135]

[0184] Returning to the driving policy module 803, in some embodiments, the driving policy module 803 can be implemented using a trained system trained by reinforcement learning. In other embodiments, the driving policy module 803 can be implemented without machine learning methods by "manually" dealing with various scenarios that may arise during autonomous navigation using specified algorithms. However, such an approach, while feasible, may result in a driving policy that is too simple and may lack the flexibility of a trained system based on machine learning. A trained system may be better equipped to handle complex navigation situations, be better able to determine whether a taxi is parked or has stopped to pick up or drop off passengers, be better able to determine whether a pedestrian is about to cross the road in front of the host vehicle, be better able to balance the unexpected behavior of other drivers against its own defensiveness, be better able to navigate busy roads containing target vehicles and / or pedestrians, be better able to determine when to suspend certain navigation rules or reinforce others, be better able to anticipate undetected but expected conditions (e.g., whether a pedestrian appears from behind a car or obstacle), etc. Trained systems based on reinforcement learning may be better equipped to deal with continuous and high-dimensional state spaces as well as continuous action spaces.

[0136]

[0185] Training the system using reinforcement learning may include learning a driving policy to map from sensed states to navigation actions. The driving policy is a function π:S→A, where S is a set of states,

number

[0137]

[0186] The system can be trained by exposing it to various navigation conditions, having the system apply a policy, and providing rewards (based on a reward function designed to reward desired navigation behavior). Based on the reward feedback, the system can "learn" the policy and become trained on producing the desired navigation behavior. For example, the learning system can learn the current state s t ∈S and set the policy

number

[0138]

[0187] The goal of reinforcement learning (RL) is to find a policy π. At time t, the state s t It is located in the operation a t A reward function r that measures the instantaneous quality of t However, at time t, action a tTaking an action affects the environment and therefore the value of future states. As a result, when deciding which action to take, one should not only consider the current reward but also future rewards. In some cases, if the system determines that more reward can be realized in the future by taking a choice that has a lower reward now, then it should take that action even if that particular action is associated with a lower reward than another choice available. To formalize this, suppose that a policy π and an initial state s are

number

[0188]

number

[0139]

[0189] Rather than restricting the horizon to T, we can discount future rewards and define the following for some fixed γ ∈ (0,1):

[0190]

number

[0140]

[0191] In any case, the optimal policy is

[0192]

number

[0193] where the expectation is over the initial state s.

[0141]

[0194] There are several possible methodologies for training a driving policy system. For example, the system can use imitation techniques (e.g., behavior cloning) that learn from state / action pairs, where the actions are selected by a good agent (e.g., a human) in response to specific observed states. Suppose a human driver is observed. The observations provide a basis for training the driving policy system, i.e., (s t ,a t ) format (s t is the state, and a t We can obtain, observe, and use many examples of π(s t )≒a t We can use supervised learning to learn the policy π such that ||π(s t )-a t It is often impossible to learn functions where || is very small. Moreover, even small errors can accumulate over time to produce large errors.

[0142]

[0195] Another technique that can be used is policy-based learning, where policies are expressed in parametric form and can be directly optimized using a suitable optimization technique (e.g., stochastic gradient descent).

number

[0143]

[0196] The system can also be trained by value-based learning (learning the Q-function or V-function). The optimal value function V * We assume that we can learn a good approximation for . An optimal policy can be constructed (e.g., by using the Bellman equation). Some versions of value-based learning can be performed offline (called "off-policy" training). Some disadvantages of value-based approaches can be attributed to their strong reliance on Markov assumptions and required approximations of complex functions (it can be harder to approximate a value function than to directly approximate a policy).

[0144]

[0197] Another technique may include model-based learning and planning (learning the probabilities of state transitions and solving an optimization problem to find the optimal V). A combination of these techniques can also be used to train a learning system. In this approach, the dynamics of the process, i.e. (s t ,a t ) and the next state s t+1 We can learn a function that gives a distribution over s. Once this function is learned, we can solve an optimization problem to find the policy π whose value is optimal. This is called "planning". One advantage of this approach is that the learning part is supervised, and we can t ,a t ,s t+1) can be applied offline by observing the learning process. As with the "imitation" approach, one disadvantage of this approach can be that small errors in the learning process can accumulate, resulting in a poorly performing policy.

[0145]

[0198] Another approach for training the driving policy module 803 may include decomposing the driving policy function into semantically significant components. Doing so allows for manual implementation of parts of the policy, which may guarantee the safety of the policy, and for other parts of the policy to be implemented using reinforcement learning methods, which may enable adaptability to many scenarios, human-like balance between defensive / aggressive behavior, and human-like negotiation with other drivers. From a technical point of view, reinforcement learning methods can combine several methodologies to provide a tractable training procedure, where most of the training can be performed using recorded data or a self-built simulator.

[0146]

[0199] In some embodiments, training of the driving policy module 803 can utilize an "option" mechanism. To illustrate this, consider a simple scenario of driving policy for a two-lane highway. In a direct RL approach, we define the state as

number

[0147]

[0200] Autonomous Cruise Control (ACC) policy, ACC :S→A: This policy always outputs a yaw rate of 0 and only changes speed to provide smooth, accident-free driving.

[0148]

[0201] ACC+Left Policy, o L:S → A: The longitudinal commands in this policy are the same as the ACC commands. The yaw rate is a simple implementation of centering the vehicle towards the center of the left lane while ensuring safe lateral movements (e.g., no moving left if there is a car on the left).

[0149]

[0202] ACC+Right Policy, o R :S→A:o L Same as, but the vehicle may be centered towards the center of the right lane.

[0150]

[0203] These policies can be called "alternatives". The policy that depends on these "alternatives" and selects the alternatives is o : We can learn S → O, where O is the set of available options. In one case, O = {o ACC ,o L ,o R} holds. For all s,

number

[0151]

[0204] In practice, a policy function can be decomposed into a choice graph 901 as shown in FIG. 9. Another example of a choice graph 1000 is shown in FIG. 10. A choice graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There is a special node called the root node 903 of the graph. This node has no input nodes. The decision process starts from the root node and traverses this graph until it reaches a "leaf" node that points to a node that has no output decision line. As shown in FIG. 9, the leaf nodes can include, for example, nodes 905, 907, and 909. When a leaf node is encountered, the driving policy module 803 can output acceleration and steering commands associated with the desired navigation behavior associated with the leaf node.

[0152]

[0205] For example, an interior node, such as nodes 911, 913, 915, etc., may implement a policy that selects a child from among its available choices. The set of available children of an interior node includes all of the nodes that are related to the particular interior node by decision lines. For example, interior node 913, shown in Figure 9 as "Merge," includes three child nodes 909, 915, and 917 ("Stay," "Overtake Right," and "Overtake Left," respectively) that are each connected to node 913 by decision lines.

[0153]

[0206] Allowing nodes to adjust their position in the hierarchy of the graph of alternatives allows flexibility in the decision-making system. For example, any of the nodes may be allowed to declare itself as "critical". Each node may implement a function "is critical" that outputs "true" if the node is within the critical section of its policy implementation. For example, the node responsible for the takeover may declare itself as critical during the operation. This may impose a constraint on the set of available children of node u, which may include all nodes v that are children of node u and for which there exists a path from v to a leaf node that passes through all nodes designated as critical. Such an approach may on the one hand allow the declaration of a desired path on the graph at each time step, while on the other hand may preserve the stability of the policy, especially in the case where the critical parts of the policy are being implemented.

[0154]

[0207] By defining a graph of options, the problem of learning a driving policy π:S→A can be decomposed into the problem of defining a policy for each node of the graph, where the policy at the internal node should be selected among the available child nodes. For some of the nodes, the individual policies can be implemented manually (e.g., by an if-then type algorithm that specifies a set of actions depending on the observed states), while for others, the policies can be implemented using a trained system built by reinforcement learning. The choice between a manual approach or a trained / learned approach can depend on the safety aspects related to the task and its relative simplicity. The graph of options can be constructed in such a way that some of the nodes are easily implemented, while other nodes can rely on trained models. Such an approach can ensure the safe operation of the system.

[0155]

[0208] The following discussion provides further details regarding the role of the choice graph of Figure 9 within the driving policy module 803. As discussed above, the input to the driving policy module is a "sensed state" outlining, for example, a map of the environment obtained from available sensors. The output of the driving policy module 803 is a set of "desires" (optionally with a set of hard constraints) that define a trajectory as a solution to an optimization problem.

[0156]

[0209] As mentioned above, the graph of options represents a hierarchical set of decisions organized as a DAG. There is a special node called the "root" of the graph. The root node is the only node that has no incoming edges (e.g., decision lines). The decision process starts from the root node and traverses the graph until a "leaf" node is reached, i.e., a node that has no outgoing edges. Each internal node should implement a policy to select one of its available children. Every leaf node should implement a policy that defines a set of desires (e.g., a set of navigational objectives for the host vehicle) based on the entire path from the root to the leaf. The set of desires, together with a set of hard constraints that are directly defined based on the sensed conditions, establish an optimization problem whose solution is the trajectory of the vehicle. The hard constraints can be used to further increase the safety of the system, and the desires can be used to provide driving comfort and human-like driving behavior of the system. The trajectory provided as a solution to the optimization problem thus defines the commands to be given to the steering, braking, and / or engine actuators to achieve the trajectory.

[0157]

[0210] Returning to FIG. 9 , the option graph 901 represents an option graph for a two-lane highway with a merging lane (meaning that at some point the third lane merges into the right or left lane of the highway). The root node 903 first determines if the host vehicle is in a simple road scenario or approaching a merging scenario. This is an example of a decision that can be made based on the sensing conditions. The simple road node 911 includes three child nodes: a stay node 909, a left overtaking node 917, and a right overtaking node 915. Staying refers to a situation where the host vehicle wants to continue traveling in the same lane. The stay node is a leaf node (no outgoing edges / lines). Thus, the stay node defines a set of wishes. The first wish defined by this node may include, for example, a desired lateral position as close as possible to the center of the current lane of travel. There may also be a wish to navigate smoothly (e.g., within a predefined or allowable maximum acceleration). The stay node may also define how the host vehicle reacts to other vehicles. For example, a stationary node can survey detected target vehicles and assign each a semantic meaning that can be translated into components of a trajectory.

[0158]

[0211] Various semantic meanings can be assigned to target vehicles in the host vehicle's environment. For example, in some embodiments, the semantic meanings can include any of the following indications: 1) Not involved: indicates that the detected vehicle in the scene is not currently involved; 2) Next lane: indicates that the detected vehicle is in an adjacent lane and should maintain an appropriate offset relative to that vehicle (the exact offset can be calculated in an optimization problem that constructs a trajectory given the desires and hard constraints, and can possibly be vehicle dependent, and the leaf of the choice graph that stays on sets the semantic type of the target vehicle that defines the desire for the target vehicle); 3) Give way: the host vehicle attempts to give way to the detected target vehicle (especially if the host vehicle determines that the target vehicle is likely to cut in the host vehicle's lane), for example by slowing down; 4) Lead: the host vehicle attempts to accept and comply with the right of way, for example by accelerating; 5) Follow: the host vehicle wishes to follow this target vehicle and maintain a smooth drive; 6) Overtake left / right: this means that the host vehicle wants to initiate a lane change to the left or right lane. Left overtaking node 917 and right overtaking node 915 are interior nodes that have not yet determined a wish.

[0159]

[0212] The next node in the options graph 901 is a gap selection node 919. This node may be responsible for selecting a gap between two target vehicles in a particular target lane that the host vehicle wishes to enter. By selecting a node of the form IDj, for some value of j, the host vehicle arrives at a leaf that specifies a desire for the trajectory optimization problem, e.g., a maneuver that the host vehicle wishes to perform to reach the selected gap. Such a maneuver may include first accelerating / braking in the current lane and proceeding to the target lane at a suitable time to enter the selected gap. If the gap selection node 919 cannot find a suitable gap, it proceeds to an abort node 921 that defines a desire to return to the center of the current lane and cancel the overtaking.

[0160]

[0213] Returning to the merging node 913, as the host vehicle approaches a merge, the host vehicle has several options that may depend on the particular situation. For example, as shown in FIG 11A, the host vehicle 1105 is traveling along a two-lane road without detecting another target vehicle in the main or merging lane 1111 of the two-lane road. In this situation, the driving policy module 803 may select the remain node 909 upon reaching the merging node 913. That is, if it does not detect a target vehicle as merging onto the road, it may be desirable to remain in one's current lane.

[0161]

[0214] 11B, the situation is slightly different, where the host vehicle 1105 detects one or more target vehicles 1107 entering the main road 1112 from the merging lane 1111. In this situation, when the driving policy module 803 faces the merging node 913, the driving policy module 803 may decide to initiate a left overtaking maneuver to avoid the merging situation.

[0162]

[0215] In FIG. 11C , the host vehicle 1105 encounters one or more target vehicles 1107 entering the main road 1112 from a merging lane 1111. The host vehicle 1105 also detects a target vehicle 1109 moving in a lane adjacent to the host vehicle's lane. The host vehicle also detects one or more target vehicles 1110 moving in the same lane as the host vehicle 1105. In this situation, the driving policy module 803 may decide to adjust the speed of the host vehicle 1105 to yield to the target vehicle 1107 and move ahead of the target vehicle 1115. This may be achieved, for example, by proceeding to the gap selection node 919, which selects the gap between ID0 (vehicle 1107) and ID1 (vehicle 1115) as the appropriate merging gap. The appropriate gap of the merging situation then defines the objective of the trajectory planner optimization problem.

[0163]

[0216] As discussed above, a node in the graph of options can declare itself as "critical", and such a declaration can ensure that the option selected passes through the critical node. Formally, each node can implement a function IsCritical. After making a forward pass from root to leaf on the graph of options and solving the trajectory planner optimization problem, a backward pass can be made from the leaf to the root. Along this backward pass, the IsCritical function of all nodes in the path can be called, and a list of all critical nodes can be saved. In the forward path corresponding to the next time slot, the driving policy module 803 can be requested to select a path from the root node to the leaf node that passes through all critical nodes.

[0164]

[0217] 11A-11C can be used to illustrate the possible benefits of this approach. For example, in a situation where an overtaking maneuver is initiated and the driving policy module 803 reaches the leaf corresponding to IDk, it is undesirable to select, for example, the node 909 to stay if the host vehicle is in the middle of the overtaking maneuver. To avoid this jumpiness, the IDj node can designate itself as critical. During the maneuver, the success of the trajectory planner can be monitored, and the function IsCritical returns a "true" value if the overtaking maneuver proceeds as intended. This approach can ensure that the overtaking maneuver continues in the next time frame (rather than jumping to another potentially inconsistent maneuver before completing the initially selected maneuver). On the other hand, if the monitoring of the maneuver indicates that the selected maneuver is not proceeding as intended or if the maneuver becomes unnecessary or impossible, the function IsCritical can return a "false" value. This can enable the gap selection node to select a different gap in the next time frame, or to abort the overtaking maneuver entirely. On the one hand, this approach may allow the declaration of a desired path through a graph of alternatives at each time step, while on the other hand, it may help promote stability of the policy in cases where it is in the critical parts of the execution.

[0165]

[0218] The strict constraints, which will be described in more detail below, can be distinguished from the navigation aspirations. For example, the strict constraints can ensure safe driving by applying an additional layer of filtering of the planned navigation operation. The involved strict constraints, which can be manually programmed and defined, can be determined from the sensed state, rather than by using a trained system built on reinforcement learning. However, in some embodiments, the trained system can learn the applicable strict constraints that will be applied and followed. Such an approach can encourage the driving policy module 803 to arrive at a selected operation that already complies with the applicable strict constraints, thereby reducing or eliminating selected operations that may require later modification to comply with the applicable strict constraints. Nevertheless, as a redundant safety measure, the strict constraints can be applied to the output of the driving policy module 803 even if the driving policy module 803 is trained to take into account the predetermined strict constraints.

[0166]

[0219] There are many examples of possible strict constraints. For example, a strict constraint can be defined in relation to guard rails at the edge of the road. Under no circumstances is the host vehicle allowed to cross the guard rails. Such a rule would cause a strict lateral constraint on the trajectory of the host vehicle. Another example of a strict constraint can include road bumps (e.g., speed control bumps), which may cause a strict constraint on the driving speed before or while crossing the bump. Strict constraints can be considered safety-critical and therefore can be defined manually, rather than relying exclusively on a trained system that learns the constraints during training.

[0167]

[0220] In contrast to strict constraints, the goal of aspirations may be to enable or achieve a comfortable driving experience. As discussed above, an example of aspiration may include a goal to position the host vehicle in a lateral position within the lane that corresponds to the center of the host vehicle's lane. Another aspiration may include the ID of a gap to move into. Note that the host vehicle does not need to be strictly in the center of the lane, and instead aspiration to be as close to the center of the lane as possible can ensure that the host vehicle is likely to move to the center of the lane if it deviates from the center of the lane. Aspirations may not be safety-critical. In some embodiments, aspirations may require negotiation with other drivers and pedestrians. One approach to constructing aspirations may utilize a graph of options, and the policies implemented within at least some nodes of the graph may be based on reinforcement learning.

[0168]

[0221] For nodes of the graph of choices 901 or 1000 that are implemented as nodes that are trained based on learning, the training process may include decomposing the problem into a supervised learning phase and a reinforcement learning phase. In the supervised learning phase:

number

number

number

number

[0169]

[0222] A key element that can be provided in some scenarios is a differentiable path from future losses / rewards back to a decision on action. In a choice graph structure, the implementation of choices including safety constraints is usually not differentiable. To overcome this issue, the selection of children at nodes of the learned policy can be probabilistic. That is, the node can output a probability vector p, which assigns a probability to be used in selecting each of the children of a particular node. Suppose a node has k children, a (1) ,...,a (k) Let be the action of the path from each child to the leaf. Thus, the resulting predicted action is

number

number

[0170]

[0223] s t , a t Given,

number

[0171]

[0224] Additionally, in some embodiments, the system may implement a multi-agent approach. For example, the system may consider data from various sources and / or imagery captured from multiple angles. Additionally, some disclosed embodiments may result in energy savings because the system may consider predictions of events that do not directly involve the host vehicle but may have an impact on the host vehicle, and may even consider predictions of events that may result in unpredictable situations involving other vehicles (e.g., radar may "see through" to unavoidable events that impact the preceding vehicle and the host vehicle, as well as predicting the high likelihood of such events).

[0172]

[0225] A trained system with imposed navigation constraints

[0226] In the context of autonomous driving, a critical concern is how to ensure that the learned policy of the trained navigation network is safe. In some embodiments, the driving policy system can be trained using constraints, so that the actions selected by the trained system may already take into account applicable safety constraints. In addition, in some embodiments, an additional layer of safety can be provided by passing the selected actions of the trained system through one or more strict constraints implicated by a particular sensed scene in the host vehicle's environment. Such an approach can ensure that the actions taken by the host vehicle are limited to those that are confirmed to satisfy the applicable safety constraints.

[0173]

[0227] At its core, the navigation system may include a learning algorithm based on a policy function that maps observed states to one or more desired actions. In some implementations, the learning algorithm is a deep learning algorithm. The desired actions may include at least one action expected to maximize the expected reward for the vehicle. In some cases, the actual action taken by the vehicle may correspond to one of the desired actions, while in other cases, the actual action taken may be determined based on the observed states, the one or more desired actions, and non-learning strict constraints (e.g., safety constraints) imposed on the learning navigation engine. These constraints may include non-driving zones surrounding various types of detected objects (e.g., target vehicles, pedestrians, static objects on or in the road, moving objects on or in the road, guard rails, etc.). In some cases, the size of the zone may vary based on the detected movement (e.g., speed and / or direction) of the detected object. Other constraints may include maximum travel speed when passing through pedestrian influence zones, maximum deceleration (to accommodate the spacing of target vehicles behind the host vehicle), mandatory stops at detected pedestrian crossings or railroad crossings, etc.

[0174]

[0228] Strict constraints used with a system trained by machine learning can provide a degree of safety in autonomous driving that may exceed the degree of safety that can be obtained based on the output of the trained system alone. For example, a machine learning system can be trained using a set of desired constraints as training guidelines, and thus the trained system can be configured with applicable navigation constraint restrictions and select actions that respect such restrictions in response to sensed navigation conditions. However, when selecting a navigation action, the trained system still has some flexibility, and thus there may be at least some situations in which the action selected by the trained system may not strictly follow the relevant navigation constraints. Thus, to require that the action selected strictly follows the relevant navigation constraints, non-machine learning components that ensure strict application of the relevant navigation constraints outside the learning / training framework can be used to combine, compare, filter, adjust, modify, etc. the output of the trained system.

[0175]

[0229] The following discussion provides further details on the trained system and the possible benefits (especially from a safety perspective) that can be obtained from combining the trained system with algorithmic components outside the training / learning framework. As discussed above, the reinforcement learning objective with the policy can be optimized by stochastic gradient ascent. The objective (e.g., expected reward) is

number

[0176]

[0230] In machine learning scenarios, objectives that include expectations can be used. However, such objectives may not return actions that are strictly bound by navigation constraints without being constrained by those constraints. For example, for a trajectory that represents a rare "turn in the road" event (e.g., an accident) that should be avoided,

number

number

number

number

[0177]

[0231]

number

number

number

number

[0178]

[0232] Lemma: π o Let p be a policy, p and r be scalars, so that with probability p,

number

number

[0233]

number

[0234] holds, and the last approximation applies when r≧1 / p.

[0179]

[0235] This explanation is in the format

number

number

number

number

number

number

number

number

[0180]

[0236] The double merge navigation situation shown in FIG. 11D provides an example that further illustrates these concepts. In a double merge, vehicles approach the merge area 1130 from both the left and right sides. From each side, a decision can be made whether a vehicle, such as vehicle 1133 or vehicle 1135, will merge into the opposite lane of the merge area 1130. In congested traffic, successfully executing a double merge may require significant negotiation skill and experience, and may be difficult to perform in a heuristic or brute force manner by enumerating all possible trajectories that all agents in the scene may take. In this double merge example, a set of desires D suitable for the double merge operation can be defined. D is the Cartesian product of the following sets: D=[0,v max ]xLx{g,t,o} where [0,v max

[0046] is the desired target speed of the host vehicle, L={1,1.5,2,2.5,3,3.5,4} is the desired lateral position in lane units, where integers indicate the center of the lane and fractions indicate the lane boundaries, and {g,t,o} are classification labels assigned to each of the other n vehicles. The other vehicles can be assigned "g" if the host vehicle should yield to them, "t" if the host vehicle should gain the right of way relative to them, or "o" if the host vehicle should maintain an offset distance relative to them.

[0181]

[0237] Below is a set of wishes (v, l, c1, ..., c n ) ∈ D can be transformed into a cost function over driving trajectories. k ,y k ), and (x i ,y i ) is the position (lateral, longitudinal) of the host vehicle (in egocentric units) at time τ·i. In some experiments, τ=0.1 seconds and k=10. Of course, other values ​​can be chosen. The cost assigned to a trajectory may include a weighted sum of individual costs assigned to the desired speed, lateral position, and labels assigned to each of the other n vehicles.

[0182]

[0238] The desired velocity v ∈ [0, v max ], the cost of the orbit relative to the velocity is

number

[0183]

[0239] Given a desired lateral position l ∈ L, the cost associated with the desired lateral position is

number

[0240] where dist(x,y,l) is the distance from point (x,y) to lane position l. Regarding the costs incurred by other vehicles, for any other vehicle, (x'1,y'1),...,(x' k ,y' k ) can represent other vehicles in the egocentric unit of the host vehicle, and i can be expressed as (x i ,y i ) and (x' j ,y' j ) is small. If there is no such point, i can be set as i=∞. If another vehicle is classified as "giving way", it may be desirable for τi>τj+0.5, meaning that the host vehicle reaches that same point at least 0.5 seconds after the other vehicle reaches the trajectory intersection. A possible formula for converting the above constraint to a cost is [τ(ji)+0.5]+.

[0184]

[0241] Similarly, if another car is classified as “taking the road”, it may be desirable for τj > τi + 0.5, which means that the cost [τ(ij) + 0.5] + If another vehicle is classified as "offset", it may be desirable for i = ∞, meaning that the host vehicle's trajectory and the offset vehicle's trajectory do not intersect. This condition can be translated into a cost by penalizing it with respect to the distance between the trajectories.

[0185]

[0242] Assigning weights to each of these costs yields a single objective function π (T) Costs that promote smooth driving can be added to the objective. Hard constraints can be added to the objective to ensure the functional safety of the track. For example, (x i ,y i ) may be prohibited from going off the road, (x i ,y i ) is the trajectory point (x' j ,y' j) for (x' j ,y' j ) may be prohibited from approaching.

[0186]

[0243] In summary, a policy πθ can be decomposed into an agnostic mapping from states to a set of desires, and a mapping from desires to actual trajectories. The latter mapping is not based on learning and can be implemented by solving an optimization problem whose costs depend on the desires and whose hard constraints can guarantee the functional safety of the policy.

[0187]

[0244] The following exposition describes the mapping from an agnostic state to a set of desires. As explained above, to comply with functional safety, a system that relies solely on reinforcement learning must use reward-based learning.

number

[0188]

[0245] For various reasons, the decision can be further decomposed into semantically meaningful components. For example, the size of D may be large and even continuous. In the double-confluence scenario described above with respect to FIG. 11D, D=[0, v max ]xLx{g,t,o}n). In addition, the gradient estimator is

number

[0189]

[0246] Returning to the concept of a graph of options, FIG. 11E shows a graph of options that may represent the double-join scenario shown in FIG. 11D. As discussed above, a graph of options may represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There may be a special node in the graph called the "root" node 1140, which is the only node that has no incoming edges (e.g., decision lines). The decision process may start from the root node and traverse the graph until it reaches a "leaf" node, i.e., a node that has no outgoing edges. Each internal node may implement a policy function that chooses one child from among its available children. There may be a default mapping from the set of traversals on the graph of options to the set of desires D. In other words, a traversal on the graph of options may be automatically converted to a desire in D. Given a node v in the graph, a parameter vector θ v may specify a policy for selecting children of v. If θ is a set of all θ v If v is a node with θ v By traversing the graph from the root to the leaves, selecting child nodes using the policy defined by

number

[0190]

[0247] In the double merge option graph 1139 of FIG. 11E, the root node 1140 may first determine whether the host vehicle is in a merge area (e.g., area 1130 of FIG. 11D) or whether instead the host vehicle is approaching a merge area and needs to prepare for a possible merge. In either case, the host vehicle may need to decide whether to change lanes (e.g., to the left or right) or whether to stay in the current lane. If the host vehicle decides to change lanes, the host vehicle may need to continue and determine whether conditions are suitable to perform a lane change maneuver (e.g., at “Go” node 1142). If it is not possible to change lanes, the host vehicle may attempt to “push” toward the desired lane (e.g., at node 1144) by aiming to be on a lane mark. Alternatively, the host vehicle may choose to “stay” in the same lane (e.g., at node 1146). This process may determine the lateral position of the host vehicle in a natural way. For example:

[0191]

[0248] This may allow the desired lateral position to be determined in a natural way. For example, if the host vehicle changes lanes from lane 2 to lane 3, the "go" node may set the desired lateral position to 3, the "stay" node may set the desired lateral position to 2, and the "push" node may set the desired lateral position to 2.5. The host vehicle may then decide to keep the "same" speed (node ​​1148), to "accelerate" (node ​​1150), or to "slow down" (node ​​1152). The host vehicle may then enter a "chain" structure 1154 that looks at other vehicles and sets their semantic meanings to values ​​in the set {g,t,o}. This process may set aspirations for other vehicles. The parameters of all nodes in the chain may be shared (similar to recurrent neural networks).

[0192]

[0249] A possible benefit of the choice is the interpretability of the results. Another possible benefit is that we can exploit the decomposable structure of the set D, and thus choose the policy at each node from a small number of possibilities. In addition, the structure may allow us to reduce the variance of the policy gradient estimator.

[0193]

[0250] As discussed above, the length of an episode in a double merge scenario may be approximately T=250 steps. This value (or any other suitable value depending on the particular navigation scenario) may allow enough time to acknowledge the consequences of the host vehicle's actions (e.g., if the host vehicle decides to change lanes in preparation for a merge, the host vehicle acknowledges the benefit only after successfully completing the merge). On the other hand, driving dynamics require the host vehicle to make decisions at a fast enough frequency (e.g., 10 Hz in the above case).

[0194]

[0251] The graph of choices may allow the effective value of T to be reduced in at least two ways. First, given the high-level decisions, rewards can be defined for low-level decisions while taking into account shorter episodes. For example, if the host vehicle has already selected the "change lane" and "go" nodes, a policy for assigning semantic meaning to the vehicle may be learned by looking at 2-3 second episodes (meaning T becomes 20-30 instead of 250). Second, for high-level decisions (such as whether to change lanes or stay in the same lane), the host vehicle may not need to make a decision every 0.1 seconds. Instead, the host vehicle may be able to make decisions less frequently (e.g., every second) or may implement a "choice termination" function, where the gradient is calculated only after every choice termination. In either case, the effective value of T may be an order of magnitude smaller than its original value. Overall, the estimators at all nodes may rely on a value of T that is an order of magnitude smaller than the original 250 steps, which may immediately translate into smaller variance.

[0195]

[0252] As discussed above, strict constraints can promote safer driving and there can be several different types of constraints. For example, strict static constraints can be determined directly from sensed conditions. These can include speed bumps, speed limits, road curvatures, intersections, etc. in the host vehicle's environment, which can imply one or more constraints on the vehicle's speed, heading, acceleration, braking (deceleration), etc. Strict static constraints can also include semantic free space, where the host vehicle is prohibited, for example, from going outside the free space and from navigating too close to physical barriers. Strict static constraints can also limit (e.g., prohibit) maneuvers that do not comply with various aspects of the vehicle's kinematic motion, for example, strict static constraints can be used to prohibit maneuvers that could lead the host vehicle to roll over, skid, or otherwise lose control.

[0196]

[0253] Strict constraints can also be vehicle related. For example, a constraint can be used that requires the vehicle to keep at least 1 meter longitudinal distance to other vehicles and at least 0.5 meter lateral distance from other vehicles. Constraints can also be applied to avoid the host vehicle keeping on a collision course with one or more other vehicles. For example, time τ can be a measure of time based on a particular scene. The predicted trajectories of the host vehicle and one or more other vehicles from the current time to time τ can be considered. If the two trajectories intersect,

number

number

number

number

[0197]

[0254] The time τ to track the trajectory of the host vehicle and one or more other vehicles can vary, except that in an intersection scenario where speeds may be low, τ can be longer and can be determined such that the host vehicle enters and exits the intersection in less than τ seconds.

[0198]

[0255] Of course, applying strict constraints on the trajectories of the vehicles requires that their trajectories be predicted. For the host vehicle, predicting the trajectory may be relatively straightforward since the host vehicle generally already knows and actually plans its intended trajectory at any given time. For other vehicles, predicting their trajectories may be less straightforward. For other vehicles, the baseline calculations for determining the predicted trajectory may depend on the current speed and heading of the other vehicles, which may be determined, for example, based on an analysis of image streams captured by one or more cameras and / or other sensors (radar, lidar, acoustic, etc.) on board the host vehicle.

[0199]

[0256] However, there may be some exceptions that simplify the problem or at least provide more confidence in the predicted trajectory of another vehicle. For example, on structured roads where there are lane indications and there may be yielding rules, the trajectory of the other vehicle may be based at least in part on the position of the other vehicle relative to the lane and based on applicable yielding rules. Thus, in some situations, when there is an observed lane structure, vehicles in adjacent lanes may be assumed to adhere to lane boundaries. That is, the host vehicle may assume that the vehicle in the adjacent lane will stay in its lane unless there is observed evidence (e.g., signal lights, strong lateral movement, movement across lane boundaries) that indicates that the vehicle in the adjacent lane will cut into the host vehicle's lane.

[0200]

[0257] Other circumstances may also provide hints as to the expected trajectory of other vehicles: for example, at stop signs, traffic lights, roundabouts, etc., where the host vehicle may have the right-of-way, it can be assumed that other vehicles will respect that right-of-way, and therefore, unless there is evidence that the rule has been observed to be violated, it can be assumed that other vehicles will proceed along a trajectory that respects the right-of-way held by the host vehicle.

[0201]

[0258] Strict constraints may also be applied with respect to pedestrians in the host vehicle's environment. For example, a buffer distance for pedestrians may be established to prohibit the host vehicle from navigating any closer than a specified buffer distance to any observed pedestrian. The buffer distance for pedestrians may be any suitable distance. In some embodiments, the buffer distance may be at least one meter to an observed pedestrian.

[0202]

[0259] Similar to the vehicle situation, strict constraints can also be applied to the relative motion between a pedestrian and a host vehicle. For example, the pedestrian's trajectory (based on heading and speed) can be monitored against the host vehicle's predicted trajectory. Given a particular pedestrian's trajectory, for every point p on the trajectory, t(p) may represent the time it takes for the pedestrian to reach point p. To keep the required buffer distance of at least one meter from the pedestrian, t(p) must be greater than the time the host vehicle reaches point p (with a sufficient time difference so that the host vehicle passes in front of the pedestrian with a difference of at least one meter), or t(p) must be less than the time the host vehicle reaches point p (e.g., if the host vehicle brakes to yield to the pedestrian). Furthermore, in the latter example, a strict constraint may require that the host vehicle reaches point p at a time sufficiently slower than the pedestrian so that the host vehicle can pass behind the pedestrian and keep the required buffer distance of at least one meter. Of course, there may be exceptions to the pedestrian strict constraint. For example, if the host vehicle has the right-of-way or is moving very slowly and there is no observed evidence of pedestrians refusing to yield to the host vehicle or navigating toward the host vehicle, the strict pedestrian constraints can be relaxed (e.g., to a narrower buffer of at least 0.75 meters or 0.50 meters).

[0203]

[0260] In some cases, constraints may be relaxed if it is determined that not all can be satisfied. For example, in a situation where the road is too narrow to allow a desired distance (e.g., 0.5 meters) from both curves or from a curve and a parked vehicle, one or more of the constraints may be relaxed if there are mitigating reasons. For example, one may proceed slowly at 0.1 meters from a curve if there are no pedestrians (or other objects) on the sidewalk. In some embodiments, constraints may be relaxed if doing so improves the user experience. For example, to avoid potholes, constraints may be relaxed to allow the vehicle to navigate closer to lane edges, curves, or pedestrians than would normally be permitted. Furthermore, when determining which constraints to relax, in some embodiments, the one or more constraints that are decided to be relaxed are those that are deemed to have the least detrimental effect on safety. For example, constraints on how close a vehicle can travel to a curve or concrete barrier may be relaxed before relaxing constraints dealing with proximity to other vehicles. In some embodiments, pedestrian constraints may be the last to be relaxed, or may never be relaxed depending on the circumstances.

[0204]

[0261] FIG. 12 illustrates an example of a scene that may be captured and analyzed during navigation of a host vehicle. For example, the host vehicle may include a navigation system (e.g., system 100) as described above that may receive a plurality of images representing the host vehicle's environment from a camera (e.g., at least one of image capture device 122, image capture device 124, and image capture device 126) associated with the host vehicle. The scene illustrated in FIG. 12 is an example of one of the images that may be captured at time t from the host vehicle's environment moving in lane 1210 along predicted trajectory 1212. The navigation system may include at least one processing device (e.g., including any of the EyeQ processors or other devices described above) that is specifically programmed to receive a plurality of images and analyze the images to determine an action in response to the scene. Among other things, the at least one processing device may implement the sensing module 801, driving policy module 803, and control module 805 illustrated in FIG. 8. The sensing module 801 may be responsible for collecting and outputting image information collected from the camera and providing the information in the form of identified navigation states to the driving policy module 803, which may constitute a trained navigation system that has been trained by machine learning methods such as supervised learning and reinforcement learning. Based on the navigation state information provided by the sensing module 801 to the driving policy module 803, the driving policy module 803 may generate a desired navigation action to be performed by the host vehicle depending on the identified navigation states (e.g., by implementing the option graph approach described above).

[0205]

[0262] In some embodiments, at least one processing device, for example using the control module 805, may directly translate the desired navigation behavior into navigation commands. However, in other embodiments, hard constraints may be applied to test the desired navigation behavior provided by the driving policy module 803 against various predefined navigation constraints that may be implicated by the scene and the desired navigation behavior. For example, if the driving policy module 803 outputs a desired navigation behavior that causes the host vehicle to follow a trajectory 1212, that navigation behavior may be tested against one or more hard constraints related to various aspects of the host vehicle's environment. For example, the captured image 1201 may determine a curve 1213, a pedestrian 1215, a target vehicle 1217, and a static object (e.g., an upturned box) present in the scene. Each of these may be associated with one or more hard constraints. For example, the curve 1213 may be associated with a static constraint that prevents the host vehicle from navigating into or over the curve onto the sidewalk 1214. The curve 1213 may also be associated with a road barrier envelope that defines a predetermined distance (e.g., a buffer zone) extending from and along the curve (e.g., 0.1 meters, 0.25 meters, 0.5 meters, 1 meter, etc.) that defines a non-navigable area for the host vehicle. Of course, static constraints may be associated with other types of roadside boundaries (e.g., guard rails, concrete pillars, cones, pylons, any other type of roadside barrier).

[0206]

[0263] It should be noted that distance and ranging can be determined by any suitable method. For example, in some embodiments, distance information can be provided by an on-board radar and / or lidar system. Alternatively or additionally, distance information can be derived by analyzing one or more images captured from the host vehicle's environment. For example, the number of pixels of a recognized object represented in an image can be determined and compared to a known field of view and focal length geometry of the image capture device to determine scale and distance. For example, velocity and acceleration can be determined by observing the change in scale between objects from image to image over a known time interval. This analysis can indicate how fast an object is moving away from or approaching the host vehicle, as well as the direction of movement towards or away from the host vehicle. A traversal speed can be determined by analyzing the position change of the object's X coordinate from one image to another over a known time period.

[0207]

[0264] The pedestrian 1215 may be associated with a pedestrian envelope that defines a buffer zone 1216. In some cases, the imposed strict constraint may prohibit the host vehicle from navigating within a distance of 1 meter from the pedestrian 1215 (in any direction relative to the pedestrian). The pedestrian 1215 may also define the location of a pedestrian influence zone 1220. This influence zone may be associated with a constraint that limits the speed of the host vehicle within the influence zone. The influence zone may extend 5 meters, 10 meters, 20 meters, etc. from the pedestrian 1215. A different speed limit may be associated with each class of the influence zone. For example, within a zone between 1 meter and 5 meters from the pedestrian 1215, the host vehicle may be limited to a first speed (e.g., 10 mph, 20 mph, etc.) that may be less than the speed limit within the pedestrian influence zone that extends from 5 meters to 10 meters. Any class for the various stages of the influence zone may be used. In some embodiments, the first stage can be narrower than 1 meter to 5 meters and may span only 1 meter to 2 meters. In other embodiments, the first stage of the influence zone may extend from 1 meter (the boundary of the non-navigable zone around the pedestrian) to a distance of at least 10 meters. The second stage may then extend from 10 meters to at least about 20 meters. The second stage may be associated with a maximum travel speed of the host vehicle that exceeds the maximum travel speed associated with the first stage of the pedestrian influence zone.

[0208]

[0265] One or more static object constraints may also be implicated by the detected scene in the host vehicle's environment. For example, in image 1201, at least one processing device may detect a static object, such as a box 1219 in a road. The detected static object may include various objects, such as trees, poles, road signs, and / or objects in the road. One or more predefined navigation constraints may be associated with the detected static object. For example, such constraints may include a static object envelope that defines a buffer zone around the object within which navigation of the host vehicle may be prohibited. At least a portion of the buffer zone may extend a predetermined distance from an edge of the detected static object. For example, in the scene represented by image 1201, a buffer zone of at least 0.1 meters, 0.25 meters, 0.5 meters, or more may be associated with box 1219, such that the host vehicle passes to the right or left of the box by at least some distance (e.g., the distance of the buffer zone) to avoid a collision with the detected static object.

[0209]

[0266] The predefined hard constraints may also include one or more target vehicle constraints. For example, a target vehicle 1217 may be detected in the image 1201. One or more hard constraints may be used to ensure that the host vehicle does not collide with the target vehicle 1217. In some cases, the target vehicle envelope may be associated with a single buffer zone distance. For example, the buffer zone may be defined by a distance of one meter surrounding the target vehicle in all directions. The buffer zone may define an area extending at least one meter from the target vehicle within which the host vehicle is prohibited from navigating.

[0210]

[0267] However, the envelope surrounding the target vehicle 1217 need not be defined by a fixed buffer distance. In some cases, the predefined strict constraints associated with the target vehicle (or any other movable object detected in the host vehicle's environment) may depend on the orientation of the host vehicle relative to the detected target vehicle. For example, in some cases, the distance of the longitudinal buffer zone (e.g., extending from the target vehicle to the front or rear of the host vehicle, such as when the host vehicle is traveling toward the target vehicle) may be at least 1 meter. The distance of the lateral buffer zone (e.g., extending from the target vehicle to either side of the host vehicle, such as when the host vehicle is traveling in the same or opposite direction as the target vehicle, such that the side of the host vehicle passes directly beside the side of the target vehicle) may be at least 0.5 meter.

[0211]

[0268] As explained above, other constraints may come into play by detecting target vehicles or pedestrians in the host vehicle's environment. For example, the predicted trajectories of the host and target vehicles 1217 may be considered, and when the two trajectories intersect (e.g., at intersection 1230), a hard constraint may be:

number

[0212]

[0269] Other hard constraints may also be used. For example, in at least some cases, a maximum deceleration rate of the host vehicle may be used. This maximum deceleration rate may be determined based on the detected distance to a target vehicle following the host vehicle (e.g., using images collected from a rear-facing camera). Hard constraints may include mandatory stops at detected pedestrian or railroad crossings or other applicable constraints.

[0213]

[0270] If the analysis of the scene in the host vehicle's environment indicates that one or more predefined navigation constraints may be engaged, those constraints may be imposed on one or more planned navigation operations of the host vehicle. For example, if the analysis of the scene results in the driving policy module 803 returning a desired navigation operation, the desired navigation operation may be tested against the one or more engaged constraints. If the desired navigation operation is determined to violate any aspect of the engaged constraints (e.g., if the desired navigation operation brings the host vehicle within 0.7 meters of the pedestrian 1215 when a predefined strict constraint requires the host vehicle to stay at least 1.0 meters from the pedestrian 1215), at least one modification to the desired navigation operation may be made based on the one or more predefined navigation constraints. Adjusting the desired navigation operation in this manner may result in the actual navigation operation of the host vehicle complying with the constraints engaged by the particular scene detected in the host vehicle's environment.

[0214]

[0271] After determining the actual navigational behavior of the host vehicle, the navigational behavior can be implemented by causing adjustment of at least one navigation actuator of the host vehicle in response to the determined actual navigational behavior of the host vehicle, which may include at least one of a steering mechanism, a brake, or an accelerator of the host vehicle.

[0215]

[0272] Prioritized Constraints

[0273] As explained above, various strict constraints can be used with the navigation system to ensure safe operation of the host vehicle. The constraints can include, among others, a minimum safe driving distance to a pedestrian, target vehicle, road barrier, or detected object, a maximum travel speed when passing within the influence zone of a detected pedestrian, or a maximum deceleration rate of the host vehicle. These constraints can be imposed by a trained system that is trained based on machine learning (supervised, reinforced, or a combination thereof), but can also be useful in untrained systems (e.g., using algorithms that directly address expected situations that arise within the scene of the host vehicle's environment).

[0216]

[0274] In any case, there may be a hierarchy of constraints. In other words, some navigation constraints may take precedence over other navigation constraints. Thus, if a situation occurs where no navigation action is available that would satisfy all involved constraints, the navigation system may determine an available navigation action that will first realize the highest priority constraint. For example, the system may have the vehicle avoid pedestrians first, even if navigation to avoid pedestrians would cause a collision with another vehicle or object detected in the road. In another example, the system may have the vehicle drive up a curb to avoid pedestrians.

[0217]

[0275] FIG. 13 shows a flow chart illustrating an algorithm for implementing a hierarchy of involved constraints determined based on an analysis of a scene in the host vehicle's environment. For example, in step 1301, at least one processor associated with the navigation system (e.g., an EyeQ processor, etc.) may receive a plurality of images representing the host vehicle's environment from an on-board camera of the host vehicle. In step 1303, a navigation state associated with the host vehicle may be determined by analyzing the one or more images representing the scene in the host vehicle's environment. For example, the navigation state may indicate, among other characteristics of the scene, that the host vehicle is traveling along a two-lane road 1210 as shown in FIG. 12, that a target vehicle 1217 is proceeding through an intersection ahead of the host vehicle, that a pedestrian 1215 is waiting to cross the road along which the host vehicle is traveling, and that there is an object 1219 ahead of the host vehicle's lane.

[0218]

[0276] In step 1305, one or more navigation constraints implicated by the host vehicle's navigation state may be determined. For example, the at least one processing device may analyze a scene in the host vehicle's environment represented by one or more captured images and then determine one or more navigation constraints implicated by objects, vehicles, pedestrians, etc. recognized by image analysis of the captured images. In some embodiments, the at least one processing device may determine at least a first predefined navigation constraint and a second predefined navigation constraint implicated by the navigation state, where the first predefined navigation constraint may be different from the second predefined navigation constraint. For example, the first navigation constraint may relate to one or more target vehicles detected in the host vehicle's environment, and the second navigation constraint may relate to pedestrians detected in the host vehicle's environment.

[0219]

[0277] In step 1307, the at least one processing device may determine a priority associated with the constraints identified in step 1305. In the illustrated example, a second predefined navigation constraint related to pedestrians may have a higher priority than a first predefined navigation constraint related to the target vehicle. Although the priority associated with a navigation constraint may be determined or assigned based on various factors, in some embodiments, the priority of a navigation constraint may relate to its relative importance from a safety perspective. For example, while it may be important to follow or satisfy all implemented navigation constraints in as many situations as possible, some constraints may be associated with stronger safety risks than other constraints and may therefore be assigned a higher priority. For example, a navigation constraint that requires the host vehicle to keep at least one meter away from pedestrians may have a higher priority than a constraint that requires the host vehicle to keep at least one meter away from the target vehicle. This may be because a collision with a pedestrian may have more severe consequences than a collision with another vehicle. Similarly, maintaining spacing between the host vehicle and the target vehicle may have a higher priority than constraints requiring the host vehicle to avoid boxes in the road, to drive below a certain speed over speed bumps, or to expose occupants of the host vehicle to less than maximum acceleration levels.

[0220]

[0278] Although the driving policy module 803 is designed to maximize safety by satisfying navigation constraints implicated by a particular scene or navigation state, it may be physically impossible to satisfy all constraints implicated by a situation. In such a situation, the priority of each implicated constraint may be used to determine which of the implicated constraints should be satisfied first, as shown in step 1309. Continuing with the above example, in a situation where both the pedestrian gap constraint and the target vehicle gap constraint cannot be satisfied and only one of the constraints can be satisfied, the higher priority of the pedestrian gap constraint may result in the constraint being satisfied before attempting to maintain a gap to the target vehicle. Thus, in a normal situation, as shown in step 1311, if both the first predefined navigation constraint and the second predefined navigation constraint can be satisfied, the at least one processing device may determine a first navigation action for the host vehicle that satisfies both the first predefined navigation constraint and the second predefined navigation constraint based on the identified navigation state of the host vehicle. However, in other situations where all involved constraints cannot be satisfied, as shown in step 1313, if both the first predefined navigation constraint and the second predefined navigation constraint cannot be satisfied, the at least one processing device may determine a second navigation action for the host vehicle based on the identified navigation state that satisfies the second predefined navigation constraint (i.e., the constraint with a higher priority) but does not satisfy the first predefined navigation constraint (which has a lower priority than the second navigation constraint).

[0221]

[0279] Next, in step 1315, the at least one processing device may cause adjustment of at least one navigation actuator of the host vehicle in response to the determined first navigation operation or the determined second navigation operation for the host vehicle to implement the determined navigation operation for the host vehicle. As with the previous example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0222]

[0280] Relaxation of constraints

[0281] As discussed above, navigation constraints may be imposed for safety. Constraints may include, among others, a minimum safe driving distance to a pedestrian, a target vehicle, a road barrier, or a detected object, a maximum travel speed when passing within a detected pedestrian's influence zone, or a maximum deceleration rate of the host vehicle. These constraints may be imposed in a learning navigation system or a non-learning navigation system. In certain circumstances, these constraints may be relaxed. For example, if the host vehicle slows down or stops near a pedestrian and proceeds slowly to communicate its intent to pass by the pedestrian, the pedestrian's response may be detected from the acquired images. If the pedestrian's response is to remain still or cease moving (and / or eye contact with the pedestrian is detected), it may be understood that the pedestrian recognizes the navigation system's intent to pass by the pedestrian. In such circumstances, the system may relax one or more of the default constraints and enforce a less strict constraint (e.g., allowing the vehicle to navigate within 0.5 meters of the pedestrian rather than within the more strict 1 meter boundary).

[0223]

[0282] FIG. 14 shows a flow chart for implementing control of a host vehicle based on relaxing one or more navigation constraints. In step 1401, at least one processing device may receive a plurality of images representing an environment of the host vehicle from a camera associated with the host vehicle. Analyzing the images in step 1403 may enable identifying a navigation state associated with the host vehicle. In step 1405, the at least one processor may determine navigation constraints associated with the navigation state of the host vehicle. The navigation constraints may include a first predefined navigation constraint that is implicated by at least one aspect of the navigation state. In step 1407, analyzing the plurality of images may determine the presence of at least one navigation constraint relaxation factor.

[0224]

[0283] The navigation constraint mitigation factors may include any suitable indicator that one or more navigation constraints may be suspended, modified, or otherwise relaxed in at least one aspect. In some embodiments, the at least one navigation constraint mitigation factor may include a determination (based on image analysis) that the pedestrian's eyes are looking in the direction of the host vehicle. In that case, it can be more safely assumed that the pedestrian recognizes the host vehicle. As a result, there can be more confidence that the pedestrian will not engage in unexpected behavior that causes the pedestrian to move into the path of the host vehicle. Other constraint mitigation factors may also be used. For example, the at least one navigation constraint mitigation factor may include a pedestrian that is determined to be not moving (e.g., one that is estimated to be unlikely to enter the path of the host vehicle) or a pedestrian that is determined to be slowing in its movement. The navigation constraint mitigation factors may also include more complex behaviors, such as a pedestrian being determined to be not moving after the host vehicle stops, and then resuming movement. In such a situation, it can be assumed that the pedestrian understands that the host vehicle has the right-of-way, and the stopping pedestrian may indicate his or her intention to yield to the host vehicle. Other conditions that may cause one or more constraints to be relaxed include the type of curb (e.g., a low curb or a gentle slope may allow the distance constraint to be relaxed), the absence of pedestrians or other objects on the sidewalk, vehicles with their engines off having a relaxed distance, or conditions where pedestrians are facing away from and / or away from the area the host vehicle is traveling into.

[0225]

[0284] Upon identifying the presence of a navigation constraint relaxation factor (e.g., in step 1407), a second navigation constraint may be determined or created in response to detecting the constraint relaxation factor. This second navigation constraint may differ from the first navigation constraint and may include at least one relaxed characteristic compared to the first navigation constraint. The second navigation constraint may include a newly generated constraint based on the first constraint, the newly generated constraint including at least one modification that relaxes the first constraint in at least one respect. Alternatively, the second constraint may constitute a predefined constraint that is less strict than the first navigation constraint in at least one respect. In some embodiments, this second constraint may be reserved for use only in situations where a constraint relaxation factor is identified in the host vehicle's environment. Whether the second constraint is newly generated or selected completely or partially from a set of predefined constraints available, application of the second navigation constraint instead of the more strict first navigation constraint (which may be applied in the absence of detection of the relevant navigation constraint relaxation factor) may be referred to as constraint relaxation and may be implemented in step 1409.

[0226]

[0285] If at least one constraint relaxation factor is detected in step 1407 and at least one constraint is relaxed in step 1409, a navigation action for the host vehicle may be determined in step 1411. The navigation action for the host vehicle may be based on the identified navigation state and may satisfy the second navigation constraint. In step 1413, the navigation action may be implemented by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action.

[0227]

[0286] As discussed above, the use of navigation constraints and relaxed navigation constraints can be used with a trained (e.g., by machine learning) navigation system or an untrained navigation system (e.g., a system programmed to respond with a predetermined action in response to a particular navigation situation). When using a trained navigation system, the availability of relaxed navigation constraints for a particular navigation situation can represent a mode switch from the response of the trained system to the response of the untrained system. For example, the trained navigation network can determine an original navigation action for the host vehicle based on a first navigation constraint. However, the action taken by the vehicle can be different from the navigation action that satisfies the first navigation constraint. Rather, the action taken can satisfy a more relaxed second navigation constraint and can be an action made by an untrained system (e.g., in response to detecting a particular condition in the host vehicle's environment, such as the presence of a navigation constraint mitigating factor).

[0228]

[0287] There are many examples of navigation constraints that may be relaxed in response to detecting constraint relaxation factors within the host vehicle's environment. For example, if a default navigation constraint includes a buffer zone associated with a detected pedestrian, and at least a portion of the buffer zone extends a predetermined distance from the detected pedestrian, a relaxed navigation constraint (created anew, recalled from memory from a predetermined set, or generated as a relaxed version of an existing constraint) may include a different or modified buffer zone. For example, the different or modified buffer zone may have a shorter distance to the pedestrian than the original or unmodified buffer zone to the detected pedestrian. As a result, if an appropriate constraint relaxation factor is detected within the host vehicle's environment, the host vehicle may be permitted to navigate closer to the detected pedestrian, taking into account the relaxed constraint.

[0229]

[0288] Relaxed characteristics of navigation constraints may include a reduced width of a buffer zone associated with at least one pedestrian as described above, but may also include a reduced width of a buffer zone associated with a target vehicle, a detected object, a roadside barrier, or any other object detected in the host vehicle's environment.

[0230]

[0289] The at least one relaxed characteristic may also include other types of modifications in the navigation constraint characteristics. For example, the relaxed characteristic may include a speed increase associated with the at least one predefined navigation constraint. The relaxed characteristic may also include an increase in the maximum allowable deceleration / acceleration associated with the at least one predefined navigation constraint.

[0231]

[0290] As noted above, constraints may be relaxed in certain circumstances, while navigation constraints may be tightened in other circumstances. For example, in some circumstances, the navigation system may determine that conditions justify tightening the normal set of navigation constraints. Such tightening may include adding new constraints to the set of default constraints or adjusting one or more aspects of the default constraints. This addition or adjustment may result in more conservative navigation relative to the set of default constraints that are applicable under normal driving conditions. Conditions that may justify tightening of constraints may include sensor failure, adverse environmental conditions (rain, snow, fog, or other conditions associated with reduced visibility or reduced vehicle traction), etc.

[0232]

[0291] FIG. 15 shows a flowchart for implementing control of a host vehicle based on reinforcing one or more navigation constraints. In step 1501, at least one processing device may receive a plurality of images representing an environment of the host vehicle from a camera associated with the host vehicle. Analyzing the images in step 1503 may enable identifying a navigation state associated with the host vehicle. In step 1505, the at least one processor may determine navigation constraints associated with the navigation state of the host vehicle. The navigation constraints may include a first predefined navigation constraint that is implicated by at least one aspect of the navigation state. In step 1507, analyzing the plurality of images may determine the presence of at least one navigation constraint reinforcing factor.

[0233]

[0292] The navigation constraints involved may include any of the navigation constraints discussed above (e.g., with respect to FIG. 12) or any other suitable navigation constraints. The navigation constraints reinforcement factors may include any indicator that one or more navigation constraints may be supplemented / reinforced in at least one aspect. The supplementation or reinforcement of the navigation constraints may be done on a set-by-set basis (e.g., by adding a new navigation constraint to the set of predefined constraints) or on a constraint-by-constraint basis (e.g., modifying a particular constraint such that the modified constraint is more restrictive than the original, or adding a new constraint corresponding to the predefined constraint, where the new constraint is more restrictive in at least one aspect than the corresponding constraint). Additionally or alternatively, the supplementation or reinforcement of the navigation constraints may refer to selecting from among the set of predefined constraints based on a hierarchy. For example, a set of reinforced constraints may be provided for selection based on whether a navigation reinforcement factor is detected in the environment of or for the host vehicle. Under normal conditions, where no reinforcement factor is detected, the navigation constraints involved may be derived from constraints applicable to normal conditions. On the other hand, if one or more constraint tightening factors are detected, the constraints involved can be derived from tightened constraints that are generated or predefined for the one or more tightening factors, the tightened constraints being more restrictive in at least one aspect than the corresponding constraints applicable under normal conditions.

[0234]

[0293] In some embodiments, the at least one navigation constraint enhancer may include detecting (e.g., based on image analysis) the presence of ice, snow, or water on a road surface in the host vehicle's environment. This determination may be based on detecting, for example, areas of higher reflectivity than expected on a dry road (e.g., indicative of ice or water on the road), white areas on the road surface indicating the presence of snow, shadows on the road consistent with the presence of longitudinal grooves on the road (e.g., tire tracks in snow), water droplets or small flakes of ice / snow on the host vehicle's windshield, or any other suitable indicator of the presence of water or ice / snow on the road surface.

[0235]

[0294] At least one navigation constraint enforcement factor may also include detecting small grains on an exterior surface of the host vehicle's windshield. Such small grains may impair image quality of one or more image capture devices associated with the host vehicle. Although described with respect to the host vehicle's windshield in the context of a camera mounted behind the host vehicle's windshield, detecting small grains on other surfaces (e.g., on a camera lens or lens cover, a headlight lens, a rear window, a taillight lens, or any other surface of the host vehicle visible to (or detected by a sensor of) an image capture device associated with the host vehicle) may also indicate the presence of a navigation constraint enforcement factor.

[0236]

[0295] Navigation constraint enhancing factors may also be detected as characteristics of one or more image capture devices. For example, a detected degradation in image quality of one or more images captured by an image capture device (e.g., a camera) associated with the host vehicle may also constitute a navigation constraint enhancing factor. The degradation in image quality may be associated with a hardware failure or partial hardware failure associated with the image capture device or an assembly associated with the image capture device. Such degradation in image quality may also be caused by environmental conditions. For example, the presence of smoke, fog, rain, snow, etc. in the air surrounding the host vehicle may contribute to degradation in image quality with respect to roads, pedestrians, target vehicles, etc. that may be in the host vehicle's environment.

[0237]

[0296] Navigation constraint enforcement factors may also relate to other aspects of the host vehicle. For example, in some circumstances, a navigation constraint enforcement factor may include a detected failure or partial failure of a system or sensor associated with the host vehicle. Such enforcement factors may include, for example, detecting a failure or partial failure of a speed sensor, GPS receiver, accelerometer, camera, radar, lidar, brakes, tires, or any other system associated with the host vehicle that may negatively affect the host vehicle's ability to navigate in relation to navigation constraints related to the host vehicle's navigation state.

[0238]

[0297] If the presence of a navigation constraint enhancing factor is identified (e.g., in step 1507), a second navigation constraint may be determined or created in response to detecting the constraint enhancing factor. This second navigation constraint may differ from the first navigation constraint and may include at least one characteristic that is enhanced relative to the first navigation constraint. The second navigation constraint may be more restrictive than the first navigation constraint because detecting a constraint enhancing factor in the host vehicle's environment or associated with the host vehicle may indicate that at least one navigation capability of the host vehicle may be degraded compared to normal operating conditions. Such degraded capabilities may include reduced road traction (e.g., ice, snow, or water on the road, reduced tire pressure, etc.), impaired visibility (e.g., rain, snow, dust, smoke, fog, etc. that reduces captured image quality), impaired detection capabilities (e.g., sensor failure or partial failure, reduced sensor performance, etc.), or any other degradation in the host vehicle's ability to navigate in response to the detected navigation conditions.

[0239]

[0298] If at least one constraint strengthening factor is detected in step 1507 and at least one constraint is strengthened in step 1509, a navigation action for the host vehicle may be determined in step 1511. The navigation action for the host vehicle may be based on the identified navigation state and may satisfy the second navigation (i.e., strengthened) constraint. In step 1513, the navigation action may be implemented by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action.

[0240]

[0299] As discussed above, the use of navigation constraints and enhanced navigation constraints can be used with a trained (e.g., by machine learning) navigation system or an untrained navigation system (e.g., a system programmed to respond with a predetermined action in response to a particular navigation situation). When using a trained navigation system, the availability of an enhanced navigation constraint for a particular navigation situation can represent a mode switch from the response of the trained system to the response of the untrained system. For example, the trained navigation network can determine an original navigation action for the host vehicle based on a first navigation constraint. However, the action taken by the vehicle can be different from the navigation action that satisfies the first navigation constraint. Rather, the action taken can be an action made by an untrained system (e.g., in response to detecting a particular condition in the host vehicle's environment, such as the presence of a navigation constraint enhancer) that can satisfy an enhanced second navigation constraint.

[0241]

[0300] There are many examples of navigation constraints that may be generated, supplemented, or enhanced in response to detecting constraint enhancement factors within the host vehicle's environment. For example, if a default navigation constraint includes a buffer zone associated with a detected pedestrian, object, vehicle, etc., and at least a portion of the buffer zone extends a predetermined distance from the detected pedestrian / object / vehicle, an enhanced navigation constraint (created anew, recalled from a predetermined set in memory, or generated as an enhanced version of an existing constraint) may include a different or modified buffer zone. For example, the different or modified buffer zone may have a longer distance to the pedestrian / object / vehicle than the original or unmodified buffer zone to the detected pedestrian / object / vehicle. As a result, if an appropriate constraint enhancement factor is detected within or in relation to the host vehicle's environment, the host vehicle may be forced to take the enhanced constraint into account and navigate further away from the detected pedestrian / object / vehicle.

[0242]

[0301] The at least one enhanced characteristic may also include other types of modifications in the navigation constraint characteristics. For example, the enhanced characteristic may include a reduction in speed associated with the at least one predefined navigation constraint. The enhanced characteristic may also include a reduction in the maximum allowable deceleration / acceleration associated with the at least one predefined navigation constraint.

[0243]

[0302] Navigation based on long-term planning

[0303] In some embodiments, the disclosed navigation system may not only respond to detected navigational states in the host vehicle's environment, but may also determine one or more navigational actions based on long-term planning. For example, the system may consider the potential impact on future navigational states of one or more navigational actions available as options for navigating with respect to the detected navigational state. Considering the effect of available actions on future states may allow the navigation system to determine a navigational action based not only on the currently detected navigational state, but also on long-term planning. Navigation using long-term planning techniques may be particularly applicable when one or more reward functions are used by the navigation system as a technique for selecting a navigational action from among available options. Potential rewards may be analyzed with respect to available navigational actions that can be performed in response to the detected current navigational state of the host vehicle. However, potential rewards may also be analyzed in relation to actions that can be performed in response to future navigational states predicted to result from the available actions for the current navigational state. As a result, in some cases, the disclosed navigation system may select a navigational action in response to a detected navigational state, even if such a navigational action may not result in the highest reward among the available actions that can be performed in response to the current navigational state. This may be especially true when the system determines that the selected action may result in a future navigation state that triggers one or more potential navigation actions that provide higher rewards than either the selected action or, in some cases, any of the actions available for the current navigation state. This principle may be expressed more simply as taking a less advantageous action now in order to provide a higher reward option in the future. Thus, the disclosed navigation system, capable of long-term planning, may select a suboptimal short-term action when long-term predictions indicate that a short-term loss in reward may result in an increase in long-term reward.

[0244]

[0304] In general, autonomous driving applications may include a set of planning problems where a navigation system may determine an immediate action to optimize a long-term objective. For example, when a vehicle faces a merging situation at a roundabout, the navigation system may determine an immediate acceleration or braking command to begin navigation into the roundabout. The immediate action for the navigation state detected at the roundabout may include an acceleration or braking command depending on the detected state, while the long-term objective is to successfully merge, and the long-term effect of the selected command is the success / failure of the merge. The planning problem can be addressed by decomposing the problem into two phases. First, supervised learning can be applied to predict the near future based on the present (assuming that the predictors are differentiable with respect to the current representation). Second, a recurrent neural network can be used to model the complete trajectory of the agent, with unexplained factors modeled as (additive) input nodes. This may allow the solution to the long-term planning problem to be found using supervised learning methods and direct optimization on the recurrent neural network. Such an approach may also allow the learning of robust policies by incorporating adversarial elements to the environment.

[0245]

[0305] Two of the most fundamental elements of an autonomous driving system are sensing and planning. Sensing deals with finding a compact representation of the current state of the environment, while planning deals with deciding which action to take to optimize a future objective. Supervised machine learning methods are useful for solving the sensing problem. One can also use machine learning algorithm frameworks for the planning part, especially the Reinforcement Learning (RL) framework as described above.

[0246]

[0306] RL can be executed by a series of successive rounds. In round t, the planner (aka agent or driving policy module 803) creates a state s representing the agents and the environment. t Then, the planner selects action a t After performing the action, the agent receives an immediate reward.

number

number

number

[0247]

[0307] Supervised learning (SL) can be considered a special case of RL, where we choose s from some distribution over S. t is sampled, and the reward function is r t =-l(a t ,yt ), where l is the loss function, and the learning side can t is the (possibly noisy) value of the optimal action to take when admitting t Observe the values ​​of . There may be some differences between the model of general RL and the specific case of SL, and these differences may make the general RL problem more difficult.

[0248]

[0308] In some SL situations, the actions (or predictions) made by the learner may not have any effect on the environment. t+1 and a t and are independent. This can have two important implications. First, in SL, the samples (s1,y1),...,(s m ,y m ) in advance, and only then can we begin to search for a policy (or predictor) that has good accuracy for the samples. In contrast, in RL, t+1 Typically, depends on the action taken (and even on the previous state), which in turn depends on the policy used to generate the action. This ties the data generation process to the policy learning process. Second, in SL, the effect of a on the performance of π is small, since actions do not affect the environment. t The contribution of the selection of is local. t only affects the value of the immediate reward. In contrast, in RL, actions taken in round t can have long-term effects on the reward value in future rounds.

[0249]

[0309] In SL, the reward shape r t =-l(a t ,y t ) along with the "correct" solution y t Knowledge of a t It can provide complete knowledge of the payoffs for all possible options, which is tThis may allow for the computation of the derivative of the reward with respect to . In contrast, in RL, the "one-shot" value of the reward may be all that can be observed for a particular choice of action taken. This can be called "bandit" feedback. In an RL-based system, if only "bandit" feedback is available, the system may not always be able to know if the action taken was the best action to take, which is one of the most important reasons why "exploration" is necessary as part of long-term navigation planning.

[0250]

[0310] Many RL algorithms rely, at least in part, on a mathematically explicit model of Markov decision processes (MDPs). The Markov assumption is that s t and a t Given s t+1 The distribution of π is fully determined. This yields a closed form for the cumulative reward of a given policy in terms of its stationary distribution over the states of the MDP. The stationary distribution of the policy can be expressed as a solution to a linear programming problem. This yields two families of algorithms: 1) optimization with respect to the primal problem, which can be called policy search, and 2) optimization with respect to the dual problem, whose variables are called the value function Vπ. The value function determines the expected cumulative reward if the MDP starts at an initial state s, from which actions are selected according to π. The quantity involved is the state-action value function Q π (s,a), which determines the cumulative reward given starting in state s, an immediately chosen action a, and an action chosen from there according to π. This Q-function can give rise to a characterization of the optimal policy (using the Bellman equation). Specifically, this Q-function can be shown to be a deterministic function from S to A (indeed, the optimal policy can be characterized as a "greedy" policy with respect to the optimal Q-function).

[0251]

[0311] One possible advantage of MDP models is that they allow the future to be coupled to the present using a Q-function. For example, given that the host vehicle is currently in state s, Qπ The value of (s,a) may indicate the effect of performing action a on the future. Thus, the Q-function provides a local measure of the quality of action a, making RL problems more similar to SL scenarios.

[0252]

[0312] Many RL algorithms approximate the V-function or Q-function in some way. Value iteration algorithms, e.g., Q-learning algorithms, can exploit the fact that the V-function and Q-function of an optimal policy can be fixed points of some operators obtained from the Bellman equations. Actor-critic policy iteration algorithms aim to learn a policy in an iterative way, where at iteration t, the "critic" is

number

[0253]

[0313] Despite the mathematical simplicity of MDPs and the convenience of switching to a Q-function representation, this approach may have some limitations. For example, in some cases, an approximate notion of a Markovian behavior state may be all that can be found. Furthermore, state transitions may depend not only on the agent's behavior, but also on the behavior of other players in the environment. For example, in the ACC example above, the dynamics of the autonomous vehicle may be Markovian, but the next state may depend on the behavior of drivers of other cars, which is not necessarily Markovian. One possible solution to this problem is to use partially observable MDPs, where it is assumed that there are Markovian states, but what can be seen are observations that are distributed according to hidden states.

[0254]

[0314] A more straightforward approach could consider a game-theoretic generalization of MDPs (e.g., in the framework of stochastic games). Indeed, algorithms for MDPs can be generalized to multi-agent games (e.g., minimax Q-learning or Nash Q-learning). Other approaches could include explicit modeling of other players and vanishing regret learning algorithms. Learning in a multi-agent setting can be more complex than learning in a single-agent setting.

[0255]

[0315] A second limitation of the Q-function representation may arise from departing from the tabular setting. A tabular setting is when the number of states and actions is small, so that Q can be represented as a table with |S| rows and |A| columns. However, if the natural representation of S and A involves Euclidean space and the state and action spaces are discretized, the number of states / actions may be exponential in scale. In such cases, it may be impractical to adopt the tabular setting. Instead, the Q-function may be approximated by some function from a parametric hypothesis class (e.g., neural networks of a certain architecture). For example, the Deep Q-Network (DQN) learning algorithm may be used. In DQN, the state space may be continuous, but the action space may remain a small discrete set. Techniques for accommodating a continuous action space are possible, but they may rely on approximating the Q-function. In any case, the Q-function may be complex and sensitive to noise, and thus difficult to learn.

[0256]

[0316] A different approach could be to use recurrent neural networks (RNNs) to address RL problems. In some cases, RNNs can be combined with concepts of multi-agent games and robustness to adversarial environments from game theory. Furthermore, this approach could be one that does not explicitly rely on the Markov assumption.

[0257]

[0317] In the following, we describe in more detail an approach for navigation by prediction-based planning, in which the state space S is

number

number

[0258]

[0318] As a first step in this approach, it can be observed that there are interesting problems where the bandit nature of the reward does not matter. For example, reward values ​​for ACC applications (discussed in more detail below) can be differentiable with respect to the current state and action. In fact, even if the reward is given in a "bandit" fashion,

number

number

number

number

number

[0259]

[0319] Similar concepts can be used to address connections between the past and the future. For example,

number

number

number

number

number

number

number

[0260]

[0320] As discussed above, the learning system can benefit from robustness to hostile environments, such as the host vehicle's environment, which may include multiple other drivers who may behave in unexpected ways. t In a model that does not impose probabilistic assumptions on v t We can consider an environment where μ is chosen in an adversarial way. t We can place restrictions on ||μ, which would otherwise make the planning problem difficult or even impossible for an adversary. One natural constraint is that ||μ t It may be necessary to require that || be bounded by a constraint.

[0261]

[0321] Robustness to hostile environments can be useful in autonomous driving applications. tChoosing , can even speed up the learning process, since it can focus the learning system towards a robust optimal policy. A simple game can be used to illustrate this concept. The states are

number

number

[0262]

[0322] Such an approach can be applied to virtually all possible navigation situations. In the following, we describe the approach applied to one example: Adaptive Cruise Control (ACC). In an ACC problem, a host vehicle may attempt to keep a sufficient distance to a target vehicle ahead (e.g., 1.5 seconds to the target vehicle). Another goal may be to drive as smoothly as possible while maintaining a desired gap. A model to represent this situation can be defined as follows. The state space is:

number

number

number

number

[0263]

[0323] The complete dynamics of the system can be described by the following equation:

number

[0264]

[0324] This can be written as the sum of two vectors.

number

[0265]

[0325] The first vector is the predictable part and the second vector is the unpredictable part. The reward for round t is defined as:

number

[0266]

[0326] The first term may introduce a penalty for non-zero acceleration, thus encouraging smooth driving. The second term is the acceleration of the target car x t Distance to and desired distance

number

[0267]

[0327] Implementing the approaches outlined above, the host vehicle's navigation system may select an action in response to observed conditions (e.g., by operation of the driving policy module 803 in the navigation system's processing unit 110). The action selected may be based not only on an analysis of rewards associated with available responsive actions with respect to the sensed navigation condition, but also on consideration and analysis of future conditions, potential actions in response to the future conditions, and rewards associated with the potential actions.

[0268]

[0328] 16 illustrates an algorithmic approach to navigation based on detection and long-term planning. For example, in step 1601, at least one processing device 110 of a navigation system for a host vehicle may receive a plurality of images. These images may capture a scene representative of the host vehicle's environment and may be provided by any of the image capture devices (e.g., cameras, sensors, etc.) described above. Analyzing one or more of these images in step 1603 may enable the at least one processing device 110 to identify a current navigation state associated with the host vehicle (as described above).

[0269]

[0329] In steps 1605, 1607, and 1609, various potential navigation actions may be determined depending on the sensed navigation conditions (e.g., to complete a merge, to smoothly follow a leading vehicle, to overtake a target vehicle, to avoid objects in the road, to slow down for a detected stop sign, to avoid an incoming target vehicle, or to complete any other navigation action that may further the navigation goals of the system). These potential navigation actions (e.g., from the first navigation action to the Nth available navigation action) may be determined based on the sensed conditions and the long-term goals of the navigation system.

[0270]

[0330] For each potential navigational action determined, the system may determine an expected reward. The expected reward may be determined according to any of the techniques described above and may include analysis of the particular potential action against one or more reward functions. Expected rewards 1606, 1608, and 1610 may be determined for each potential navigational action (e.g., the first, second, and Nth) determined in steps 1605, 1607, and 1609, respectively.

[0271]

[0331] In some cases, the host vehicle's navigation system may make a selection among available potential actions based on values ​​associated with expected rewards 1606, 1608, and 1610 (or any other type of indicator of expected rewards). For example, in some situations, the action that results in the highest expected reward may be selected.

[0272]

[0332] In other cases, particularly where the navigation system engages in long-term planning to determine a navigation action for the host vehicle, the system may not select the potential action that will result in the highest expected reward. Rather, the system may look to the future and analyze whether there may be an opportunity to realize a higher reward later if a low-reward action is selected depending on the current navigation state. For example, a future state may be determined for any or all of the potential actions determined in steps 1605, 1607, and 1609. Each future state determined in steps 1613, 1615, and 1617 may represent a future navigation state that is expected to occur based on the current navigation state as modified by the respective potential action (e.g., the potential action determined in steps 1605, 1607, and 1609).

[0273]

[0333] For each future state predicted in steps 1613, 1615, and 1617, one or more future actions (as navigation options available depending on the determined future state) may be determined and evaluated. In steps 1619, 1621, and 1623, for example, a value or any other type of indicator of an expected reward associated with one or more of the future actions may be developed (e.g., based on one or more reward functions). The expected reward associated with the one or more future actions may be evaluated by comparing values ​​of the reward functions associated with the respective future actions or by comparing any other indicator associated with the expected reward.

[0274]

[0334] In step 1625, the navigation system for the host vehicle may select a navigation action for the host vehicle based not only on potential actions identified for the current navigation state (e.g., in steps 1605, 1607, and 1609), but also on expected rewards determined as a result of available future potential actions depending on predicted future states (e.g., determined in steps 1613, 1615, and 1617) and based on comparing expected rewards. The selection in step 1625 may be based on the analysis of options and rewards performed in steps 1619, 1621, and 1623.

[0275]

[0335] The selection of a navigation action at step 1625 may be based solely on comparing expected rewards associated with alternative future actions. In this case, the navigation system may select an action for the current state based solely on comparing expected rewards resulting from actions for potential future navigation states. For example, the system may select the potential action identified at steps 1605, 1607, or 1609 that is associated with the highest future reward value as determined by the analysis at steps 1619, 1621, and 1623.

[0276]

[0336] The selection of a navigation action in step 1625 may be based solely on comparing current action options (as discussed above). In this situation, the navigation system may select the potential action identified in steps 1605, 1607, or 1609 that is associated with the highest expected reward 1606, 1608, or 1610. This selection may be made with little or no consideration of future expected rewards for available navigation actions depending on the future navigation state or anticipated future navigation states.

[0277]

[0337] On the other hand, in some cases, the selection of the navigation action in step 1625 may be based on comparing expected rewards associated with both future action options and current action options. This may in fact be one of the principles of navigation based on long-term planning. In order to realize potentially higher rewards for subsequent navigation actions expected to become available depending on future navigation states, the expected rewards for future actions may be analyzed to determine whether any may justify selecting an action with a lower reward depending on the current navigation state. As an example, the value or other indicator of expected reward 1606 may indicate the highest expected reward among rewards 1606, 1608, and 1610. On the other hand, expected reward 1608 may indicate the lowest expected reward among rewards 1606, 1608, and 1610. Rather than simply selecting the potential action determined in step 1605 (i.e., the action that results in the highest expected reward 1606), the analysis of future states, potential future actions, and future rewards may be used in making the selection of the navigation action in step 1625. In one example, it may be determined that a reward identified in step 1621 (in response to at least one future action for the future state determined in step 1615 based on the second potential action determined in step 1607) is likely to be higher than expected reward 1606. Based on this comparison, the second potential action determined in step 1607 may be selected over the first potential action determined in step 1605 even though expected reward 1606 is higher than expected reward 1608. In one example, the potential navigation action determined in step 1605 may include merging in front of the detected target vehicle, while the potential navigation action determined in step 1607 may include merging behind the target vehicle.Although the expected reward 1606 for merging in front of the target vehicle may be higher than the expected reward 1608 associated with merging behind the target vehicle, it may be determined that merging behind the target vehicle may result in a future state where there may be an option for an action that provides a higher potential reward than the expected rewards 1606, 1608, or other rewards based on actions available depending on the current sensed navigation state.

[0278]

[0338] The selection among the potential actions in step 1625 may be based on any suitable comparison of expected rewards (or any other metric or indicator of benefit associated with one potential action over another). In some cases, as described above, a second potential action may be selected in preference to a first potential action if the second potential action is predicted to result in at least one future action associated with a higher expected reward than the reward associated with the first potential action. In other cases, more complex comparisons may be used. For example, rewards associated with the action options depending on the predicted future states may be compared to multiple expected rewards associated with the determined potential actions.

[0279]

[0339] In some scenarios, the action and expected reward based on the predicted future state may influence the selection of a potential action for the current state if at least one of the future actions is expected to result in a reward higher than any of the expected rewards (e.g., expected rewards 1606, 1608, 1610, etc.) expected as a result of the potential action for the current state. In some cases, the future action option that results in the highest expected reward (e.g., among the expected rewards associated with the potential action for the sensed current state and the expected rewards associated with the potential future action options for the potential future navigation states) may be used as a guide for selecting a potential action for the current navigation state. That is, after identifying the future action options that result in the highest expected reward (or a reward above a predetermined threshold, etc.), the potential action that leads to the future state associated with the identified future action that results in the highest expected reward may be selected in step 1625.

[0280]

[0340] In other cases, the selection of available actions may be based on the determined difference between the expected rewards. For example, the second potential action determined in step 1607 may be selected if the difference between the expected reward associated with the future action determined in step 1621 and the expected reward 1606 exceeds the difference between the expected reward 1608 and the expected reward 1606 (assuming a + sign difference). In another example, the second potential action determined in step 1607 may be selected if the difference between the expected reward associated with the future action determined in step 1621 and the expected reward associated with the future action determined in step 1619 exceeds the difference between the expected reward 1608 and the expected reward 1606.

[0281]

[0341] Several examples have been described for selecting among potential actions for a current navigation state. However, any other suitable comparison technique or criteria can be used to select an available action by long-term planning based on an analysis of actions and rewards spanning predicted future states. In addition, while FIG. 16 shows two layers in the long-term planning analysis (e.g., a first layer that considers rewards resulting from potential actions for the current state and a second layer that considers rewards resulting from future action options depending on predicted future states), analysis based on more layers may be possible. For example, rather than basing the long-term planning analysis on one or two layers, three, four, or even more layers of analysis can be used when selecting among potential actions available depending on the current navigation state.

[0282]

[0342] After selecting among the potential actions responsive to the sensed navigation state, in step 1627, the at least one processor may cause adjustment of at least one navigation actuator of the host vehicle responsive to the selected potential navigation action. The navigation actuator may include any suitable device for controlling at least one aspect of the host vehicle. For example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0283]

[0343] Navigation based on others' inferred aggression

[0344] The target vehicle can be monitored by analyzing the acquired image stream to determine indicators of driving aggressiveness. Aggression is described herein as a qualitative or quantitative parameter, but other characteristics, i.e., perceived attention level (potential driver deficiencies, distractions - cell phone, drowsiness, etc.) can be used. In some cases, the target vehicle can be considered to have a defensive posture, and in some cases, the target vehicle can be considered to have a more aggressive posture. Navigation behavior can be selected or developed based on the indicators of aggressiveness. For example, in some cases, the relative speed, relative acceleration, increase in relative acceleration, pursuit distance, etc., relative to the host vehicle can be tracked to determine whether the target vehicle is aggressive or defensive. If the target vehicle is determined to have a level of aggressiveness above a threshold, for example, the host vehicle can be inclined to give way to the target vehicle. The level of aggressiveness of the target vehicle can also be determined based on the determined behavior of the target vehicle with respect to one or more obstacles in the path or near the target vehicle (e.g., a preceding vehicle, an obstacle in the road, a traffic light, etc.).

[0284]

[0345] As an introduction to this concept, an example experiment involving a host vehicle merging into a roundabout is described, where the navigation goal is to exit through the roundabout. The situation can begin with the host vehicle reaching the roundabout entrance and can end with the host vehicle reaching the roundabout exit (e.g., the second exit). Success can be measured based on whether the host vehicle always maintains a safe distance from all other vehicles, whether the host vehicle completes the route as quickly as possible, and whether the host vehicle follows a policy of smooth acceleration. In this discussion, N TTarget vehicles may be randomly placed on the roundabout. To model the mixture of hostile and typical behaviors, the target vehicles may be modeled with an "aggressive" driving policy with probability p, so that when the host vehicle attempts to merge in front of the target vehicle, the aggressive target vehicle accelerates. The target vehicles may be modeled with a "defensive" driving policy with probability 1-p, so that the target vehicle slows down and lets the host vehicle merge. In this experiment, p=0.5, and no information about the type of other drivers may be given to the host vehicle's navigation system. The type of other drivers may be randomly selected at the start of the episode.

[0285]

[0346] The navigation state can be represented as the host vehicle (agent) speed and position, and the target vehicle's position, speed, and acceleration. To distinguish between aggressive and defensive drivers based on the current state, it may be important to keep observing the target's acceleration. All target vehicles may travel on a one-dimensional curve that outlines the roundabout's path. The host vehicle may travel on its own one-dimensional curve that intersects with the target vehicle's curve at the junction, which is the origin of both curves. To model reasonable driving, the absolute value of all vehicle accelerations can be capped by a constant. Since driving in the opposite direction is not allowed, the speed can also be passed through a ReLU. Note that by not allowing driving in the opposite direction, the agent cannot regret its past actions, which may require long-term planning.

[0286]

[0347] As explained above, the next state s t+1 is the predictable part

number

number

number

[0287]

[0348] FIG. 18 shows a flow chart illustrating an example of an algorithm for navigating a host vehicle based on the predicted aggressiveness of other vehicles. In the example of FIG. 18, a level of aggressiveness associated with at least one target vehicle can be inferred based on the observed behavior of the target vehicle with respect to objects in the target vehicle's environment. For example, in step 1801, at least one processing device (e.g., processing device 110) of the host vehicle's navigation system can receive a plurality of images representing the host vehicle's environment from a camera associated with the host vehicle. In step 1803, analyzing one or more of the received images can enable at least one processor to identify a target vehicle (e.g., vehicle 1703) in the host vehicle's environment. In step 1805, analyzing one or more of the received images can enable at least one processing device to identify at least one obstacle for the target vehicle in the host vehicle's environment. The objects may include debris in the road, a stoplight / traffic light, a pedestrian, another vehicle (e.g., a vehicle moving in front of the target vehicle, a parked vehicle, etc.), a box in the road, a road barrier, a curve, or any other type of object that may be encountered in the host vehicle's environment. In step 1807, analyzing one or more of the received images may enable the at least one processing device to determine at least one navigation characteristic of the target vehicle relative to at least one identified obstacle for the target vehicle.

[0288]

[0349] In order to generate an appropriate navigation response to the target vehicle, various navigation characteristics can be used to infer the level of aggressiveness of the detected target vehicle. For example, such navigation characteristics may include the relative acceleration between the target vehicle and at least one identified obstacle, the distance of the target vehicle from the obstacle (e.g., the trailing distance of a target vehicle behind another vehicle), and / or the relative speed between the target vehicle and the obstacle, etc.

[0289]

[0350] In some embodiments, the navigation characteristics of the target vehicle may be determined based on output from sensors (e.g., radar, speed sensor, GPS, etc.) associated with the host vehicle. However, in some cases, the navigation characteristics of the target vehicle may be determined based in part or in full on analyzing images of the host vehicle's environment. For example, the image analysis techniques described, by way of example, above and in U.S. Pat. No. 9,168,868, incorporated herein by reference, may be used to recognize the target vehicle in the host vehicle's environment. Monitoring the position of the target vehicle in captured images over a period of time and / or monitoring the position in captured images of one or more features associated with the target vehicle (e.g., tail lights, headlights, bumpers, wheels, etc.) may enable the relative distance, speed, and / or acceleration between the target vehicle and the host vehicle or between the target vehicle and one or more other objects in the host vehicle's environment to be determined.

[0290]

[0351] The level of aggressiveness of an identified target vehicle may be inferred from any suitable observed navigation characteristic or any combination of observed navigation characteristics of the target vehicle. For example, the determination of aggressiveness may be based on any observed characteristic and one or more predefined threshold levels or any other suitable qualitative or quantitative analysis. In some embodiments, a target vehicle may be considered aggressive if it is observed to follow a host vehicle or another vehicle at a distance less than a predefined aggressive distance threshold. On the other hand, a target vehicle may be considered defensive if it is observed to follow a host vehicle or another vehicle at a distance greater than a predefined aggressive distance threshold. The predefined aggressive distance threshold need not be the same as the predefined defensive distance threshold. In addition, either or both of the predefined aggressive distance threshold and the predefined defensive distance threshold may include a range of values ​​rather than a clear boundary value. Furthermore, neither the predefined aggressive distance threshold nor the predefined defensive distance threshold need be fixed. Rather, these values ​​or range values ​​may shift over time, and different thresholds / threshold ranges may be applied based on the observed characteristics of the target vehicle. For example, the threshold applied may depend on one or more other characteristics of the target vehicle. A higher observed relative speed and / or acceleration may justify the application of a larger threshold / threshold range. Conversely, a lower relative speed and / or acceleration, including zero relative speed and / or acceleration, may justify the application of a smaller distance threshold / threshold range when making an offensive / defensive inference.

[0291]

[0352] The offensive / defensive inference may also be based on relative speed and / or relative acceleration thresholds. A target vehicle may be considered aggressive if its observed relative speed and / or its relative acceleration with respect to another vehicle is above a predetermined level or range. A target vehicle may be considered defensive if its observed relative speed and / or its relative acceleration with respect to another vehicle is below a predetermined level or range.

[0292]

[0353] The aggressive / defensive determination may be made based solely on any observed navigation characteristic, but the determination may also depend on any combination of observed characteristics. For example, as noted above, in some cases, a target vehicle may be deemed aggressive based solely on being observed to follow another vehicle at a distance less than a certain threshold or range. However, in other cases, a target vehicle may be deemed aggressive if it follows another vehicle less than a predetermined amount (which may be the same as or different from the threshold that would be applied if the determination were based solely on distance) and has a relative speed and / or relative acceleration above a predetermined amount or range. Similarly, a target vehicle may be deemed defensive based solely on being observed to follow another vehicle at a distance greater than a certain threshold or range. However, in other cases, a target vehicle may be deemed defensive if it follows another vehicle more than a predetermined amount (which may be the same as or different from the threshold that would be applied if the determination were based solely on distance) and has a relative speed and / or relative acceleration below a predetermined amount or range. The system 100 may determine aggressive / defensive, for example, if the vehicle exceeds 0.5G acceleration or deceleration (e.g., a jerk of 5 m / s3), if the vehicle has a lateral acceleration of 0.5G when changing lanes or on a curve, if the vehicle forces another vehicle to do any of the above, if the vehicle changes lanes and forces another vehicle to give way exceeding a deceleration of 0.3G or a jerk of 3 m / s3, and / or if the vehicle makes a two lane change without stopping.

[0293]

[0354] It should be understood that a reference to a quantity above a range may indicate that the quantity is above or within all values ​​associated with the range. Similarly, a reference to a quantity below a range may indicate that the quantity is below or within all values ​​associated with the range. In addition, while the described examples of making offensive / defensive inferences have been described with respect to distance, relative acceleration, and relative velocity, any other suitable quantities may be used. For example, time to collision calculations may be used, or any indirect indicator of the distance, acceleration, and / or velocity of the target vehicle. It should also be noted that while the above examples focus on the target vehicle relative to other vehicles, offensive / defensive inferences may be made by observing the navigation characteristics of the target vehicle relative to any other type of obstacle (e.g., pedestrians, road barriers, traffic lights, debris, etc.).

[0294]

[0355] Returning to the example shown in FIG. 17A and FIG. 17B, as the host vehicle 1701 approaches the roundabout, a navigation system including at least one processing device may receive an image stream from a camera associated with the host vehicle. Based on one or more analyses of the received images, any of the target vehicles 1703, 1705, 1706, 1708, and 1710 may be identified. Furthermore, the navigation system may analyze one or more navigation characteristics of the identified target vehicles. The navigation system may recognize that a gap between the target vehicles 1703 and 1705 represents a first opportunity for a potential merge into the roundabout. The navigation system may analyze the target vehicle 1703 to determine an indicator of aggressiveness associated with the target vehicle 1703. If the target vehicle 1703 is deemed aggressive, the host vehicle's navigation system may decide to yield to the vehicle 1703 rather than merge in front of the vehicle 1703. On the other hand, if the target vehicle 1703 is deemed defensive, the host vehicle's navigation system may attempt to complete the merging maneuver in front of the vehicle 1703 .

[0295]

[0356] When the host vehicle 1701 reaches the roundabout, at least one processing device of the navigation system can analyze the captured image to determine navigation characteristics associated with the target vehicle 1703. For example, based on the image, it can be determined that the vehicle 1703 is following the vehicle 1705 at a distance that provides sufficient clearance for the host vehicle 1701 to safely enter. Indeed, it may be determined that the vehicle 1703 is following the vehicle 1705 at a distance that exceeds an aggressive distance threshold, and therefore, based on this information, the host vehicle's navigation system may be inclined to identify the target vehicle 1703 as defensive. However, in some circumstances, multiple navigation characteristics of the target vehicle can be analyzed in making the aggressive / defensive determination as discussed above. Expanding the analysis, the host vehicle's navigation system can determine that the target vehicle 1703 is following behind the target vehicle 1705 at a non-aggressive distance, but that the vehicle 1703 has a relative speed and / or relative acceleration with respect to the vehicle 1705 that exceeds one or more thresholds associated with aggressive behavior. Indeed, the host vehicle 1701 may determine that the target vehicle 1703 is accelerating relative to vehicle 1705, closing the gap between vehicles 1703 and 1705. Based on further analysis of the relative speed, acceleration, and distance (as well as the rate at which the gap between vehicles 1703 and 1705 is closing), the host vehicle 1701 may determine that the target vehicle 1703 is behaving aggressively. Thus, while there may be a sufficient gap for the host vehicle to navigate safely, the host vehicle 1701 may expect that merging in front of the target vehicle 1703 will result in a vehicle navigating aggressively directly behind the host vehicle. Furthermore, if the host vehicle 1701 merges in front of vehicle 1703, based on behavior observed by image analysis or other sensor output, the target vehicle 1703 may be expected to continue accelerating toward the host vehicle 1701 or proceed toward the host vehicle 1701 at a non-zero relative speed. Such a situation may be undesirable from a safety standpoint and may also result in discomfort for passengers in the host vehicle.For this reason, as shown in FIG. 17B, host vehicle 1701 may decide to yield to vehicle 1703 and merge into the roundabout behind vehicle 1703 and in front of vehicle 1710, which is deemed defensive based on one or more analyses of its navigation characteristics.

[0296]

[0357] Returning to FIG. 18, in step 1809, at least one processing device of the host vehicle's navigation system may determine a navigation action for the host vehicle (e.g., merge in front of vehicle 1710 and behind vehicle 1703) based on the at least one identified navigation characteristic of the target vehicle relative to the identified obstacle. To implement the navigation action (step 1811), the at least one processing device may cause adjustment of at least one navigation actuator of the host vehicle in response to the determined navigation action. For example, the brakes may be applied to give way to vehicle 1703 in FIG. 17A, and the accelerator may be applied along with steering the host vehicle's wheels to cause the host vehicle to enter the roundabout behind vehicle 1703 as shown in FIG. 17B.

[0297]

[0358] As described in the examples above, the host vehicle's navigation can be based on the target vehicle's navigation characteristics relative to another vehicle or object. Additionally, the host vehicle's navigation can be based solely on the target vehicle's navigation characteristics without specific reference to another vehicle or object. For example, in step 1807 of FIG. 18, analyzing a plurality of images captured from the host vehicle's environment can enable determining at least one navigation characteristic of the identified target vehicle that is indicative of a level of aggressiveness associated with the target vehicle. The navigation characteristics can include speed, acceleration, etc., without needing reference to another object or target vehicle to make an aggressive / defensive determination. For example, observed acceleration and / or speed associated with the target vehicle that exceeds a predetermined threshold or falls within or exceeds a range of values ​​can indicate aggressive behavior. Conversely, observed acceleration and / or speed associated with the target vehicle that falls below a predetermined threshold or falls within or exceeds a range of values ​​can indicate defensive behavior.

[0298]

[0359] Of course, in some examples, observed navigation characteristics (e.g., position, distance, acceleration, etc.) may be referenced to the host vehicle to make an aggressive / defensive determination. For example, observed navigation characteristics of a target vehicle indicative of a level of aggressiveness associated with the target vehicle may include an increase in relative acceleration between the target vehicle and the host vehicle, the trailing distance of the target vehicle behind the host vehicle, the relative speed between the target vehicle and the host vehicle, etc.

[0299]

[0360] Selecting a navigation rule based on location

[0361] In some embodiments, the host vehicle's navigation system can select a navigation rule based on the applicability of the rule to the vehicle. For example, the host vehicle's navigation system can monitor the host vehicle's location (e.g., based on GPS, location broadcast beacons, recognized traffic signs, etc., or any combination thereof) and select a navigation rule (e.g., turn left with yielding on green or turn left only on green arrow signals) based on the location. Described below are systems and methods for applying such navigation rules to the autonomous control of a vehicle.

[0300]

[0362] 19, in combination with the other modules discussed above, the selection and application of navigation rules can be partitioned into two modules including, for example, a jurisdiction determination module 1902 and a navigation rule determination module 1904. In some embodiments, modules 1902 and 1904 can be stored in memory unit 140 and / or memory unit 150 of system 100, or modules 1902 and 1904 (or portions thereof) can be stored remotely from system 100 (e.g., stored in a server accessible to system 100, for example, by wireless transceiver 172).

[0301]

[0363] The jurisdiction determination module 1902, which may store instructions executable by the processing unit 110, may handle various tasks related to identifying one or more jurisdictions with which a target vehicle is associated. Such tasks may depend on inputs from various sensors and detection systems associated with the host vehicle, such as images or image streams from one or more on-board cameras, GPS location information, user feedback, user input to one or more user interface devices, radar, lidar, etc.

[0302]

[0364] For example, the jurisdiction determination module 1902 may identify a jurisdiction in which the host vehicle is traveling based on at least one indicator of the host vehicle's location. The at least one indicator may include one or more of a GPS signal or other digital or analog radio signal that includes information about the host vehicle's location. In some embodiments, the at least one indicator of the host vehicle's location may be based at least in part on analysis of a plurality of images. For example, the images may include road signs having characters, and the characters may be analyzed by optical character recognition to determine the vehicle's location.

[0303]

[0365] Determining a jurisdiction may be based on a physical presence within a particular area determined by any one or more of the sensors and detection systems described above. Such areas may include geographic boundaries, such as, for example, a state, country, city, etc. In some embodiments, multiple jurisdictions may be applicable simultaneously, for example, when the host vehicle is within a city boundary and a country boundary simultaneously. In some embodiments, a jurisdiction may include a particular type of area (e.g., urban, suburban, recreational, shopping, entertainment, dining, etc.).

[0304]

[0366] Additionally or alternatively, the jurisdiction may be determined based on attributes associated with the location or environment of the host vehicle. In some embodiments, the jurisdiction determination module 1902 may determine that the host vehicle is traveling along a particular type of road, such as a highway, which corresponds to a particular jurisdiction that is subject to particular regulations and / or traffic laws. The jurisdiction determination module 1902 may determine other attributes, such as the jurisdiction based on the time and / or date (e.g., weekend, holiday, peak hour, etc.), the status of the location as private or public property, the status of the vehicle (e.g., number of occupants), etc. Such attributes may be determined by sensors and sensing systems associated with the host vehicle as described above, by the map database 160, or by a combination of factors.

[0305]

[0367] In some embodiments, the output of the jurisdiction determination module 1902 may be communicated to a navigation rules determination module 1904. The navigation rules determination module 1904 may store instructions executable by the processing unit 110 and may implement a desired driving policy to determine one or more navigation actions for the host vehicle to take depending on the determined jurisdiction. In some embodiments, the navigation rules determination module 1904 may cause a navigation change of the host vehicle based on the identified navigation state of the host vehicle and based on the determined at least one navigation rule specific to the identified jurisdiction.

[0306]

[0368] For example, the navigation rule determination module 1904 may determine that the jurisdiction determined by the jurisdiction determination module 1902 prohibits a left turn at a traffic light intersection when there is no green arrow indicated by the traffic light. Alternatively, the navigation rule determination module 1904 may determine that the jurisdiction permits a left turn at the intersection with yielding when there is a green light indicated by the traffic light. Other rules determined by the navigation rule determination module 1904 may include rules that prohibit or permit turning when a red light is indicated, rules that prohibit turning when pedestrians are present, and the like. In some embodiments, multiple rules may apply to a particular scenario. In such a case, the navigation rule determination module 1904 may determine whether the rules are consistent or conflicting. If the rules are consistent, the navigation rule determination module 1904 may proceed with the host vehicle according to the rules. If the multiple rules are conflicting, the navigation rule determination module 1904 may prioritize the rules and apply the highest prioritized rule while adhering to safety standards. For example, a rule based on an urban jurisdiction may take precedence over a rule based on a suburban jurisdiction.

[0307]

[0369] The navigation rules determination module 1904 may determine navigation rules applicable to various scenarios. In some embodiments, the navigation rules determination module may determine rules related to specific lanes of a multi-lane road associated with a particular jurisdiction. For example, the determined jurisdiction may restrict access to one or more specific lanes to a vehicle based on weight or number of axles, or may restrict high occupancy vehicle lanes based on the number of passengers in the vehicle. The navigation rules determination module 1904 may determine that the jurisdiction prohibits travel in a lane (e.g., the left lane) of a multi-lane road except when passing another vehicle, or prohibits access to a particular lane or shoulder outside of specified dates and / or times.

[0308]

[0370] The navigation rule determination module 1904 may further determine navigation rules based on characteristics known to be in the determined jurisdiction. Such characteristics may include speed limits, road characteristics (e.g., lane width), traffic light timing, toll schedules, etc. Known characteristics may also include "implicit rules" for a particular location. Following such implicit rules may avoid undesirable behavior by the host vehicle. For example, the rule determination module 1904 may determine rules to provide additional distance between the host vehicle and the target vehicle, especially in areas known to be hilly, traffic-prone, or collision-prone. Such rules may prevent unnecessary braking, improve fuel economy, or avoid collisions. Furthermore, such implicit rules may be developed by reinforcement learning as described above with respect to the driving policy module 803. In some embodiments, a driving policy may be created and adopted based on the jurisdiction in which it was trained. In addition to or in place of the considerations discussed above with respect to the control module 805, the control module 805 may develop control instructions for one or more actuators or controlled devices associated with the host vehicle based on output from the navigation rule determination module 1904.

[0309]

[0371] FIG. 20 illustrates an example of a scene that may be captured and analyzed during navigation of the host vehicle described with respect to FIG. 12. The scene illustrated in FIG. 20 includes several features that may be affected by jurisdictional rules. For example, traffic light 2002 includes a row of multiple traffic lights that may include a green arrow and a yellow arrow in the leftmost row. Proper interpretation of the various traffic lights of traffic light 2002 may be affected by the jurisdiction in which traffic light 2002 is located. Sign 2004 indicates that vehicles turning left must yield to oncoming traffic. Additionally, sign 2006 indicates that left turns are prohibited from 7:00 a.m. to 7:00 p.m., with the exception of holidays. Navigation rule determination module 1904 may determine one or more rules based on information defining the requirements indicated by signs 2004 and / or 2006. For example, navigation rule determination module 1904 may determine a time zone associated with the jurisdiction and determine if the host vehicle is moving within the time period specified by sign 2006. The navigation rule determination module 1904 may also determine holidays observed in the jurisdiction and determine whether the host vehicle is in motion during holidays observed in the jurisdiction.

[0310]

[0372] Sign 2008 displays the name "Hartom St." The navigation rules determination module 1904 may determine whether Hartom Street spans multiple jurisdictions and whether the road includes different attributes or is subject to different laws as a result. For example, the navigation rules determination module 1904 may determine that Hartom Street is a one-way street within the boundaries of a particular city, country, or other jurisdiction.

[0311]

[0373] The sign 2010 indicates not to turn while pedestrians are present. The navigation rule determination module 1904 may determine whether rules relating to the presence or absence of pedestrians may be subject to the relevant jurisdiction. For example, the navigation rule determination module 1904 may determine that a particular jurisdiction defines the presence or absence of pedestrians based on a particular distance from a road, a distance within a road from the road boundary, etc.

[0312]

[0374] Figure 21 is an example flowchart 2100 illustrating an example process for selecting navigation rules for a host vehicle's navigation system based on location. In the example of Figure 21, navigation rules may be established or modified based on a jurisdiction associated with the host vehicle's location.

[0313]

[0375] In step 2102, at least one processing device (e.g., processing device 110) of the host vehicle's navigation system may receive a plurality of images from a camera associated with the host vehicle representing the host vehicle's environment, e.g., as described above with respect to Figures 2A-2F and 5A-5F. In step 2104, analyzing one or more of the received images may enable the at least one processor to identify a navigation state of the host vehicle, e.g., as described above with respect to detection module 801.

[0314]

[0376] In step 2106, the at least one processing device may identify a jurisdiction associated with the host vehicle. The identification may be performed by the jurisdiction determination module 1902 described above. For example, the jurisdiction determination module 1902 may identify the associated jurisdiction based on one or more combinations of GPS, image analysis of the image received in step 2102, and user input that the host vehicle is associated with multiple jurisdictions including cities and states. More specifically, the jurisdiction determination module 1902 may determine the association with a particular city and state based on a combination of desired route information entered by a user, location information received from a GPS receiver, and analyzing an image including signs, such as sign 2008 shown in FIG. 20. More specifically, the realm of jurisdiction information may be narrowed to jurisdictions that are within the route required for navigation. A particular jurisdiction may be identified by comparing the location found by the GPS signal to a database of jurisdiction locations. The location may be verified by analyzing the multiple images by verifying that an intersection identified by optical character recognition of road names reflected on road signs is actually within the identified jurisdiction.

[0315]

[0377] In step 2108, the at least one processing device may determine at least one navigation rule specific to the jurisdiction identified in step 2106. The navigation rule may be determined by any of the rule determination modules 1904 described above. To the extent multiple rules apply based on jurisdiction, as outlined above, the rule determination module 1904 may determine whether the rules match and, if not, which rule governs at a particular location.

[0316]

[0378] In step 2110, the at least one processing device may cause a navigation change in the host vehicle based on the identified navigation state of the host vehicle and based on the determined at least one navigation rule specific to the identified jurisdiction. In some embodiments, the at least one processing device of the navigation system of the host vehicle may determine a navigation action of the host vehicle required by the navigation change (e.g., turn left, stop, yield to oncoming traffic, do not overtake, do not change lanes, change lanes only to overtake, etc.) based on the determined navigation rule.

[0317]

[0379] In one embodiment, the processing device may determine based on navigation conditions that a left turn is desirable. The processing device may have identified a jurisdiction based on analyzing the vehicle's location as determined by a GPS signal and imagery received by the processor to extract Hartom St. from sign 2008 (e.g., by OCR techniques). The processing device may have determined that the location is governed by a single jurisdiction that yields to oncoming traffic but allows a left turn on a green light without an arrow. Based on this determination, the processing device may begin to turn while yielding to oncoming traffic.

[0318]

[0380] To implement the navigation action, the at least one processing device can cause at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action, such as braking to yield to oncoming traffic and accelerating along with steering the wheels of the host vehicle to turn left when the oncoming traffic has passed.

[0319]

[0381] Overtaking command function in the navigation system

[0382] In some embodiments, the host vehicle's navigation system may select a navigation rule based on a desire to overtake the target vehicle. Overtaking may be desirable for any of a number of reasons why following the target vehicle is undesirable, such as the size, type, impact on visibility of the target vehicle, impact on the comfort of the host vehicle's passengers, anticipated impact on the safety of the host vehicle's passengers, etc. Described below are systems and methods for applying such navigation rules to the autonomous control of a vehicle.

[0320]

[0383] For example, in one embodiment, a navigation system for a host vehicle may include at least one processing device (e.g., processing unit 110) that is programmed to receive a plurality of images from at least one camera (e.g., at least one of image capture units 122, 124, and 126) that represent the environment of the host vehicle. The at least one processing device may be further programmed to analyze the plurality of images to identify at least one target vehicle within the environment of the host vehicle, and to analyze the plurality of images to identify at least one characteristic of the target vehicle. The at least one characteristic of the host vehicle may relate to one or more aspects of the target vehicle (e.g., vehicle size, shape, type, etc.). After identifying the at least one characteristic, the at least one processing device may be further programmed to cause at least one navigation change of the host vehicle to initiate overtaking of the target vehicle.

[0321]

[0384] 22, in combination with the other modules discussed above, the selection and application of navigation rules can be partitioned into two modules, including a target vehicle characteristic determination module 2202 and a navigation change initiation module 2204. In some embodiments, modules 2202 and 2204 can be stored in memory unit 140 and / or memory unit 150 of system 100, or modules 2202 and 2204 (or portions thereof) can be stored remotely from system 100 (e.g., stored in a server accessible to system 100, e.g., by wireless transceiver 172).

[0322]

[0385] The target vehicle characteristic determination module 2202, which may store instructions executable by the processing unit 110, may handle various tasks related to identifying one or more characteristics of a target vehicle detected by the detection module 801. Such tasks may depend on images or image streams from one or more on-board cameras, user feedback, user input to one or more user interface devices, inputs from sensors and detection systems associated with the host vehicle, such as radar, lidar, etc.

[0323]

[0386] In some embodiments, the target vehicle characteristic determination module can determine the size and / or shape of the target vehicle. Such determination can be absolute, such as a measurement in linear distance (e.g., feet, meters, etc.), or can be relative with respect to characteristics relevant to the driving context, such as percent lane width, percent maximum height, portion of the field of view obscured by the target vehicle, etc. The dimensions and / or shape of the vehicle can be determined by analyzing the received image. For example, known distances in the image can be calculated and / or identified, and the known distances compared to the vehicle dimensions to determine the actual size of the vehicle. The relevance of the shapes can be determined by comparing shapes found in the received image with known shapes found in vehicles and other objects.

[0324]

[0387] The target vehicle characteristic determination module 2202 may also determine the type of vehicle. In some embodiments, the vehicle category (e.g., sedan, truck, van, motorcycle, commercial, etc.) may be determined. The target vehicle characteristic determination module 2202 may determine the type of vehicle by determining the shape of the target vehicle and comparing it to a database of vehicle shapes. Alternatively, the target vehicle characteristic determination module may use certain characteristics of the target vehicle, such as the size of the tires of the target vehicle, to make the determination. A trained system may be used that is trained to identify shapes or other characteristics in one or more images, and such use may gain confidence as additional determinations are completed. Additionally, the target vehicle characteristic determination module may identify characters found on the vehicle, such as characters on the tires or characters on the vehicle, by optical character recognition of the image. The target vehicle characteristic determination module 2202 may also determine whether the target vehicle falls into a category such as a construction vehicle, agricultural equipment, off-road vehicle, taxi, police vehicle, emergency vehicle, etc.

[0325]

[0388] The target vehicle characteristic determination module 2202 can determine other characteristics of the target vehicle, particularly characteristics that may be relevant to a decision to overtake the target vehicle. For example, the target vehicle characteristic determination module can detect conditions that may indicate a safety hazard to a nearby host vehicle, such as unsafe objects (e.g., flapping fabric, loose attachment straps, untied panels, open doors, unlocked doors, other unsafe objects, etc.).

[0326]

[0389] Additionally, the target vehicle characteristic determination module 2202 may determine other conditions or events associated with the target vehicle, such as the emission of material from the target vehicle (e.g., liquids, solids such as gravel or ice, particulate matter such as smoke or soot, etc.) As with the identification of vehicle characteristics described above, such determination may be made by analyzing the received imagery by comparison with shapes and / or sizes of objects in a database, by a learning algorithm, or a combination.

[0327]

[0390] In some embodiments, the target vehicle characterization module 2202 may receive an indication of particulate matter from particle detector hardware installed in the host vehicle, and based on the indication, the target vehicle characterization module 2202 may determine that the particulate matter is, for example, smoke or soot emitted by the target vehicle.

[0328]

[0391] The target vehicle characteristic determination module 2202 can output information indicative of the determined target vehicle characteristics. This output can be conclusive in nature, simply indicating that a characteristic is present, or can indicate the extent of the characteristic, such as the magnitude of large vehicle size or the extent of exhaust emissions.

[0329]

[0392] The output of the characteristic determination module 2202 may be communicated to a navigation change initiation module 2204. The navigation change initiation module 2204 may store instructions executable by the processing unit 110 and may implement a desired driving policy as described above with respect to the driving policy module 803 to determine one or more navigation actions for the host vehicle to take responsive to the determined characteristics of the target vehicle. The navigation change initiation module 2204 may cause at least one navigation change of the host vehicle to begin overtaking the target vehicle.

[0330]

[0393] The navigation change initiation module 2204 can determine a navigation change based on the hostile situation of the target vehicle, such as the impact on visibility, the impact on the comfort of the host vehicle passengers, the expected impact on the safety of the host vehicle passengers, etc. Driving policies can be adopted to avoid the observed target vehicle or the hostile situation, such as slowing down the host vehicle, moving the host vehicle into another lane, or having the host vehicle pass the target vehicle.

[0331]

[0394] In some embodiments, the navigation change initiation module 2204 can initiate a change based on the size of the target vehicle. A size-based navigation change can be initiated by taking into account several factors and / or metrics, such as height, width, area, shape, etc. The navigation change initiation module 2204 can also initiate a change based on the speed at which the target vehicle is traveling. For example, the navigation change initiation module can initiate an overtaking if the target vehicle is traveling below a speed limit.

[0332]

[0395] In some embodiments, the navigation change initiation module 2204 can initiate changes based on safety considerations. For example, driving behaviors such as weaving, frequent or sudden braking, failure to stay in a lane, etc. can be used as factors by the navigation change initiation module 2204.

[0333]

[0396] In some embodiments, the initiation navigation change module 2204 can initiate changes based on the type of vehicle. For example, the initiation navigation change module 2204 can cause the host vehicle to overtake a particular type of vehicle, such as a construction vehicle or an agricultural vehicle, or can move the host vehicle further away from certain types of vehicles than from other vehicles.

[0334]

[0397] The navigation change initiation module 2204 can also base navigation actions on other factors, such as user input. For example, the navigation change initiation module 2204 can receive input from a user interface operated by a passenger of the host vehicle. In some embodiments, input from an occupant or passenger of the host vehicle can include inputs, such as non-mechanical inputs. Such inputs can be used to initiate a navigation change, to confirm a navigation change, to override a navigation change, to reinforce the appropriateness of a navigation change, to request that a navigation change not be made in the future, or in any other combination with information received from the characterization module 2202.

[0335]

[0398] The navigation change initiation module 2204 may initiate a navigation change by defining one or more spatial targets for navigation. A spatial target may be a physical location to which the host vehicle intends to navigate to make the navigation change. The physical location may be defined absolutely (e.g., latitude and longitude), relative to one or more fixed objects, relative to a target vehicle, or by any other means of defining a spatial location. A spatial target may be associated with a temporal designation that may set an expected or intended time or time window for reaching the spatial target. Thus, in some embodiments, the processing device may be configured to provide a set of control commands to navigate the vehicle to the selected spatial target within a selected time frame. The determination of the spatial target and / or the temporal designation may be made automatically or in response to a user input. Additionally, the navigation change initiation module 2204 may determine whether a safe overtaking maneuver is possible by determining whether there is an available overtaking maneuver that will not cause an accident attributable to the vehicle.

[0336]

[0399] The initiation navigation change module 2204 may be configured to invoke a machine learning policy module, such as described above with respect to the driving policy module 803. The machine learning policy module may determine a set of actions for the vehicle to beg...

Claims

1. 1. A navigation system for a host vehicle, comprising: at least one processing device comprising a circuit and a memory, the memory being configured to, when executed by the circuit, cause the at least one processing device to: receiving a plurality of images captured by a camera, the plurality of images representing an environment of the host vehicle; analyzing the plurality of images to identify the presence of a navigation rule interruption condition in the environment of the host vehicle, the navigation rule interruption condition including anomalous behavior by one or more other vehicles, the anomalous behavior including the one or more other vehicles weaving into an adjacent lane of the host vehicle; temporarily suspending at least one navigation rule in response to identifying the navigation rule suspension condition; causing at least one navigation change in the host vehicle that is not subject to the at least one navigation rule that was temporarily suspended; and A navigation system including instructions for executing the navigation system.

2. The navigation system of claim 1 , wherein the at least one navigation change is caused by adjustment of a navigation actuator of the host vehicle.

3. The navigation system of claim 2 , wherein the navigation actuator comprises at least one of a steering mechanism, a brake, or an accelerator.

4. 4. A navigation system according to claim 1, wherein the at least one navigation rule that is temporarily suspended prohibits driving on the shoulder under normal conditions.

5. 5. The navigation system according to claim 1, wherein the at least one navigation rule that is temporarily suspended prohibits, under normal conditions, driving in a high occupancy vehicle lane with less than the required number of passengers.

6. 6. A navigation system according to claim 1, wherein the at least one navigation rule that is temporarily suspended establishes that, under normal conditions, vehicles within a roundabout have the right-of-way over vehicles entering the roundabout.

7. The navigation system of claim 1 , wherein the at least one navigation rule that is temporarily suspended establishes a navigation right-of-way under normal driving conditions.

8. 8. A navigation system according to claim 1, wherein the at least one navigation rule that is temporarily suspended prohibits navigation in a selected area under normal driving conditions.

9. The navigation system of claim 1 , wherein causing the at least one navigation change results in the host vehicle pulling onto a shoulder.

10. The navigation system of claim 1 , wherein causing the at least one navigation change results in the host vehicle entering a high occupancy vehicle lane.

11. 11. A navigation system according to claim 1, wherein causing the at least one navigation change results in the host vehicle entering a roundabout without right-of-way.

12. 12. The navigation system of claim 1, wherein causing the at least one navigation change results in the host vehicle entering another travel lane without right-of-way.

13. 13. A navigation system according to claim 1, wherein the navigation rule interruption condition includes a height of an obstacle in a road ahead of the host vehicle, and interrupting the navigation rule is based on the height exceeding a threshold.

14. 14. The navigation system of claim 1, wherein the navigation rule interruption condition includes a target vehicle entering a lane of the host vehicle.

15. 15. A navigation system according to claim 1, wherein the at least one navigation rule that is temporarily suspended is replaced with a temporary rule.

16. The navigation system of claim 15, wherein the temporary rule is more relaxed than the rule that was suspended.

17. 17. The navigation system of claim 1, further comprising instructions to execute determining whether the navigation rule interruption condition merits interruption of the at least one navigation rule based on one or more factors, the one or more factors including at least one of the safety of passengers in the host vehicle, the safety of persons in the vicinity of the host vehicle, an imminent risk of collision with a target vehicle or another object, and the possibility of loss of the host vehicle.

18. 20. The navigation system of claim 17, wherein the one or more factors include a plurality of factors, and a cumulative score related to the weights or importance of the plurality of factors is considered to determine whether interruption of the at least one navigation rule is merited.

19. 1. A method of navigation for a host vehicle, comprising the steps executed by at least one processing device: receiving a plurality of images captured by a camera, the plurality of images representing an environment of the host vehicle; analyzing the plurality of images to identify the presence of a navigation rule interruption condition in the environment of the host vehicle, the navigation rule interruption condition including anomalous behavior by one or more other vehicles, the one or more other vehicles weaving into an adjacent lane of the host vehicle; temporarily suspending at least one navigation rule in response to identifying the navigation rule suspension condition; causing at least one navigation change in the host vehicle that is not subject to the at least one navigation rule that has been temporarily suspended; A method for providing the above.

20. a program that, when executed by at least one processing device, causes the at least one processing device to perform a method for navigating a host vehicle, the method comprising: receiving a plurality of images captured by a camera, the plurality of images representing an environment of the host vehicle; analyzing the plurality of images to identify the presence of a navigation rule interruption condition in the environment of the host vehicle, the navigation rule interruption condition including anomalous behavior by one or more other vehicles, the one or more other vehicles weaving into an adjacent lane of the host vehicle; temporarily suspending at least one navigation rule in response to identifying the navigation rule suspension condition; causing at least one navigation change in the host vehicle that is not subject to the at least one navigation rule that has been temporarily suspended; A program that includes:

Images