Systems, methods, and computer-readable media for navigating a host vehicle

Abstract

The present disclosure provides a system, method and computer-readable medium for navigating a host vehicle. The system includes: at least one processor programmed to: receive at least one image representing an environment of the host vehicle from an image capture device; detect at least one obstacle in the environment of the host vehicle based on analysis of the at least one image; determine the speed of the host vehicle and the predicted path of the host vehicle; monitor a driver input to at least one of a throttle control, a brake control or a steering control associated with the host vehicle; determine whether the driver input will cause the host vehicle to navigate within a proximity buffer relative to at least one obstacle, wherein the proximity buffer is determined based on the determined speed, the maximum acceleration capability of the host vehicle and the maximum braking capability of the host vehicle and a reaction time associated with the host vehicle.

Description

[0001] This application is a divisional application of the invention patent application filed on March 20, 2019, with application number 201980002445.4 (international application number PCT / IB2019 / 000281) and entitled "System and method for navigating a vehicle".

[0002] Cross-references to related applications

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 645,479, filed March 20, 2018; U.S. Provisional Patent Application No. 62 / 646,579, filed March 22, 2018; U.S. Provisional Patent Application No. 62 / 718,554, filed August 14, 2018; U.S. Provisional Patent Application No. 62 / 724,355, filed August 29, 2018; U.S. Provisional Patent Application No. 62 / 772,366, filed November 28, 2018; and U.S. Provisional Patent Application No. 62 / 777,914, filed December 11, 2018. All of the above applications are incorporated herein by reference in their entirety. Technical Field

[0004] This disclosure generally relates to autonomous vehicle navigation. Furthermore, this disclosure relates to systems and methods for navigation based on potential accident liability constraints. Background Technology

[0005] With continuous technological advancements, the goal of fully autonomous vehicles capable of navigating on roads is emerging. Autonomous vehicles may need to consider a wide variety of factors and make appropriate decisions based on those factors to safely and accurately reach their desired destination. For example, autonomous vehicles may need to process and interpret visual information (e.g., information captured from cameras), information from radar or lidar, and may also use information from other sources (e.g., from GPS devices, speed sensors, accelerometers, suspension sensors, etc.). Simultaneously, to navigate to their destination, autonomous vehicles may also need to identify their location within a specific roadway (e.g., a specific lane in a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, observe traffic signals and signs, move from one road to another at appropriate intersections or junctions, and respond to any other situations that occur or progress during vehicle operation. Furthermore, navigation systems may need to comply with certain imposed constraints. In some cases, these constraints may relate to the interaction between the primary vehicle and one or more other objects (such as other vehicles, pedestrians, etc.). In other cases, constraints may relate to rules of responsibility to be followed when performing one or more navigation actions for the primary vehicle.

[0006] In the field of autonomous driving, there are two important considerations for feasible autonomous vehicle systems. The first is the standardization of safety assurance, including the requirements that every autonomous vehicle must meet to ensure safety, and how to verify these requirements. The second is scalability, because engineering solutions that lead to cost relief cannot scale to millions of vehicles and may hinder the widespread adoption of autonomous vehicles, or even their less widespread adoption. Therefore, an interpretable mathematical model for safety assurance and a system design that complies with safety assurance requirements while scalable to millions of vehicles are needed. Summary of the Invention

[0007] Embodiments consistent with this disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation features. For example, consistent with the disclosed embodiments, the disclosed system may include one, two, or more cameras monitoring the vehicle's environment. The disclosed system may provide a navigation response based, for example, analysis of images captured by one or more cameras. The navigation response may also take into account other data, including, for example, global positioning system (GPS) data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data.

[0008] In one embodiment, a system for navigating a primary vehicle may include at least one processing device programmed to receive at least one image representing the primary vehicle's environment. The at least one image may be received from an image capture device. The at least one processing device may be programmed to determine a planned navigation action for achieving a navigation objective of the primary vehicle based on at least one driving strategy. The at least one processing device may be further programmed to analyze the at least one image to identify a target vehicle in the primary vehicle's environment and to determine the next state distance between the primary vehicle and the target vehicle if the planned navigation action is taken. The at least one processing device may be programmed to determine the primary vehicle's maximum braking capacity, maximum acceleration capacity, and current speed. The at least one processing device may also be programmed to determine the primary vehicle's current stopping distance based on the primary vehicle's current maximum braking capacity, current maximum acceleration capacity, and current speed. The at least one processing device may be further programmed to determine the target vehicle's current speed and to assume the target vehicle's maximum braking capacity based on at least one identified characteristic of the target vehicle. The at least one processing device may also be programmed to perform a planning and navigation action if the determined current stopping distance of the master vehicle is less than the sum of the determined next state distance and the target vehicle travel distance determined based on the current speed of the target vehicle and the assumed maximum braking capacity of the target vehicle.

[0009] In one embodiment, a system for navigating a master vehicle may include at least one processing device. The at least one processing device may be programmed to receive at least one image representing the master vehicle's environment from an image capture device. The at least one processing device may also be programmed to determine a planned navigation action for achieving a navigation objective of the master vehicle. The planned navigation action may be based on at least one driving strategy. The at least one processing device may be programmed to analyze at least one image to identify a target vehicle in the master vehicle's environment. The at least one processing device may be further programmed to determine the next-state distance between the master vehicle and the target vehicle that would result if the planned navigation action were taken. The at least one processing device may be programmed to determine the current speed of the master vehicle and the current speed of the target vehicle. The at least one processing device may be programmed to assume the maximum braking rate capability of the target vehicle based on at least one identified characteristic of the target vehicle. The at least one processing device may be further programmed to perform the planned navigation action if, for the determined current speed of the master vehicle and at a predetermined submaximum braking rate less than the master vehicle's maximum braking rate capability, the master vehicle can stop within a master vehicle stopping distance, the master vehicle stopping distance being less than the sum of the determined next-state distance and the target vehicle's travel distance determined based on the target vehicle's current speed and the assumed maximum braking rate capability of the target vehicle.

[0010] In one embodiment, a system for navigating a host vehicle may include at least one processing device. The at least one processing device may be programmed to receive at least one image representing the host vehicle's environment from an image capture device. The at least one processing device may also be programmed to determine a planned navigation action for achieving a navigation objective of the host vehicle. The planned navigation action may be based on at least one driving strategy. The at least one processing device may be programmed to analyze at least one image to identify a target vehicle in the host vehicle's environment. The at least one processing device may be further programmed to determine the next state distance between the host vehicle and the target vehicle if the planned navigation action is taken. The at least one processing device may be programmed to determine the host vehicle's current speed. The at least one processing device may be further programmed to determine the target vehicle's current speed and assume the target vehicle's maximum braking rate capability based on at least one identified characteristic of the target vehicle. The at least one processing device can be programmed to perform a planned navigation action if, for the determined current speed of the master vehicle and for a predetermined braking rate curve, the master vehicle can stop within a master vehicle stopping distance, wherein the master vehicle stopping distance is less than the sum of a determined next state distance and a target vehicle travel distance determined based on the current speed of the target vehicle and the assumed maximum braking rate capability of the target vehicle, wherein the predetermined braking rate curve gradually increases from the second-maximum braking rate of the master vehicle to the maximum braking rate.

[0011] In one embodiment, a system for braking a host vehicle may include at least one processing device programmed to perform one or more operations. The at least one processing device may be programmed to receive an output representing the host vehicle's environment from at least one sensor. The at least one processing device may be further programmed to detect a target vehicle in the host vehicle's environment based on the output. The at least one processing device may be programmed to determine the host vehicle's current speed and the current distance between the host vehicle and the target vehicle. Based at least on the host vehicle's current speed and the current distance between the host vehicle and the target vehicle, the at least one processor may be programmed to determine whether braking conditions exist. If braking conditions are determined to exist, the at least one processor may be programmed to apply braking devices associated with the host vehicle according to a predetermined braking curve, the predetermined braking curve comprising segments starting at the host vehicle's submaximum braking rate and gradually increasing to the host vehicle's maximum braking rate.

[0012] In one embodiment, an autonomous system for selectively replacing human driver control of a primary vehicle may include at least one processing device. The at least one processing device may be programmed to receive at least one image representing the primary vehicle's environment from an image capture device, and to detect at least one obstacle in the primary vehicle's environment based on analysis of the at least one image. The at least one processing device may be programmed to monitor driver input to at least one of a throttle control, brake control, or steering control associated with the primary vehicle. The at least one processing device may also be programmed to determine whether the driver input would cause the primary vehicle to navigate within a proximity buffer relative to the at least one obstacle. If the at least one processing device determines that the driver input would not cause the primary vehicle to navigate within the proximity buffer relative to the at least one obstacle, then the at least one processing device may be programmed to allow the driver input to cause a corresponding change in one or more primary vehicle motion control systems. If the at least one processing device determines that the driver input would cause the primary vehicle to navigate within the proximity buffer relative to the at least one obstacle, then the at least one processing device may be programmed to prevent the driver input from causing a corresponding change in one or more primary vehicle motion control systems.

[0013] In one embodiment, a navigation system for navigating an autonomous primary vehicle based on at least one navigation target may include at least one processor. The at least one processor may be programmed to receive sensor outputs from one or more sensors indicating at least one aspect of the primary vehicle's motion relative to its environment. The sensor outputs may be generated at a first time, later than the data acquisition time when the measurement or data acquisition on which the sensor outputs are based, is acquired, and earlier than a second time when the sensor outputs are received by the at least one processor. The at least one processor may be programmed to generate a prediction of at least one aspect of the primary vehicle's motion, for a motion prediction time, based at least in part on the received sensor outputs and an estimate of how at least one aspect of the primary vehicle's motion changes during the time interval between the data acquisition time and the motion prediction time. The at least one processor may be programmed to determine a planned navigation action for the primary vehicle, based at least in part on at least one navigation target of the primary vehicle and on the generated prediction of at least one aspect of the primary vehicle's motion. The at least one processor may be further configured to generate navigation commands for implementing at least a portion of the planned navigation actions. The at least one processor may be programmed to provide the navigation commands to at least one actuation system of the primary vehicle. The navigation command can be provided such that at least one actuation system receives the navigation command at a third time, which is later than the second time and earlier than or substantially equal to the actuation time at which a component of at least one actuation system responds to the received command. In some embodiments, the motion prediction time is after the data acquisition time and earlier than or equal to the actuation time.

[0014] In one embodiment, a system for navigating a master vehicle includes: at least one processing device programmed to: receive at least one image representing the environment of the master vehicle from an image capturing device; determine a planned navigation action for achieving a navigation objective of the master vehicle based on at least one driving strategy; analyze the at least one image to identify a target vehicle in the environment of the master vehicle; determine a next-state distance between the master vehicle and the target vehicle if the planned navigation action is taken; determine the master vehicle's maximum braking capacity, maximum acceleration capacity, and current speed, wherein the master vehicle's maximum braking capacity is determined based on at least one factor associated with the master vehicle or the environment of the master vehicle; determine the master vehicle's current stopping distance based on the master vehicle's current maximum braking capacity, current maximum acceleration capacity, and current speed; determine the target vehicle's current speed and assume the target vehicle's maximum braking capacity based on at least one identified characteristic of the target vehicle; and perform the planned navigation action if the determined current stopping distance of the master vehicle is less than the sum of the determined next-state distance and the target vehicle's travel distance determined based on the target vehicle's current speed and the assumed target vehicle's maximum braking capacity.

[0015] In one embodiment, a method for navigating a primary vehicle includes: receiving at least one image representing the environment of the primary vehicle from an image capture device; determining a planned navigation action for achieving a navigation objective of the primary vehicle based on at least one driving strategy; analyzing the at least one image to identify a target vehicle in the environment of the primary vehicle; determining a next-state distance between the primary vehicle and the target vehicle that would result if the planned navigation action were taken; determining the primary vehicle's maximum braking capacity, maximum acceleration capacity, and current speed, wherein the primary vehicle's maximum braking capacity is determined based on at least one factor associated with the primary vehicle or its environment; determining the primary vehicle's current stopping distance based on the primary vehicle's current maximum braking capacity, current maximum acceleration capacity, and current speed; determining the target vehicle's current speed and assuming the target vehicle's maximum braking capacity based on at least one identified characteristic of the target vehicle; and performing the planned navigation action if the determined current stopping distance of the primary vehicle is less than the sum of the determined next-state distance and the target vehicle's travel distance determined based on the target vehicle's current speed and the assumed target vehicle's maximum braking capacity.

[0016] In one embodiment, a system for a host vehicle may include: at least one processor programmed to: receive at least one image representing the environment of the host vehicle from an image capture device; detect at least one obstacle in the environment of the host vehicle based on analysis of the at least one image; determine the speed of the host vehicle and a predicted path of the host vehicle; monitor driver input to at least one of a throttle control, a brake control, or a steering control associated with the host vehicle; and determine whether the driver input will cause the host vehicle to navigate within an approach buffer relative to the at least one obstacle, wherein the approach buffer is determined based on the determined speed, the maximum acceleration capability of the host vehicle, the maximum braking capability of the host vehicle, and the reaction time associated with the host vehicle.

[0017] In one embodiment, a method for navigating a master vehicle may include: receiving from an image capturing device at least one image representing the environment of the master vehicle; detecting at least one obstacle in the environment of the master vehicle based on analysis of the at least one image; determining the speed of the master vehicle and a predicted path of the master vehicle; monitoring driver input to at least one of a throttle control, a brake control, or a steering control associated with the master vehicle; and determining whether the driver input will cause the master vehicle to navigate within an approach buffer relative to the at least one obstacle, wherein the approach buffer is determined based on the determined speed, the maximum acceleration capability of the master vehicle, the maximum braking capability of the master vehicle, and the reaction time associated with the master vehicle.

[0018] In one embodiment, a non-transitory computer-readable medium may store instructions that, when executed by a processor, cause the processor to perform operations including: receiving from an image capture device at least one image representing the environment of a primary vehicle; detecting at least one obstacle in the environment of the primary vehicle based on analysis of the at least one image; determining the speed of the primary vehicle and a predicted path of the primary vehicle; monitoring driver input to at least one of a throttle control, a brake control, or a steering control associated with the primary vehicle; and determining whether the driver input would cause the primary vehicle to navigate within an approach buffer relative to the at least one obstacle, wherein the approach buffer is determined based on the determined speed, the maximum acceleration capability of the primary vehicle, the maximum braking capability of the primary vehicle, and the reaction time associated with the primary vehicle.

[0019] Consistent with other disclosed embodiments, a non-transitory computer-readable storage medium may store program instructions that can be executed by at least one processing device and perform any of the steps and / or methods described herein.

[0020] The foregoing general description and the following detailed description are merely exemplary and explanatory, and not intended to limit the scope of the claims. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various embodiments disclosed. In the drawings:

[0022] Figure 1 It is a schematic representation of an example system consistent with the disclosed embodiments.

[0023] Figure 2A It is an illustrative side view representation of an example vehicle that includes a system consistent with the disclosed embodiments.

[0024] Figure 2B It is consistent with the disclosed embodiments. Figure 2A The illustration shows a top view of the vehicle and system.

[0025] Figure 2C This is an illustrative top view representation of another embodiment of a vehicle that includes a system consistent with the disclosed embodiments.

[0026] Figure 2D This is an illustrative top view representation of yet another embodiment of a vehicle that includes a system consistent with the disclosed embodiments.

[0027] Figure 2E This is an illustrative top view representation of yet another embodiment of a vehicle that includes a system consistent with the disclosed embodiments.

[0028] Figure 2F This is a schematic representation of an example vehicle control system consistent with the disclosed embodiments.

[0029] Figure 3A It is an illustrative representation of the interior of a vehicle, including a rearview mirror and a user interface for a vehicle imaging system, consistent with the disclosed embodiments.

[0030] Figure 3B This is an illustration of an example of a camera mounted, consistent with the disclosed embodiments, configured to be positioned behind the rearview mirror and against the vehicle's windshield.

[0031] Figure 3C It is consistent with the disclosed embodiments. Figure 3B The diagram shows the camera installation from different perspectives.

[0032] Figure 3D This is an illustration of an example of a camera mounted behind a rearview mirror and against a vehicle windshield, consistent with the disclosed embodiments.

[0033] Figure 4 This is an example block diagram of a memory configured to store instructions for performing one or more operations, consistent with the disclosed embodiments.

[0034] Figure 5A This is a flowchart illustrating an example process for causing one or more navigation responses based on monocular image analysis, consistent with the disclosed embodiments.

[0035] Figure 5B This is a flowchart illustrating an example process for detecting one or more vehicles and / or pedestrians in a set of images, consistent with the disclosed embodiments.

[0036] Figure 5C This is a flowchart illustrating an example process for detecting road markings and / or lane geometry information in a set of images, consistent with the disclosed embodiments.

[0037] Figure 5D This is a flowchart illustrating an example process for detecting traffic lights in a set of images, consistent with the disclosed embodiments.

[0038] Figure 5E This is a flowchart illustrating an example process for generating one or more navigation responses based on a vehicle path, consistent with the disclosed embodiments.

[0039] Figure 5F This is a flowchart illustrating an example process for determining whether a vehicle ahead is changing lanes, consistent with the disclosed embodiments.

[0040] Figure 6 This is a flowchart illustrating an example process for causing one or more navigation responses based on stereo image analysis, consistent with the disclosed embodiments.

[0041] Figure 7 This is a flowchart illustrating an example process consistent with the disclosed embodiments for causing one or more navigation responses based on the analysis of three sets of images.

[0042] Figure 8 It is a block diagram representation of a module that is consistent with the disclosed embodiments and can be implemented by one or more specially programmed processing devices of the navigation system of an autonomous vehicle.

[0043] Figure 9 It is a navigation options diagram consistent with the disclosed embodiments.

[0044] Figure 10 It is a navigation options diagram consistent with the disclosed embodiments.

[0045] Figure 11A , Figure 11B and Figure 11C A schematic representation of navigation options for the primary vehicle in a merge zone, consistent with the disclosed embodiments, is provided.

[0046] Figure 11D An illustrated description of a dual-lane merging scenario consistent with the disclosed embodiments is provided.

[0047] Figure 11E An option diagram, consistent with the disclosed embodiments, is provided that is potentially useful in dual-lane merging scenarios.

[0048] Figure 12 A representative image capturing the environment of the host vehicle and a graph of potential navigation constraints, consistent with the disclosed embodiments, is provided.

[0049] Figure 13 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0050] Figure 14 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0051] Figure 15 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0052] Figure 16 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0053] Figure 17A and 17B An illustrative diagram of a main vehicle navigating into a roundabout, consistent with the disclosed embodiments, is provided.

[0054] Figure 18 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0055] Figure 19 An example of a main vehicle traveling on a multi-lane highway, consistent with the disclosed embodiments, is shown.

[0056] Figure 20A and 20B An example of a vehicle cutting in front of another vehicle, consistent with the disclosed embodiments, is shown.

[0057] Figure 21 An example of a vehicle following another vehicle, consistent with the disclosed embodiments, is shown.

[0058] Figure 22 An example of a vehicle leaving a parking lot and merging into a potentially busy road, consistent with the disclosed embodiments, is shown.

[0059] Figure 23 A vehicle traveling on a road, consistent with the disclosed embodiments, is shown.

[0060] Figures 24A-24D Four example scenarios consistent with the disclosed embodiments are shown.

[0061] Figure 25 An example scenario consistent with the disclosed embodiments is shown.

[0062] Figure 26 An example scenario consistent with the disclosed embodiments is shown.

[0063] Figure 27 An example scenario consistent with the disclosed embodiments is shown.

[0064] Figure 28A and 28B An example of a scenario consistent with the disclosed embodiments, in which a vehicle is following another vehicle, is shown.

[0065] Figure 29A and 29B Example blame is shown in a scenario consistent with the disclosed embodiments.

[0066] Figure 30A and 30B Example attribution is shown in a scenario consistent with the disclosed embodiments.

[0067] Figures 31A-31D An example attribution in a drift scenario consistent with the disclosed embodiments is shown.

[0068] Figure 32A and 32B An example attribution is shown in a two-way traffic scenario consistent with the disclosed embodiments.

[0069] Figure 33A and 33B An example attribution is shown in a two-way traffic scenario consistent with the disclosed embodiments.

[0070] Figure 34A and 34B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0071] Figure 35A and 35B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0072] Figure 36A and 36B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0073] Figure 37A and37B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0074] Figure 38A and 38B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0075] Figure 39A and 39B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0076] Figure 40A and 40B An example attribution is shown in a traffic light scenario consistent with the disclosed embodiments.

[0077] Figure 41A and 41B An example attribution is shown in a traffic light scenario consistent with the disclosed embodiments.

[0078] Figure 42A and 42B An example attribution is shown in a traffic light scenario consistent with the disclosed embodiments.

[0079] Figures 43A-43C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is shown.

[0080] Figures 44A-44C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is shown.

[0081] Figure 45A-45C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is shown.

[0082] Figures 46A-46D An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is shown.

[0083] Figure 47A and 47B An example scenario consistent with the disclosed embodiments is shown, in which a vehicle follows another vehicle.

[0084] Figure 48 This illustrates an exemplary process for navigating a master vehicle that is consistent with the disclosed embodiments.

[0085] Figures 49A-49D An example scenario consistent with the disclosed embodiments is shown, in which a vehicle follows another vehicle.

[0086] Figure 50 This is a flowchart illustrating an exemplary process for braking a master vehicle consistent with the disclosed embodiments.

[0087] Figure 51 This is a flowchart illustrating an exemplary process for navigating a master vehicle, consistent with the disclosed embodiments.

[0088] Figures 52A-52D An example of a main vehicle approaching a buffer zone is shown, consistent with the disclosed embodiments.

[0089] Figure 53A and 53B An example scenario including a proximity buffer consistent with the disclosed embodiments is shown.

[0090] Figure 54A and 54B An example scenario including a proximity buffer consistent with the disclosed embodiments is shown.

[0091] Figure 55 A flowchart for selectively replacing human driver control of the master vehicle, consistent with the disclosed embodiments, is provided.

[0092] Figure 56 This is a flowchart illustrating an exemplary process for navigating a master vehicle, consistent with the disclosed embodiments.

[0093] Figures 57A-57C An example scenario consistent with the disclosed embodiments is shown.

[0094] Figure 58 This is a flowchart illustrating an exemplary process for navigating a master vehicle, consistent with the disclosed embodiments. Detailed Implementation

[0095] The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and the following description to refer to the same or similar parts. Although several exemplary embodiments are described herein, modifications, adjustments, and other implementations are possible. For example, components shown in the drawings may be replaced, added, or modified, and the exemplary methods described herein may be modified by replacing, reordering, removing, or adding steps to the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. Rather, the appropriate scope is defined by the appended claims.

[0096] Overview of autonomous vehicles

[0097] As used throughout this disclosure, the term "autonomous vehicle" means a vehicle capable of performing at least one navigation change without driver input. "Navigation change" refers to one or more of the vehicle's steering, braking, or acceleration / deceleration. "Autonomous" means that the vehicle does not need to be fully automatic (e.g., fully operable without a driver or driver input). Rather, autonomous vehicles include those capable of operating under driver control during certain time periods and without driver control during other time periods. Autonomous vehicles may also include those that control only certain aspects of vehicle navigation (such as steering (e.g., maintaining vehicle path between lane constraints) or certain steering maneuvers in certain situations (but not all situations)) but can delegate other aspects to the driver (e.g., braking or braking in certain situations). In some cases, an autonomous vehicle may handle some or all aspects of the vehicle's braking, rate control, and / or steering.

[0098] Traffic infrastructure, featuring lane markings, traffic signs, and traffic lights designed to provide visual information to drivers, has been constructed because human drivers typically rely on visual cues and observation to control vehicles. Given these design characteristics of traffic infrastructure, autonomous vehicles can include cameras and processing units that analyze visual information captured from the vehicle's environment. Visual information can include, for example, images representing components of traffic infrastructure (e.g., lane markings, traffic signs, traffic lights, etc.) that can be observed by the driver, as well as other obstacles (e.g., other vehicles, pedestrians, debris, etc.). Furthermore, autonomous vehicles can utilize stored information, such as information that provides a model of the vehicle's environment during navigation. For example, a vehicle can use GPS data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data to provide information about its environment while the vehicle is in motion, and the vehicle (and other vehicles) can use this information to position itself on the model. Some vehicles are also capable of communicating with each other, sharing information, and altering hazards or changes around the vehicle, such as those affecting other vehicles.

[0099] System Overview

[0100] Figure 1This is a block diagram representation of system 100 consistent with the disclosed example embodiments. System 100 may include various components depending on the requirements of a particular implementation. In some embodiments, system 100 may include a processing unit 110, an image acquisition unit 120, a position sensor 130, one or more memory units 140, 150, a map database 160, a user interface 170, and a wireless transceiver 172. Processing unit 110 may include one or more processing devices. In some embodiments, processing unit 110 may include an application processor 180, an image processor 190, or any other suitable processing device. Similarly, depending on the requirements of a particular application, image acquisition unit 120 may include any number of image acquisition devices and components. In some embodiments, image acquisition unit 120 may include one or more image capture devices (e.g., a camera, a charge-coupled device (CCD), or any other type of image sensor), such as image capture device 122, image capture device 124, and image capture device 126. System 100 may also include a data interface 128 that communicatively connects processing unit 110 to image acquisition unit 120. For example, data interface 128 may include any wired and / or wireless links for transmitting image data acquired by image acquisition unit 120 to processing unit 110.

[0101] Wireless transceiver 172 may include one or more devices configured to exchange transmissions over an air interface to one or more networks (e.g., cellular, Internet, etc.) using radio frequency, infrared frequency, magnetic field, or electric field. Wireless transceiver 172 may use any well-known standard to send and / or receive data (e.g., Wi-Fi, etc.). (Bluetooth Smart, 802.15.4, ZigBee, etc.). Such transmissions can include communication from the master vehicle to one or more remotely located servers. Such transmissions can also include (one-way or two-way) communication between the master vehicle and one or more target vehicles in the master vehicle's environment (e.g., to coordinate the master vehicle's navigation in consideration of or in conjunction with target vehicles in the master vehicle's environment), or even broadcast transmissions to unspecified receivers near the sending vehicle.

[0102] Both application processor 180 and image processor 190 can include various types of hardware-based processing devices. For example, either or both of application processor 180 and image processor 190 can include a microprocessor, preprocessor (such as an image preprocessor), graphics processor, central processing unit (CPU), auxiliary circuitry, digital signal processor, integrated circuit, memory, or any other type of device suitable for running applications and for image processing and analysis. In some embodiments, application processor 180 and / or image processor 190 can include any type of single-core or multi-core processor, mobile device microcontroller, central processing unit, etc. Various processing devices can be used, including, for example, those available from, such as… Processors obtained from manufacturers, and can include various architectures (e.g., x86 processors, ...). wait).

[0103] In some embodiments, application processor 180 and / or image processor 190 may include components that can be accessed from... Any EyeQ series processor chip obtained. These processor designs all include multiple processing units with local memory and instruction sets. Such a processor may include video input for receiving image data from multiple image sensors, and may also include video output capability. In one example, It uses 90 nanometer-micrometer technology that operates at 332 MHz. The architecture consists of two floating-point hyper-threaded 32-bit RISC CPUs ( (core), five Visual Computing Engines (VCE), and three Vector Microcode Processors The system consists of a Denali 64-bit mobile DDR controller, a 128-bit internal Sonics Interconnect, dual 16-bit video input and 18-bit video output controllers, a 16-channel DMA, and several peripherals. The MIPS34K CPU manages these five VCEs and three VMPs. TM And DMA, a second MIPS34K CPU and multi-channel DMA, and other peripherals. These five VCEs, three... The MIPS34K CPU can perform the intensive vision computations required for versatile bundled applications. In another example, as a third-generation processor and... Six times stronger It can be used in the disclosed embodiments. In other examples, it can be used in the disclosed embodiments. and / or Of course, any newer or future EyeQ processing devices can also be used with the disclosed embodiments.

[0104] Any processing device disclosed herein can be configured to perform certain functions. Configuring a processing device (such as any described EyeQ processor or other controller or microprocessor) to perform certain functions may include programming computer-executable instructions and making those instructions available to the processing device for execution during its operation. In some embodiments, configuring the processing device may include programming the processing device directly using architectural instructions. In other embodiments, configuring the processing device may include storing executable instructions on memory accessible to the processing device during operation. For example, the processing device may access the memory during operation to obtain and execute the stored instructions. In any case, a processing device configured to perform the sensing, image analysis, and / or navigation functions disclosed herein represents a dedicated hardware-based system for controlling multiple hardware-based components of a master vehicle.

[0105] although Figure 1 Two separate processing devices included in processing unit 110 are depicted, but more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to perform the tasks of application processor 180 and image processor 190. In other embodiments, these tasks may be performed by two or more processing devices. Additionally, in some embodiments, system 100 may include one or more processing units 110 without other components such as image acquisition unit 120.

[0106] Processing unit 110 may include various types of devices. For example, processing unit 110 may include various devices such as controllers, image preprocessors, central processing units (CPUs), auxiliary circuitry, digital signal processors, integrated circuits, memory, or any other type of device for image processing and analysis. Image preprocessors may include video processors for capturing, digitizing, and processing images from image sensors. CPUs may include any number of microcontrollers or microprocessors. Auxiliary circuitry may be any number of circuits known in the art, including caches, power supplies, clocks, and input / output circuits. Memory may store software that controls the operation of the system when executed by the processor. Memory may include databases and image processing software. Memory may include any number of random access memories, read-only memories, flash memory, disk drives, optical storage, magnetic tape storage, removable storage, and other types of storage. In one instance, the memory may be separate from processing unit 110. In another instance, the memory may be integrated into processing unit 110.

[0107] Each memory unit 140, 150 may include software instructions that, when executed by a processor (e.g., application processor 180 and / or image processor 190), can control the operation of various aspects of system 100. For example, these memory units may include various database and image processing software, as well as trained systems such as neural networks and deep neural networks. Memory units may include random access memory, read-only memory, flash memory, disk drives, optical storage, magnetic tape storage, removable storage, and / or any other type of storage. In some embodiments, memory units 140, 150 may be decoupled from application processor 180 and / or image processor 190. In other embodiments, these memory units may be integrated into application processor 180 and / or image processor 190.

[0108] The position sensor 130 may include any type of device suitable for determining the location associated with at least one component of the system 100. In some embodiments, the position sensor 130 may include a GPS receiver. Such a receiver can determine the user's location and speed by processing signals broadcast by Global Positioning System satellites. The location information from the position sensor 130 may be made available to the application processor 180 and / or the image processor 190.

[0109] In some embodiments, system 100 may include components such as a speed sensor (e.g., a speedometer) for measuring the rate of vehicle 200. System 100 may also include one or more accelerometers (single-axis or multi-axis) for measuring the acceleration of vehicle 200 along one or more axes.

[0110] Memory units 140 and 150 may include a database, or data organized in any other form, indicating the location of known landmarks. Sensory information about the environment (such as images from lidar or stereo processing of two or more images, radar signals, depth information) can be processed together with location information (such as GPS coordinates, vehicle self-motion, etc.) to determine the vehicle's current position relative to known landmarks and to improve the vehicle's position. Certain aspects of this technology are included in what is known as REM... TM In the positioning technology, the technology is sold by the assignee of this application.

[0111] User interface 170 may include any device suitable for providing information to or receiving input from one or more users of system 100. In some embodiments, user interface 170 may include user input devices, including, for example, a touchscreen, microphone, keyboard, pointer device, tracking wheel, camera, knob, button, etc. Using such input devices, users can provide information input or commands to system 100 by typing instructions or information, providing voice commands, selecting menu options on the screen using buttons, pointers, or eye-tracking capabilities, or by any other technology suitable for transmitting information to system 100.

[0112] User interface 170 may be equipped with one or more processing devices configured to provide and receive information from the user, and process that information for use by, for example, application processor 180. In some embodiments, such processing devices may execute instructions to recognize and track eye movements, receive and interpret voice commands, recognize and interpret touches and / or gestures made on a touchscreen, respond to keyboard input or menu selections, etc. In some embodiments, user interface 170 may include a display, speakers, haptic devices, and / or any other devices for providing output information to the user.

[0113] Map database 160 may include any type of database for storing map data useful to system 100. In some embodiments, map database 160 may include data relating to the location of various items in a reference coordinate system, including roads, water features, geographic features, commercial areas, points of interest, restaurants, gas stations, etc. Map database 160 may not only store the locations of these items but also descriptors associated with them, including, for example, names associated with any stored features. In some embodiments, map database 160 may be physically located together with other components of system 100. Alternatively or additionally, map database 160 or a portion thereof may be located remotely relative to other components of system 100 (e.g., processing unit 110). In such embodiments, information from map database 160 may be downloaded via a wired or wireless data connection to a network (e.g., via cellular networks and / or the Internet, etc.). In some cases, map database 160 may store a sparse data model comprising a polynomial representation of certain road features (e.g., lane markings) or the target trajectory of a primary vehicle. The map database 160 may also include stored representations of various identified landmarks, which can be used to determine or update the known position of the master vehicle relative to the target trajectory. Landmark representations may include data fields such as landmark type, landmark location, and other potential identifiers.

[0114] Image capture devices 122, 124, and 126 may all include any type of device suitable for capturing at least one image from the environment. Furthermore, any number of image capture devices can be used to acquire images for input to the image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. Reference will be made below. Figures 2B to 2E Image capture devices 122, 124 and 126 are further described.

[0115] One or more cameras (e.g., image capture devices 122, 124, and 126) may be part of a sensing block included on the vehicle. The sensing block may include a variety of other sensors, and any or all of these sensors may be relied upon to form a sensed navigation state of the vehicle. In addition to cameras (front, side, rear, etc.), other sensors (such as radar, lidar, and acoustic sensors) may be included in the sensing block. Additionally, the sensing block may include one or more components configured to transmit and send / receive information relating to the vehicle's environment. For example, such components may include a wireless transceiver (RF, etc.) that can receive sensor-based information or any other type of information relating to the environment of the main vehicle from a source located at a distance relative to the main vehicle. This information may include sensor output information or related information received from vehicle systems other than the main vehicle. In some embodiments, this information may include information received from a remote computing device, a central server, etc. Furthermore, cameras may be configured in many different ways: a single camera unit, multiple cameras, a camera array, a long FOV, a short FOV, a wide-angle lens, a fisheye lens, etc.

[0116] System 100 or its various components can be integrated into a variety of different platforms. In some embodiments, system 100 may be included on vehicle 200, such as... Figure 2A As shown. For example, vehicle 200 may be equipped with the above-mentioned features. Figure 1 The system 100 described includes the processing unit 110 and any other components. In some embodiments, the vehicle 200 may be equipped with only a single image capture device (e.g., a camera), while in other embodiments, such as combining... Figures 2B to 2E The embodiments discussed can use multiple image capture devices. For example, Figure 2A Either of the image capture devices 122 and 124 of the vehicle 200 shown may be part of an ADAS (Advanced Driver Assistance System) imaging set.

[0117] The image capture device, included on the vehicle 200 and as part of the image acquisition unit 120, can be placed in any suitable location. In some embodiments, such as Figures 2A to 2E ,as well as Figures 3A to 3C As shown, the image capture device 122 can be located near the rearview mirror. This location provides a line of sight similar to that of the driver of vehicle 200, which can help determine what is visible and invisible to the driver. The image capture device 122 can be placed anywhere near the rearview mirror, and placing the image capture device 122 on the driver's side of the mirror can also help obtain an image representing the driver's field of vision and / or line of sight.

[0118] Other locations for the image capturing device of the image acquisition unit 120 can also be used. For example, the image capturing device 124 can be located on or within the bumper of the vehicle 200. This location is particularly suitable for image capturing devices with a wide field of view. The line of sight of the image capturing device located on the bumper may be different from that of the driver, and therefore, the bumper image capturing device and the driver may not always see the same object. The image capturing devices (e.g., image capturing devices 122, 124, and 126) can also be located in other locations. For example, the image capturing device can be located on or within one or both of the side mirrors of the vehicle 200, on the roof of the vehicle 200, on the hood of the vehicle 200, on the trunk of the vehicle 200, on the side of the vehicle 200, mounted on any window of the vehicle 200, placed behind or in front of any window of the vehicle 200, and in or near lighting equipment mounted on the front and / or rear of the vehicle 200.

[0119] In addition to the image capture equipment, vehicle 200 may also include various other components of system 100. For example, processing unit 110 may be included on vehicle 200, integrated with or separate from the vehicle's engine control unit (ECU). Vehicle 200 may also be equipped with position sensor 130 such as a GPS receiver, and may also include map database 160 and memory units 140 and 150.

[0120] As previously discussed, the wireless transceiver 172 can receive data via one or more networks (e.g., cellular networks, the Internet, etc.) and / or other networks. For example, the wireless transceiver 172 can upload data collected by the system 100 to one or more servers and download data from one or more servers. Via the wireless transceiver 172, the system 100 can receive, for example, periodic or on-demand updates to data stored in map database 160, memory 140, and / or memory 150. Similarly, the wireless transceiver 172 can upload any data from the system 100 (e.g., images captured by image acquisition unit 120, data received by position sensor 130 or other sensors, vehicle control system, etc.) and / or any data processed by processing unit 110 to one or more servers.

[0121] System 100 can upload data to a server (e.g., to the cloud) based on privacy level settings. For example, System 100 can implement privacy level settings to specify or restrict the types of data (including metadata) that can uniquely identify a vehicle and / or its driver / owner that are sent to the server. These settings can be set by a user via, for example, a wireless transceiver 172, can be initialized by factory default settings, or by data received by the wireless transceiver 172.

[0122] In some embodiments, system 100 may upload data according to a “high” privacy level, and, if configured, system 100 may transmit data (e.g., route-related location information, captured images, etc.) without any details about a specific vehicle and / or driver / owner. For example, when uploading data according to a “high” privacy setting, system 100 may not include the vehicle identification number (VIN) or the name of the vehicle’s driver or owner, and may instead transmit data (such as captured images and / or restricted route-related location information).

[0123] Other privacy levels may also be considered. For example, system 100 may transmit data to the server at an “intermediate” privacy level and may include additional information not included at a “high” privacy level, such as vehicle model and / or vehicle type (e.g., passenger vehicle, SUV, truck, etc.). In some embodiments, system 100 may upload data at a “low” privacy level. At a “low” privacy level setting, system 100 may upload data and include information sufficient to uniquely identify a particular vehicle, owner / driver, and / or part or all of the route traveled by the vehicle. For example, this “low” privacy level data may include one or more of the following: VIN, driver / owner name, vehicle origin before departure, vehicle's intended destination, vehicle model and / or vehicle type, etc.

[0124] Figure 2A It is an illustrative side view representation of an example vehicle imaging system consistent with the disclosed embodiments. Figure 2B yes Figure 2A The illustrated top view of the embodiment shown is exemplary. Figure 2B As shown, the disclosed embodiments may include a vehicle 200, which includes a system 100 in its body, the system 100 having a first image capturing device 122 located near and / or close to the driver of the rearview mirror of the vehicle 200, a second image capturing device 124 located on or in the bumper area of ​​the vehicle 200 (e.g., one of the bumper areas 210), and a processing unit 110.

[0125] like Figure 2C As shown, both image capturing devices 122 and 124 can be located near the rearview mirror of vehicle 200 and / or close to the driver. Furthermore, although... Figure 2B and Figure 2C Two image capture devices 122 and 124 are shown; it should be understood that other embodiments may include more than two image capture devices. For example, in Figure 2D and Figure 2E In the embodiment shown, the first image capture device 122, the second image capture device 124, and the third image capture device 126 are included in the system 100 of the vehicle 200.

[0126] like Figure 2D As shown, image capture device 122 may be located near and / or close to the rearview mirror of vehicle 200, and image capture devices 124 and 126 may be located above or in the bumper area of ​​vehicle 200 (e.g., one of the bumper areas 210). And as... Figure 2EAs shown, image capturing devices 122, 124, and 126 may be located near the rearview mirror and / or close to the driver's seat of vehicle 200. The disclosed embodiments are not limited to any particular number and configuration of image capturing devices, and the image capturing devices may be located in any suitable position within or on vehicle 200.

[0127] It should be understood that the disclosed embodiments are not limited to vehicles and can be applied to other scenarios. It should also be understood that the disclosed embodiments are not limited to a specific type of vehicle 200 and can be applied to all types of vehicles, including cars, trucks, trailers, and other types of vehicles.

[0128] The first image capturing device 122 may include any suitable type of image capturing device. Image capturing device 122 may include an optical axis. In one example, image capturing device 122 may include an Aptina M9V024 WVGA sensor with a global shutter. In other embodiments, image capturing device 122 may provide a resolution of 1280 × 960 pixels and may include a rolling shutter. Image capturing device 122 may include various optical elements. In some embodiments, it may include one or more lenses, for example, for providing the desired focal length and field of view for the image capturing device. In some embodiments, image capturing device 122 may be associated with a 6mm lens or a 12mm lens. In some embodiments, as shown in FIG2D, image capturing device 122 may be configured to capture images with a desired field of view (FOV) 202. For example, image capturing device 122 may be configured to have a conventional FOV, such as in the range of 40 degrees to 56 degrees, including a 46-degree FOV, a 50-degree FOV, a 52-degree FOV, or a larger FOV. Alternatively, the image capture device 122 can be configured to have a narrow FOV in the range of 23 to 40 degrees, such as a 28-degree FOV or a 36-degree FOV. Furthermore, the image capture device 122 can be configured to have a wide FOV in the range of 100 to 180 degrees. In some embodiments, the image capture device 122 may include a wide-angle bumper camera or a camera with an FOV of up to 180 degrees. In some embodiments, the image capture device 122 may be a 7.2M (megapixel) image capture device with an aspect ratio of approximately 2:1 (e.g., H×V = 3800×1900 pixels) and a horizontal FOV of approximately 100 degrees. Such an image capture device can be used instead of a three-image capture device configuration. Due to significant lens distortion, in embodiments where the image capture device uses a radially symmetrical lens, the vertical FOV of such an image capture device can be significantly less than 50 degrees. For example, such a lens may not be radially symmetrical, which would allow a vertical FOV greater than 50 degrees with a horizontal FOV of 100 degrees.

[0129] The first image capturing device 122 can acquire multiple first images of a scene associated with the vehicle 200. Each of the multiple first images can be acquired as a series of image scan lines, which can be captured using a rolling shutter. Each scan line may include multiple pixels.

[0130] The first image capturing device 122 may have a scan rate associated with the acquisition of each of the first series of image scan lines. The scan rate may refer to the rate at which the image sensor can acquire image data associated with each pixel contained in a particular scan line.

[0131] Image capturing devices 122, 124, and 126 may include any suitable type and number of image sensors, such as CCD sensors or CMOS sensors. In one embodiment, a CMOS image sensor and a rolling shutter may be employed, such that each pixel in a row is read one at a time, and scanning of each row continues on a line-by-line basis until the entire image frame has been captured. In some embodiments, rows may be captured sequentially from top to bottom relative to the frame.

[0132] In some embodiments, one or more of the image capturing devices disclosed herein (e.g., image capturing devices 122, 124, and 126) may constitute a high-resolution imager and may have a resolution of greater than 5M pixels, 7M pixels, 10M pixels, or more pixels.

[0133] The use of a rolling shutter can cause pixels in different rows to be exposed and captured at different times, which can cause distortion and other image artifacts in the captured image frame. On the other hand, when the image capture device 122 is configured to utilize global or synchronous shutter operation, all pixels can be exposed for the same amount of time and during a common exposure period. As a result, the image data in the frame collected from a system using a global shutter represents a snapshot of the entire FOV (such as FOV 202) at a specific time. In contrast, in a rolling shutter application, each row in the frame is exposed and data is captured at different times. Therefore, in an image capture device with a rolling shutter, moving objects may appear distorted. This phenomenon will be described in more detail below.

[0134] The second image capturing device 124 and the third image capturing device 126 can be any type of image capturing device. Similar to the first image capturing device 122, each of image capturing devices 124 and 126 may include an optical axis. In one embodiment, each of image capturing devices 124 and 126 may include an Aptina M9V024WVGA sensor with a global shutter. Alternatively, each of image capturing devices 124 and 126 may include a rolling shutter. Similar to image capturing device 122, image capturing devices 124 and 126 can be configured to include various lenses and optics. In some embodiments, the lenses associated with image capturing devices 124 and 126 may provide a field of view (FOV) (such as FOV 204 and 206) that is equal to or narrower than the FOV associated with image capturing device 122 (such as FOV 202). For example, image capturing devices 124 and 126 may have an FOV of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less.

[0135] Image capturing devices 124 and 126 can acquire multiple second and third images of a scene associated with vehicle 200. Each of these multiple second and third images can be acquired as a second series of image scan lines and a third series of image scan lines, which can be captured using a rolling shutter. Each scan line or line can have multiple pixels. Image capturing devices 124 and 126 can have a second scan rate and a third scan rate associated with the acquisition of each image scan line included in the second and third series.

[0136] Each image capture device 122, 124, and 126 can be positioned at any suitable location and orientation relative to the vehicle 200. The relative positions of the image capture devices 122, 124, and 126 can be selected to facilitate the fusion of information acquired from the image capture devices. For example, in some embodiments, the field of view (FOV) associated with image capture device 124 (such as FOV 204) may partially or completely overlap with the FOV associated with image capture device 122 (such as FOV 202) and the FOV associated with image capture device 126 (such as FOV 206).

[0137] Image capturing devices 122, 124, and 126 can be located at any suitable relative height on vehicle 200. In one example, a height difference may exist between image capturing devices 122, 124, and 126, which can provide sufficient parallax information to enable stereoscopic analysis. For example, as Figure 2AAs shown, the two image capture devices 122 and 124 are at different heights. For example, a lateral displacement difference may also exist between image capture devices 122, 124, and 126, providing additional parallax information for the stereo analysis of processing unit 110. Figure 2C and Figure 2D As shown, the difference in lateral displacement can be expressed by d x In some embodiments, there may be forward or backward displacement (e.g., range displacement) between image capture devices 122, 124, and 126. For example, image capture device 122 may be located 0.5 to 2 meters or more behind image capture devices 124 and / or 126. This type of displacement allows one of the image capture devices to cover potential blind spots of the other(one or more) image capture devices(s).

[0138] Image capture device 122 may have any suitable resolution capability (e.g., the number of pixels associated with an image sensor), and the resolution of one or more image sensors associated with image capture device 122 may be higher, lower, or the same as the resolution of one or more image sensors associated with image capture devices 124 and 126. In some embodiments, the one or more image sensors associated with image capture device 122 and / or image capture devices 124 and 126 may have a resolution of 640×480, 1024×768, 1280×960, or any other suitable resolution.

[0139] The frame rate (e.g., at which the image capture device acquires a set of pixel data for one image frame and then continues capturing pixel data associated with the next image frame) can be controllable. The frame rate associated with image capture device 122 can be higher, lower, or the same as the frame rate associated with image capture devices 124 and 126. The frame rates associated with image capture devices 122, 124, and 126 can depend on various factors that may affect the timing of the frame rate. For example, one or more of image capture devices 122, 124, and 126 may include selectable pixel delay periods that are applied before or after acquiring image data associated with one or more pixels of the image sensors in image capture devices 122, 124, and / or 126. Typically, image data corresponding to each pixel can be acquired based on the clock rate used for the device (e.g., one pixel per clock cycle). Additionally, in embodiments including a rolling shutter, one or more of the image capture devices 122, 124, and 126 may include a selectable horizontal blanking period applied before or after acquiring image data associated with a row of pixels of the image sensors in the image capture devices 122, 124, and / or 126. Furthermore, one or more of the image capture devices 122, 124, and 126 may include a selectable vertical blanking period applied before or after acquiring image data associated with image frames of the image capture devices 122, 124, and 126.

[0140] These timing controls enable synchronization of frame rates associated with image capture devices 122, 124, and 126, even if each has a different line scan rate. Furthermore, as will be discussed in more detail below, these selectable timing controls, along with other factors (e.g., image sensor resolution, maximum line scan rate, etc.), enable synchronization of image capture from areas where the field of view (FOV) of image capture device 122 overlaps with one or more FOVs of image capture devices 124 and 126, even if the field of view of image capture device 122 differs from the FOVs of image capture devices 124 and 126.

[0141] The frame rate timing in image capture devices 122, 124, and 126 can depend on the resolution of the associated image sensor. For example, assuming that the line scan rates are similar for two devices, if one device includes an image sensor with a resolution of 640×480 and the other device includes an image sensor with a resolution of 1280×960, then more time is required to acquire one frame of image data from the sensor with the higher resolution.

[0142] Another factor that may affect the timing of image data acquisition in image capture devices 122, 124, and 126 is the maximum line scan rate. For example, acquiring one line of image data from an image sensor included in image capture devices 122, 124, and 126 will require a certain minimum amount of time. Assuming no pixel delay period is added, this minimum amount of time for acquiring one line of image data will be related to the maximum line scan rate for a particular device. Devices offering a higher maximum line scan rate have the potential to offer a higher frame rate than devices with a lower maximum line scan rate. In some embodiments, one or more of image capture devices 124 and 126 may have a maximum line scan rate higher than the maximum line scan rate associated with image capture device 122. In some embodiments, the maximum line scan rate of image capture devices 124 and / or 126 may be 1.25, 1.5, 1.75, or 2 times or more of the maximum line scan rate of image capture device 122.

[0143] In another embodiment, image capturing devices 122, 124, and 126 may have the same maximum line scan rate, but image capturing device 122 may operate at a scan rate less than or equal to its maximum scan rate. The system may be configured such that one or more of image capturing devices 124 and 126 operate at a line scan rate equal to the line scan rate of image capturing device 122. In other instances, the system may be configured such that the line scan rate of image capturing devices 124 and / or image capturing device 126 may be 1.25, 1.5, 1.75, or 2 or more times the line scan rate of image capturing device 122.

[0144] In some embodiments, image capturing devices 122, 124, and 126 may be asymmetrical. That is, they may include cameras with different fields of view (FOV) and focal lengths. For example, the fields of view of image capturing devices 122, 124, and 126 may include any desired area of ​​the environment of vehicle 200. In some embodiments, one or more of image capturing devices 122, 124, and 126 may be configured to acquire image data from the environment in front of vehicle 200, behind vehicle 200, to the side of vehicle 200, or a combination thereof.

[0145] Furthermore, the focal length associated with each image capturing device 122, 124, and / or 126 can be selectable (e.g., by including a suitable lens, etc.) so that each device acquires an image of an object at a desired distance range relative to the vehicle 200. For example, in some embodiments, image capturing devices 122, 124, and 126 can acquire images of approaching objects within a few meters of the vehicle. Image capturing devices 122, 124, and 126 can also be configured to acquire images of objects at a greater distance from the vehicle (e.g., 25 meters, 50 meters, 100 meters, 150 meters, or more). Furthermore, the focal lengths of image capturing devices 122, 124, and 126 can be selected such that one image capturing device (e.g., image capturing device 122) can acquire images of objects relatively close to the vehicle (e.g., within 10 or 20 meters), while other image capturing devices (e.g., image capturing devices 124 and 126) can acquire images of objects farther away from the vehicle (e.g., greater than 20 meters, 50 meters, 100 meters, 150 meters, etc.).

[0146] According to some embodiments, the field of view (FOV) of one or more image capture devices 122, 124, and 126 may have a wide angle. For example, an FOV of 140 degrees may be advantageous, especially for image capture devices 122, 124, and 126 that can be used to capture images of areas near vehicle 200. For example, image capture device 122 may be used to capture images of areas to the right or left of vehicle 200, and in these embodiments, it may be desirable for image capture device 122 to have a wide FOV (e.g., at least 140 degrees).

[0147] The field of view associated with each of the image capturing devices 122, 124, and 126 can depend on its respective focal length. For example, as the focal length increases, the corresponding field of view decreases.

[0148] Image capturing devices 122, 124, and 126 can be configured to have any suitable field of view. In one particular example, image capturing device 122 can have a horizontal FOV of 46 degrees, image capturing device 124 can have a horizontal FOV of 23 degrees, and image capturing device 126 can have a horizontal FOV between 23 degrees and 46 degrees. In another example, image capturing device 122 can have a horizontal FOV of 52 degrees, image capturing device 124 can have a horizontal FOV of 26 degrees, and image capturing device 126 can have a horizontal FOV between 26 degrees and 52 degrees. In some embodiments, the ratio of the FOV of image capturing device 122 to the FOV of image capturing device 124 and / or image capturing device 126 can vary from 1.5 to 2.0. In other embodiments, this ratio can vary between 1.25 and 2.25.

[0149] System 100 may be configured such that the field of view of image capture device 122 at least partially or completely overlaps with the field of view of image capture devices 124 and / or 126. In some embodiments, system 100 may be configured such that the field of view of image capture devices 124 and 126, for example, falls within (e.g., is narrower than) the field of view of image capture device 122 and shares a common center with the field of view of image capture device 122. In other embodiments, image capture devices 122, 124, and 126 may capture adjacent FOVs, or may have partial overlap in their FOVs. In some embodiments, the field of view of image capture devices 122, 124, and 126 may be aligned such that the center of the narrower FOV image capture device 124 and / or 126 may be located in the lower half of the field of view of the wider FOV device 122.

[0150] Figure 2F This is a schematic representation of an example vehicle control system consistent with the disclosed embodiments. (As...) Figure 2F As indicated, vehicle 200 may include a throttle control system 220, a braking system 230, and a steering system 240. System 100 may provide inputs (e.g., control signals) to one or more of the throttle control system 220, braking system 230, and steering system 240 via one or more data links (e.g., any wired and / or wireless link for transmitting data). For example, based on analysis of images acquired by image capture devices 122, 124, and / or 126, system 100 may provide control signals to one or more of the throttle control system 220, braking system 230, and steering system 240 to navigate vehicle 200 (e.g., by inducing acceleration, steering, lane changes, etc.). Furthermore, system 100 may receive inputs from one or more of the throttle control system 220, braking system 230, and steering system 240 indicative of the operating conditions of vehicle 200 (e.g., speed, whether vehicle 200 is braking and / or steering, etc.). The following is combined with... Figures 4 to 7 Further details will be provided.

[0151] like Figure 3AAs shown, vehicle 200 may also include a user interface 170 for interacting with the driver or passengers of vehicle 200. For example, the user interface 170 in a vehicle application may include a touchscreen 320, a knob 330, a button 340, and a microphone 350. The driver or passengers of vehicle 200 may also interact with system 100 using handles (e.g., located on or near the steering wheel of vehicle 200, including, for example, a turn signal handle), buttons (e.g., located on the steering wheel of vehicle 200), etc. In some embodiments, microphone 350 may be located adjacent to rearview mirror 310. Similarly, in some embodiments, image capturing device 122 may be located near rearview mirror 310. In some embodiments, user interface 170 may also include one or more speakers 360 (e.g., speakers of a vehicle audio system). For example, system 100 may provide various notifications (e.g., alarms) via speaker 360.

[0152] Figures 3B to 3D This is an example of an example camera mount 370 configured consistent with the disclosed embodiments, located behind a rearview mirror (e.g., rearview mirror 310) and opposite the vehicle's windshield. Figure 3B As shown, camera mount 370 may include image capturing devices 122, 124, and 126. Image capturing devices 124 and 126 may be located behind a light shield 380, wherein the light shield 380 may be flush relative to a vehicle windshield and comprises a composite of a film and / or anti-reflective material. For example, the light shield 380 may be positioned such that it is aligned with a vehicle windshield having a matching bevel. In some embodiments, each of image capturing devices 122, 124, and 126 may be located behind the light shield 380, for example in Figure 3D The embodiments described herein are not limited to any particular configuration of the image capturing devices 122, 124 and 126, the camera mount 370 and the light shield 380. Figure 3C yes Figure 3B The image shown is an example of a camera mount 370 from a frontal view.

[0153] As those skilled in the art who benefit from this disclosure will understand, many variations and / or modifications can be made to the disclosed embodiments. For example, not all components are necessary for the operation of system 100. Furthermore, any component may be located in any suitable part of system 100 and components may be rearranged into various configurations while providing the functionality of the disclosed embodiments. Thus, the foregoing configurations are exemplary, and regardless of the configurations discussed above, system 100 can provide a wide range of functions to analyze the surroundings of vehicle 200 and navigate vehicle 200 in response to that analysis.

[0154] As discussed in more detail below and according to various disclosed embodiments, system 100 can provide various features related to autonomous driving and / or driver assistance technologies. For example, system 100 can analyze image data, location data (e.g., GPS location information), map data, speed data, and / or data from sensors included in vehicle 200. System 100 can collect data from, for example, image acquisition unit 120, position sensor 130, and other sensors for analysis. Furthermore, system 100 can analyze the collected data to determine whether vehicle 200 should take a certain action, and then automatically take the determined action without human intervention. For example, when vehicle 200 is navigating without human intervention, system 100 can automatically control the braking, acceleration, and / or steering of vehicle 200 (e.g., by sending control signals to one or more of throttle adjustment system 220, braking system 230, and steering system 240). Additionally, system 100 can analyze the collected data and issue warnings and / or alerts to vehicle occupants based on the analysis of the collected data. Additional details regarding various embodiments of system 100 are provided below.

[0155] Forward Multi-Imaging System

[0156] As discussed above, system 100 can provide driver assistance functions using a multi-camera system. The multi-camera system may use one or more cameras facing forward of the vehicle. In other embodiments, the multi-camera system may include one or more cameras facing the side or rear of the vehicle. In one embodiment, for example, system 100 may use a dual-camera imaging system, wherein a first camera and a second camera (e.g., image capture devices 122 and 124) may be located at the front and / or side of the vehicle (e.g., vehicle 200). Other camera configurations are consistent with the disclosed embodiments, and the configurations disclosed herein are exemplary. For example, system 100 may include a configuration of any number of cameras (e.g., one, two, three, four, five, six, seven, eight, etc.). Furthermore, system 100 may include a camera “cluster.” For example, a camera cluster (including any suitable number of cameras, such as one, four, eight, etc.) may be forward-facing relative to the vehicle, or may be oriented in any other direction (e.g., rearward, sideways, angled, etc.). Therefore, system 100 may include multiple camera clusters, each oriented in a specific direction to capture images from a specific area of ​​the vehicle environment.

[0157] The first camera may have a field of view that is larger than, smaller than, or partially overlaps with the field of view of the second camera. Furthermore, the first camera may be connected to a first image processor to perform monocular image analysis on images provided by the first camera, and the second camera may be connected to a second image processor to perform monocular image analysis on images provided by the second camera. The outputs of the first and second image processors (e.g., processed information) may be combined. In some embodiments, the second image processor may receive images from both the first and second cameras to perform stereo analysis. In another embodiment, system 100 may use a three-camera imaging system, where each camera has a different field of view. Therefore, such a system can make decisions based on information obtained from objects located at varying distances in front of and to the sides of the vehicle. Reference to monocular image analysis may refer to instances of image analysis performed based on images captured from a single viewpoint (e.g., from a single camera). Stereo image analysis may refer to instances of image analysis performed based on two or more images captured utilizing one or more variations of image capture parameters. For example, images captured suitable for performing stereo image analysis may include images captured from two or more different locations, from different fields of view, using different focal lengths, or with parallax information, etc.

[0158] For example, in one embodiment, system 100 may use image capture devices 122 to 126 to implement a three-camera configuration. In this configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees or other values ​​selected from the range of approximately 20 to 45 degrees), image capture device 124 may provide a wide field of view (e.g., 150 degrees or other values ​​selected from the range of approximately 100 to approximately 180 degrees), and image capture device 126 may provide an intermediate field of view (e.g., 46 degrees or other values ​​selected from the range of approximately 35 to approximately 60 degrees). In some embodiments, image capture device 126 may serve as a main camera or a base camera. Image capture devices 122 to 126 may be located behind rearview mirror 310 and substantially side-by-side (e.g., 6 cm apart). Furthermore, in some embodiments, as discussed above, one or more of image capture devices 122 to 126 may be mounted behind a sun visor 380 flush with the windshield of vehicle 200. This shielding can work to reduce the impact of any reflections from inside the vehicle on the image capture devices 122 to 126.

[0159] In another embodiment, as described above... Figure 3B and 3CThe wide field-of-view camera discussed above (e.g., image capture device 124 in the example above) can be mounted below the narrow field-of-view camera and the main field-of-view camera (e.g., image capture devices 122 and 126 in the example above). This configuration provides a free line of sight from the wide field-of-view camera. To reduce reflections, the camera can be mounted close to the windshield of the vehicle 200, and a polarizer can be included on the camera to dampen reflected light.

[0160] A three-camera system can provide certain performance characteristics. For example, some embodiments may include the ability to verify object detection by one camera based on detection results from another camera. In the three-camera configuration discussed above, processing unit 110 may include, for example, three processing devices (e.g., three EyeQ series processor chips as discussed above), each dedicated to processing images captured by one or more of the image capture devices 122 to 126.

[0161] In a three-camera system, the first processing device can receive images from both the main camera and the narrow field-of-view camera, and perform visual processing on the narrow FOV camera to detect, for example, other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the first processing device can calculate the pixel parallax between the images from the main camera and the narrow camera, and create a 3D reconstruction of the vehicle 200's environment. The first processing device can then combine the 3D reconstruction with 3D map data, or combine the 3D reconstruction with 3D information calculated based on information from the other camera.

[0162] The second processing device can receive images from the main camera and perform visual processing to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the second processing device can calculate camera displacement and, based on this displacement, calculate the parallax of pixels between consecutive images and create a 3D reconstruction of the scene (e.g., a structure from motion). The second processing device can then send the 3D-reconstructed structure from motion to the first processing device for combination with the stereoscopic 3D images.

[0163] The third processing device can receive images from a wide field of view (FOV) camera and process them to detect vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. The third processing device can also execute additional processing instructions to analyze the images to identify moving objects, such as vehicles changing lanes or pedestrians.

[0164] In some embodiments, allowing the image-based information stream to be captured and processed independently can provide opportunities for redundancy in the system. This redundancy may include, for example, using a first image capture device and images processed from that device to verify and / or supplement information obtained by capturing and processing image information from at least a second image capture device.

[0165] In some embodiments, system 100 uses two image capture devices (e.g., image capture devices 122 and 124) to provide navigation assistance to vehicle 200, and uses a third image capture device (e.g., image capture device 126) to provide redundancy and verify the analysis of data received from the other two image capture devices. For example, in this configuration, image capture devices 122 and 124 can provide images for stereo analysis by system 100 to navigate vehicle 200, while image capture device 126 can provide images for monocular analysis by system 100 to provide redundancy and verification of information based on images captured from image capture devices 122 and / or 124. That is, image capture device 126 (and the corresponding processing device) can be considered as providing a redundant subsystem for providing checks on the analysis obtained from image capture devices 122 and 124 (e.g., to provide an automatic emergency braking (AEB) system). Furthermore, in some embodiments, the redundancy and verification of received data may be supplemented based on information received from one or more sensors (e.g., radar, lidar, acoustic sensors, information received from one or more transceivers outside the vehicle, etc.).

[0166] Those skilled in the art will recognize that the camera configurations, camera placements, number of cameras, camera positions, etc., described above are merely examples. These components, and other components described regarding the overall system, can be assembled and used in a variety of different configurations without departing from the scope of the disclosed embodiments. Further details regarding the use of a multi-camera system to provide driver assistance and / or autonomous vehicle functionality are as follows.

[0167] Figure 4 This is an exemplary functional block diagram of memory 140 and / or memory 150 that can store / program instructions for performing one or more operations consistent with embodiments of this disclosure. Although memory 140 is referred to below, those skilled in the art will recognize that instructions can be stored in memory 140 and / or memory 150.

[0168] like Figure 4As shown, memory 140 may store monocular image analysis module 402, stereo image analysis module 404, velocity and acceleration module 406, and navigation response module 408. The disclosed embodiments are not limited to any particular configuration of memory 140. Furthermore, application processor 180 and / or image processor 190 may execute instructions stored in any of the modules 402 to 408 contained in memory 140. Those skilled in the art will understand that in the following discussion, references to processing unit 110 may individually or uniformly refer to application processor 180 and image processor 190. Therefore, any of the following processing steps may be performed by one or more processing devices.

[0169] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis on a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensing information (e.g., information from radar) to perform monocular image analysis. As combined below... Figures 5A to 5D As described, the monocular image analysis module 402 may include instructions for detecting a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other features associated with the vehicle's environment. Based on this analysis, system 100 (e.g., via processing unit 110) may induce one or more navigation responses in vehicle 200, such as steering, lane changes, changes in acceleration, etc., as discussed below in conjunction with navigation response module 408.

[0170] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis on a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensing information (e.g., information from radar, lidar, etc.) to perform monocular image analysis. As combined below... Figures 5A to 5D As described, the monocular image analysis module 402 may include instructions for detecting a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other features associated with the vehicle's environment. Based on this analysis, system 100 (e.g., via processing unit 110) may induce one or more navigation responses in vehicle 200, such as steering, lane changes, changes in acceleration, etc., as discussed below in conjunction with determining navigation responses.

[0171] In one embodiment, the stereo image analysis module 404 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform stereo image analysis on a first set and a second set of images acquired by a combination of image capture devices selected from any combination of image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the first set and the second set of images with additional sensing information (e.g., information from radar) to perform stereo image analysis. For example, the stereo image analysis module 404 may include instructions for performing stereo image analysis based on the first set of images acquired by image capture device 124 and the second set of images acquired by image capture device 126. (See below for further details.) Figure 6 As described, the stereo image analysis module 404 may include instructions for detecting a set of features within a first and second set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, etc. Based on this analysis, the processing unit 110 may induce one or more navigation responses in the vehicle 200, such as steering, lane changes, changes in acceleration, etc., as discussed below in conjunction with the navigation response module 408. Furthermore, in some embodiments, the stereo image analysis module 404 may implement techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems.

[0172] In one embodiment, the speed and acceleration module 406 may store software configured to analyze data received from one or more computing and electromechanical devices in the vehicle 200 configured to cause changes in the speed and / or acceleration of the vehicle 200. For example, the processing unit 110 may execute instructions associated with the speed and acceleration module 406 to calculate a target speed of the vehicle 200 based on data obtained from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include, for example, target position, speed and / or acceleration, the position and / or speed of the vehicle 200 relative to nearby vehicles, pedestrians, or road objects, and positional information of the vehicle 200 relative to lane markings on the road. Furthermore, the processing unit 110 may calculate the target speed of the vehicle 200 based on sensor inputs (e.g., information from radar) and inputs from other systems of the vehicle 200 (such as the throttle control system 220, braking system 230, and / or steering system 240). Based on the calculated target rate, the processing unit 110 can transmit electronic signals to the throttle adjustment system 220, braking system 230 and / or steering system 240 of the vehicle 200, for example by physically pressing the brake or releasing the accelerator of the vehicle 200 to trigger changes in speed and / or acceleration.

[0173] In one embodiment, the navigation response module 408 may store software that can be executed by the processing unit 110 to determine the desired navigation response based on data obtained from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. This data may include position and rate information associated with nearby vehicles, pedestrians, and road objects, target position information of vehicle 200, etc. Additionally, in some embodiments, the navigation response may be (partially or entirely) based on map data, the predetermined position of vehicle 200, and / or the relative velocity or relative acceleration between vehicle 200 and one or more objects detected from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. The navigation response module 408 may also determine the desired navigation response based on sensor inputs (e.g., information from radar) and inputs from other systems of vehicle 200 (such as the throttle control system 220, braking system 230, and steering system 240 of vehicle 200). Based on the desired navigation response, the processing unit 110 can transmit electronic signals to the throttle adjustment system 220, braking system 230, and steering system 240 of the vehicle 200 to trigger the desired navigation response, for example, by turning the steering wheel of the vehicle 200 to achieve a predetermined angle of rotation. In some embodiments, the processing unit 110 can use the output of the navigation response module 408 (e.g., the desired navigation response) as input to the execution of the speed and acceleration module 406 to calculate the change in the rate of the vehicle 200.

[0174] Furthermore, any of the modules disclosed herein (e.g., modules 402, 404, and 406) can implement techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems.

[0175] Figure 5A This is a flowchart illustrating an example processing 500A for inducing one or more navigation responses based on monocular image analysis, consistent with the disclosed embodiments. In step 510, processing unit 110 may receive multiple images via data interface 128 between processing unit 110 and image acquisition unit 120. For example, a camera included in image acquisition unit 120 (such as image capture device 122 with a field of view 202) may capture multiple images of a region in front of vehicle 200 (e.g., or to the side or rear of the vehicle) and transmit them to processing unit 110 via a data connection (e.g., digital, wired, USB, wireless, Bluetooth, etc.). In step 520, processing unit 110 may execute monocular image analysis module 402 to analyze the multiple images, as described below. Figures 5B to 5DIn a further detailed description, by performing this analysis, the processing unit 110 can detect a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, etc.

[0176] In step 520, processing unit 110 may also execute monocular image analysis module 402 to detect various road hazards, such as components of truck tires, fallen road signs, loose cargo, small animals, etc. Road hazards may vary in structure, shape, size, and color, which may make their detection more difficult. In some embodiments, processing unit 110 may execute monocular image analysis module 402 to perform multi-frame analysis on the multiple images to detect road hazards. For example, processing unit 110 may estimate camera motion between consecutive image frames and calculate disparity in pixels between frames to construct a 3D map of the road. Processing unit 110 can then use this 3D map to detect road surfaces and hazards present on the road surface.

[0177] In step 530, processing unit 110 may execute navigation response module 408 based on the analysis performed in step 520 and as described above. Figure 4 The described techniques induce one or more navigation responses. Navigation responses may include, for example, steering, lane changes, acceleration changes, etc. In some embodiments, processing unit 110 may induce one or more navigation responses using data obtained from the execution of speed and acceleration module 406. Furthermore, multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof. For example, processing unit 110 may induce a lane change and then acceleration by, for example, sequentially transmitting control signals to the steering system 240 and throttle adjustment system 220 of vehicle 200. Alternatively, processing unit 110 may induce a lane change and then acceleration by, for example, simultaneously transmitting control signals to the braking system 230 and steering system 240 of vehicle 200.

[0178] Figure 5BThis is a flowchart illustrating an example process 500B for detecting one or more vehicles and / or pedestrians in a set of images, consistent with the disclosed embodiments. Processing unit 110 may implement process 500B by executing monocular image analysis module 402. In step 540, processing unit 110 may determine a set of candidate objects representing possible vehicles and / or pedestrians. For example, processing unit 110 may scan one or more images, compare the images with one or more predetermined patterns, and identify possible locations within each image that may contain objects of interest (e.g., vehicles, pedestrians, or portions thereof). The predetermined patterns may be designed to achieve a high "false hit" rate and a low "missed" rate. For example, processing unit 110 may use a low similarity threshold on the predetermined patterns to identify candidate objects as possible vehicles or pedestrians. Doing so allows processing unit 110 to reduce the likelihood of missing (e.g., not identifying) candidate objects representing vehicles or pedestrians.

[0179] In step 542, processing unit 110 may filter the group of candidate objects based on classification criteria to exclude certain candidates (e.g., irrelevant or less relevant objects). Such criteria can be derived from various attributes associated with object types stored in a database (e.g., a database stored in memory 140). Attributes may include object shape, size, texture, location (e.g., relative to vehicle 200), etc. Therefore, processing unit 110 may use one or more sets of criteria to reject false candidates from the group of candidate objects.

[0180] In step 544, processing unit 110 may analyze multiple frames of images to determine whether an object in the group of candidate objects represents a vehicle and / or a pedestrian. For example, processing unit 110 may track detected candidate objects across consecutive frames and accumulate frame-by-frame data associated with the detected objects (e.g., size, position relative to vehicle 200, etc.). Furthermore, processing unit 110 may estimate parameters of the detected objects and compare the object's frame-by-frame position data with its predicted position.

[0181] In step 546, processing unit 110 can construct a set of measurements for the detected objects. Such measurements may include, for example, position, velocity, and acceleration values ​​(relative to vehicle 200) associated with the detected objects. In some embodiments, processing unit 110 can construct these measurements based on a series of time-based observational estimation techniques, such as Kalman filters or linear quadratic estimation (LQE), and / or based on modeling data available for different object types (e.g., cars, trucks, pedestrians, bicycles, road signs, etc.). The Kalman filter may be based on a measurement of the object's scale, where the scale measurement is proportional to the time of collision (e.g., the amount of time it takes for vehicle 200 to arrive at the object). Therefore, by performing steps 540 through 546, processing unit 110 can identify vehicles and pedestrians appearing within the set of captured images and obtain information associated with those vehicles and pedestrians (e.g., position, velocity, size). Based on this identification and the obtained information, processing unit 110 can induce one or more navigation responses in vehicle 200, as described above. Figure 5A As described.

[0182] In step 548, processing unit 110 may perform optical flow analysis on one or more images to reduce the likelihood of detecting "false hits" and missing candidate objects representing vehicles or pedestrians. Optical flow analysis may refer to, for example, analyzing motion patterns relative to vehicle 200, associated with other vehicles and pedestrians, and distinct from road surface motion in one or more images. Processing unit 110 can calculate the motion of candidate objects by observing different positions of the objects across multiple image frames captured at different times. Processing unit 110 can use this position and time value as input to a mathematical model used to calculate the motion of candidate objects. Therefore, optical flow analysis can provide an alternative method for detecting vehicles and pedestrians near vehicle 200. Processing unit 110 may perform optical flow analysis in conjunction with steps 540 to 546 to provide redundancy in detecting vehicles and pedestrians and improve the reliability of system 100.

[0183] Figure 5CThis is a flowchart illustrating an example process 500C for detecting road markings and / or lane geometry information in a set of images, consistent with the disclosed embodiments. Processing unit 110 may implement process 500C by executing monocular image analysis module 402. In step 550, processing unit 110 may detect a set of objects by scanning one or more images. To detect segments of lane markings, lane geometry information, and other relevant road markings, processing unit 110 may filter the set of objects to exclude those determined to be irrelevant (e.g., potholes, pebbles, etc.). In step 552, processing unit 110 may group segments detected in step 550 that belong to the same road marking or lane marking together. Based on this grouping, processing unit 110 may generate a model, such as a mathematical model, representing the detected segments.

[0184] In step 554, processing unit 110 may construct a set of measurements associated with the detected segment. In some embodiments, processing unit 110 may create a projection of the detected segment from the image plane to a real-world plane. This projection may be characterized using a cubic polynomial with coefficients corresponding to physical properties such as the detected road's location, slope, curvature, and derivative of curvature. In generating this projection, processing unit 110 may consider variations in the road surface, as well as the pitch and roll rates associated with vehicle 200. Furthermore, processing unit 110 may model the road elevation by analyzing positional and motion cues appearing on the road surface. Additionally, processing unit 110 may estimate the pitch and roll rates associated with vehicle 200 by tracking a set of feature points in one or more images.

[0185] In step 556, processing unit 110 can perform multi-frame analysis, for example, by tracking the detected segments across consecutive image frames and accumulating frame-by-frame data associated with the detected segments. Because processing unit 110 performs multi-frame analysis, the set of measurements constructed in step 554 can become more reliable and associated with increasingly higher confidence levels. Therefore, by performing steps 550 to 556, processing unit 110 can identify road markings appearing in the set of captured images and obtain lane geometry information. Based on this identification and the obtained information, processing unit 110 can induce one or more navigation responses in vehicle 200, as described above. Figure 5A As described.

[0186] In step 558, processing unit 110 may consider additional information sources to further generate a safety model of vehicle 200 in its surrounding environment. Processing unit 110 can use this safety model to define an environment in which system 100 can safely perform autonomous control of vehicle 200. To generate this safety model, in some embodiments, processing unit 110 may consider the positions and movements of other vehicles, detected curbs and guardrails, and / or general road shape descriptions extracted from map data (such as data from map database 160). By considering additional information sources, processing unit 110 can provide redundancy for detecting road markings and lane geometry, and increase the reliability of system 100.

[0187] Figure 5D This is a flowchart illustrating an example process 500D for detecting traffic lights in a set of images, consistent with the disclosed embodiments. Processing unit 110 may implement process 500D by executing monocular image analysis module 402. In step 560, processing unit 110 may scan the set of images and identify objects appearing in the images at locations that may contain traffic lights. For example, processing unit 110 may filter the identified objects to construct a set of candidate objects, excluding those that could not possibly correspond to traffic lights. Filtering may be based on various attributes associated with traffic lights, such as shape, size, texture, and location (e.g., relative to vehicle 200). Such attributes may be based on multiple examples of traffic lights and traffic control signals and stored in a database. In some embodiments, processing unit 110 may perform multi-frame analysis on the set of candidate objects reflecting possible traffic lights. For example, processing unit 110 may track candidate objects across consecutive image frames, estimate the real-world location of the candidate objects, and filter out moving objects (which are unlikely to be traffic lights). In some embodiments, processing unit 110 may perform color analysis on the candidate objects and identify the relative positions of detected colors appearing within possible traffic lights.

[0188] In step 562, processing unit 110 can analyze the geometry of the intersection. This analysis can be based on any combination of: (i) the number of lanes detected on either side of vehicle 200, (ii) markings detected on the road (such as arrow markings), and (iii) a description of the intersection extracted from map data (e.g., data from map database 160). Processing unit 110 can use information obtained from the execution of monocular analysis module 402 for this analysis. Furthermore, processing unit 110 can determine the correspondence between traffic lights detected in step 560 and lanes appearing near vehicle 200.

[0189] In step 564, as vehicle 200 approaches the intersection, processing unit 110 can update the confidence level associated with the analyzed intersection geometry and the detected traffic lights. For example, comparing the number of traffic lights estimated to be present at the intersection with the actual number present at the intersection may affect the confidence level. Therefore, based on this confidence level, processing unit 110 can delegate control to the driver of vehicle 200 to improve safety conditions. By performing steps 560 to 564, processing unit 110 can identify traffic lights appearing in the set of captured images and analyze intersection geometry information. Based on this identification and analysis, processing unit 110 can induce one or more navigation responses in vehicle 200, as described above. Figure 5A As described.

[0190] Figure 5E This is a flowchart illustrating an example process 500E for inducing one or more navigation responses in a vehicle based on a vehicle path, consistent with the disclosed embodiments. In step 570, processing unit 110 may construct an initial vehicle path associated with vehicle 200. The vehicle path can be represented using a set of points expressed in coordinates (x, z), and the distance d between any two points in this set is... i The distance can fall within a range of 1 to 5 meters. In one embodiment, processing unit 110 can use two polynomials, such as a left-road polynomial and a right-road polynomial, to construct an initial vehicle path. Processing unit 110 can calculate the geometric midpoint between the two polynomials and offset each point included in the resulting vehicle path by a predetermined offset (e.g., intelligent lane offset), if any (zero offset could correspond to driving in the middle of the lane). This offset can be in a direction perpendicular to the line segment between any two points in the vehicle path. In another embodiment, processing unit 110 can use a polynomial and an estimated lane width to offset each point in the vehicle path by half the estimated lane width plus a predetermined offset (e.g., intelligent lane offset).

[0191] In step 572, processing unit 110 may update the vehicle path constructed in step 570. Processing unit 110 may use a higher resolution to reconstruct the vehicle path constructed in step 570, such that the distance d between two points in the set representing the vehicle path is... k less than the above distance d i For example, the distance d kIt can fall within the range of 0.1 to 0.3 meters. The processing unit 110 can reconstruct the vehicle path using the parabolic spline algorithm, which can generate a cumulative distance vector S corresponding to the total length of the vehicle path (i.e., based on the set of points representing the vehicle path).

[0192] In step 574, processing unit 110 can determine the look-ahead point (expressed in coordinates as (x, y, x) based on the updated vehicle path constructed in step 572. l ,z l Processing unit 110 can extract a forward-looking point from the accumulated distance vector S, and this forward-looking point can be associated with a forward-looking distance and a forward-looking time. The forward-looking distance can have a lower limit ranging from 10 meters to 20 meters and can be calculated as the product of the vehicle 200's speed and the forward-looking time. For example, as the vehicle 200's speed decreases, the forward-looking distance can also decrease (e.g., until it reaches the lower limit). The forward-looking time can range from 0.5 to 1.5 seconds and can be inversely proportional to the gain of one or more control loops associated with the navigation response in the vehicle 200, such as the heading error tracking control loop. For example, the gain of this heading error tracking control loop can depend on the bandwidth of the yaw rate loop, the steering actuator loop, the vehicle's lateral dynamics, etc. Therefore, the higher the gain of the heading error tracking control loop, the shorter the forward-looking time.

[0193] In step 576, processing unit 110 can determine the heading error and yaw rate commands based on the foresight point determined in step 574. Processing unit 110 can do this by calculating the arctangent of the foresight point, for example, arctan(x). l ,z l The heading error is determined by the processing unit 110. The processing unit 110 can determine the yaw rate command as the product of the heading error and the high-level control gain. If the forward look-ahead distance is not at the lower limit, the high-level control gain can be equal to: (2 / forward look-ahead time). Otherwise, the high-level control gain can be equal to: (2 × vehicle speed 200 / forward look-ahead distance).

[0194] Figure 5F This is a flowchart illustrating an example process 500F for determining whether a vehicle ahead is changing lanes, consistent with the disclosed embodiments. In step 580, processing unit 110 may determine navigation information associated with the vehicle ahead (e.g., a vehicle traveling in front of vehicle 200). For example, processing unit 110 may use the above combination... Figure 5A and Figure 5BThe described technology determines the position, speed (e.g., direction and rate), and / or acceleration of a vehicle ahead. Processing unit 110 can also utilize a combination of the above. Figure 5E The described technique determines one or more road polynomials, forward viewpoints (associated with vehicle 200), and / or a snail trail (e.g., a set of points describing the path taken by the vehicle ahead).

[0195] In step 582, processing unit 110 may analyze the navigation information determined in step 580. In one embodiment, processing unit 110 may calculate the distance between the tracking trajectory and the road polynomial (e.g., along the trajectory). If the variation in this distance along the trajectory exceeds a predetermined threshold (e.g., 0.1 to 0.2 meters on a straight road, 0.3 to 0.4 meters on a moderately curved road, and 0.5 to 0.6 meters on a sharp turn), processing unit 110 may determine that the vehicle ahead is likely changing lanes. In the case where multiple vehicles are detected traveling in front of vehicle 200, processing unit 110 may compare the tracking trajectory associated with each vehicle. Based on this comparison, processing unit 110 may determine that a vehicle whose tracking trajectory does not match the tracking trajectories of other vehicles is likely changing lanes. Processing unit 110 may additionally compare the curvature of the tracking trajectory (associated with the vehicle ahead) with the expected curvature of the road segment in which the vehicle ahead is traveling. The desired curvature can be extracted from map data (e.g., data from map database 160), from road polynomials, from tracking trajectories of other vehicles, from existing knowledge about the road, etc. If the difference between the curvature of the tracking trajectory and the desired curvature of the road segment exceeds a predetermined threshold, the processing unit 110 can determine that the vehicle ahead is likely changing lanes.

[0196] In another embodiment, processing unit 110 may compare the instantaneous position of the vehicle ahead with a forward viewpoint (as associated with vehicle 200) over a specific time period (e.g., 0.5 to 1.5 seconds). If the distance between the instantaneous position of the vehicle ahead and the forward viewpoint changes during this specific time period, and the cumulative sum of the changes exceeds a predetermined threshold (e.g., 0.3 to 0.4 meters on a straight road, 0.7 to 0.8 meters on a moderately curved road, and 1.3 to 1.7 meters on a sharp turn), then processing unit 110 may determine that the vehicle ahead is likely changing lanes. In another embodiment, processing unit 110 may analyze the geometry of the tracking trajectory by comparing the lateral distance traveled along the tracking trajectory with the desired curvature of the tracking trajectory. The desired radius of curvature can be determined according to the formula: (δ... z 2 +δ x 2 ) / 2 / (δx ), where δ x Indicates the lateral distance traveled and δ z The longitudinal distance traveled is represented. If the difference between the lateral distance traveled and the desired curvature exceeds a predetermined threshold (e.g., 500 to 700 meters), the processing unit 110 can determine that the vehicle ahead is likely changing lanes. In another embodiment, the processing unit 110 can analyze the position of the vehicle ahead. If the position of the vehicle ahead obscures the road polynomial (e.g., the vehicle ahead is overlaid on the road polynomial), the processing unit 110 can determine that the vehicle ahead is likely changing lanes. If the position of the vehicle ahead is such that another vehicle is detected in front of the vehicle ahead and the tracking trajectories of the two vehicles are not parallel, the processing unit 110 can determine that the (closer) vehicle ahead is likely changing lanes.

[0197] In step 584, processing unit 110 may determine whether the vehicle 200 ahead is changing lanes based on the analysis performed in step 582. For example, processing unit 110 may make this determination based on a weighted average of the various analyses performed in step 582. In this approach, for example, a decision by processing unit 110 based on a particular type of analysis that the vehicle ahead is likely to change lanes may be assigned a value "1" (and "0" to indicate that the vehicle ahead is unlikely to change lanes). Different analyses performed in step 582 may be assigned different weights, and the disclosed embodiments are not limited to any particular combination of analysis and weights. Furthermore, in some embodiments, the analysis may utilize a trained system (e.g., a machine learning or deep learning system) that may estimate the future path ahead of the vehicle's current position, for example, based on an image captured at the current location.

[0198] Figure 6 This is a flowchart illustrating an example process 600 for inducing one or more navigation responses based on stereoscopic image analysis, consistent with the disclosed embodiments. In step 610, processing unit 110 may receive first and second plurality of images via data interface 128. For example, a camera included in image acquisition unit 120 (such as image capture devices 122 and 124 having fields of view 202 and 204) may capture first and second plurality of images of an area in front of vehicle 200 and transmit them to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first and second plurality of images via two or more data interfaces. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0199] In step 620, the processing unit 110 can execute the stereo image analysis module 404 to perform stereo image analysis on the first and second plurality of images to create a 3D map of the road in front of the vehicle and detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, etc. Stereo image analysis can be performed in a manner similar to the above combination. Figures 5A-5D The steps described are performed in a manner consistent with the description. For example, processing unit 110 may execute stereo image analysis module 404 to detect candidate objects (e.g., vehicles, pedestrians, road markings, traffic lights, road hazards, etc.) in the first and second plurality of images, filter out subsets of candidate objects based on various criteria, and perform multi-frame analysis, construct measurements, and determine confidence levels for the remaining candidate objects. In performing the above steps, processing unit 110 may consider information from both the first and second plurality of images, rather than information from a single set of images. For example, processing unit 110 may analyze differences in pixel-level data (or other subsets of data from the two streams of captured images) of candidate objects appearing in both the first and second plurality of images. As another example, processing unit 110 may estimate the position and / or velocity of a candidate object (e.g., relative to vehicle 200) by observing whether a candidate object appears in one of the plurality of images but not in another, or by other differences that may exist relative to objects appearing in the two image streams. For example, the position, velocity, and / or acceleration relative to vehicle 200 may be determined based on features such as trajectory, position, and movement characteristics associated with an object appearing in one or both of the image streams.

[0200] In step 630, processing unit 110 may execute navigation response module 408 to perform the analysis performed in step 620 and as described above. Figure 4 The described technology induces one or more navigation responses in vehicle 200. Navigation responses may include, for example, steering, lane changing, acceleration change, speed change, braking, etc. In some embodiments, processing unit 110 may use data obtained from the execution of speed and acceleration module 406 to induce the one or more navigation responses. Furthermore, multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof.

[0201] Figure 7This is a flowchart illustrating an example process 700 consistent with the disclosed embodiments for evoking one or more navigation responses based on the analysis of three sets of images. In step 710, processing unit 110 may receive first, second, and third plurality of images via data interface 128. For example, cameras included in image acquisition unit 120 (such as image capture devices 122, 124, and 126 having fields of view 202, 204, and 206) may capture first, second, and third plurality of images of areas in front of and / or to the sides of vehicle 200 and transmit them to processing unit 110 via digital connections (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive first, second, and third plurality of images via three or more data interfaces. For example, each of image capture devices 122, 124, and 126 may have an associated data interface for transmitting data to processing unit 110. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0202] In step 720, the processing unit 110 can analyze the first, second, and third images to detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and road hazards. This analysis can be performed in a manner similar to the combination described above. Figures 5A-5D and Figure 6 The steps described are performed in a manner that allows for execution. For example, processing unit 110 may perform monocular image analysis on each of the first, second, and third plurality of images (e.g., via execution by monocular image analysis module 402 and based on the above combination). Figures 5A-5D The steps described above). Alternatively, processing unit 110 may perform stereoscopic image analysis on the first and second plurality of images, the second and third plurality of images, and / or the first and third plurality of images (e.g., via execution by stereoscopic image analysis module 404 and based on the above combination). Figure 6 (The steps described). Processed information corresponding to the analysis of the first, second, and / or third plurality of images can be combined. In some embodiments, processing unit 110 can perform a combination of monocular and stereoscopic image analysis. For example, processing unit 110 can perform monocular image analysis on the first plurality of images (e.g., via execution of monocular image analysis module 402) and perform stereoscopic image analysis on the second and third plurality of images (e.g., via execution of stereoscopic image analysis module 404). The configuration of image capturing devices 122, 124, and 126—including their respective positions and fields of view 202, 204, and 206—can affect the type of analysis performed on the first, second, and third plurality of images. The disclosed embodiments are not limited to a specific configuration of image capturing devices 122, 124, and 126 or the type of analysis performed on the first, second, and third plurality of images.

[0203] In some embodiments, processing unit 110 may perform tests on system 100 based on the images acquired and analyzed in steps 710 and 720. Such tests can provide an indicator of the overall performance of system 100 for certain configurations of image acquisition devices 122, 124, and 126. For example, processing unit 110 may determine the proportion of "false hits" (e.g., situations where system 100 incorrectly determines the presence of vehicles or pedestrians) and "missed hits."

[0204] In step 730, processing unit 110 may induce one or more navigation responses in vehicle 200 based on information obtained from two of the first, second, and third plurality of images. The selection of two of the first, second, and third plurality of images may depend on various factors, such as, for example, the number, type, and size of objects detected in each of the plurality of images. Processing unit 110 may also make the selection based on image quality and resolution, the effective field of view reflected in the image, the number of captured frames, and the degree to which one or more objects of interest actually appear in the frames (e.g., the percentage of frames in which objects appear, the proportion of objects appearing in each such frame), etc.

[0205] In some embodiments, processing unit 110 can select two pieces of information obtained from a first, second, and third plurality of images by determining the degree of consistency between information obtained from one image source and information obtained from other image sources. For example, processing unit 110 can combine processed information obtained from each of image capture devices 122, 124, and 126 (whether through monocular analysis, stereo analysis, or any combination of both) and determine consistent visual indicators (e.g., lane markings, detected vehicles and their positions and / or paths, detected traffic lights, etc.) between each captured image from image capture devices 122, 124, and 126. Processing unit 110 can also exclude inconsistent information between the captured images (e.g., vehicles changing lanes, lane models indicating that a vehicle is too close to vehicle 200, etc.). Therefore, processing unit 110 can select two pieces of information obtained from the first, second, and third plurality of images based on the determination of consistent and inconsistent information.

[0206] Navigation responses may include, for example, steering, lane changes, and changes in acceleration. Processing unit 110 may base its responses on the analysis performed in step 720 and the above-described combination. Figure 4The described techniques induce one or more navigation responses. Processing unit 110 may also induce one or more navigation responses using data obtained from the execution of velocity and acceleration module 406. In some embodiments, processing unit 110 may induce one or more navigation responses based on the relative position, relative velocity, and / or relative acceleration between the vehicle 200 and objects detected within any of the first, second, and third plurality of images. The plurality of navigation responses may occur simultaneously, sequentially, or in any combination thereof.

[0207] Reinforcement learning and trained navigation systems

[0208] The following sections discuss autonomous driving and the systems and methods used to achieve autonomous control of a vehicle, whether that control is fully autonomous (self-driving vehicle) or partially autonomous (e.g., one or more driver assistance systems or functions). Figure 8 As shown, the autonomous driving task can be divided into three main modules, including a sensing module 801, a driving strategy module 803, and a control module 805. In some embodiments, modules 801, 803, and 805 may be stored in memory units 140 and / or 150 of system 100, or modules 801, 803, and 805 (or portions thereof) may be stored remotely from system 100 (e.g., in a server accessible to system 100 via, for example, wireless transceiver 172). Furthermore, any module disclosed herein (e.g., modules 801, 803, and 805) may implement techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems.

[0209] The sensing module 801, implemented using processing unit 110, can handle various tasks related to sensing the navigation state in the environment of the host vehicle. These tasks may rely on inputs from various sensors and sensing systems associated with the host vehicle. These inputs may include images or image streams from one or more onboard cameras, GPS location information, accelerometer outputs, user feedback or user input to one or more user interface devices, radar, lidar, etc. Sensing data, including data from cameras and / or any other available sensors, as well as map information, can be collected, analyzed, and formulated into a “sensed state” that describes information extracted from the scene in the host vehicle’s environment. The sensed state may include sensed information related to target vehicles, lane markings, pedestrians, traffic lights, road geometry, lane shape, obstacles, distance to other objects / vehicles, relative speed, relative acceleration, and any other potentially sensed information. Supervised machine learning can be implemented to generate a sensed state output based on the sensed data provided to the sensing module 801. The output of the sensing module may represent the host vehicle’s sensed navigation “state,” which can be passed to the driving strategy module 803.

[0210] While a sensed state can be generated based on image data received from one or more cameras or image sensors associated with the primary vehicle, any suitable sensor or combination of sensors can also be used to generate a sensed state used in navigation. In some embodiments, a sensed state can be generated without relying on captured image data. In fact, any navigation principles described herein can be applied to sensed states generated based on captured image data as well as sensed states generated using other non-image-based sensors. The sensed state can also be determined via sources external to the primary vehicle. For example, a sensed state can be generated, in whole or in part, based on information received from sources remote from the primary vehicle (e.g., sensor information, processed state information, etc., shared from other vehicles, shared from a central server, or from any other information source relating to the navigation state of the primary vehicle).

[0211] Driving strategy module 803 (discussed in more detail below and implemented using processing unit 110) can implement a desired driving strategy to determine one or more navigation actions to be taken by the master vehicle in response to sensed navigation state. If no other agents (e.g., target vehicles or pedestrians) are present in the master vehicle's environment, the sensed state input to driving strategy module 803 can be processed in a relatively straightforward manner. The task becomes more complex when the sensed state requires negotiation with one or more other agents. Techniques used to generate the output of driving strategy module 803 can include reinforcement learning (discussed in more detail below). The output of driving strategy module 803 can include at least one navigation action of the master vehicle and may include desired acceleration (which can be translated into updated velocity of the master vehicle), desired yaw rate of the master vehicle, desired trajectory, and other potential desired navigation actions.

[0212] Based on the output from the driving strategy module 803, the control module 805, which can also be implemented using the processing unit 110, can generate control commands for one or more actuators or controlled devices associated with the host vehicle. Such actuators and devices may include accelerators, one or more steering controllers, brakes, signal transmitters, displays, or any other actuators or devices that can be controlled as part of navigation operations associated with the host vehicle. Aspects of control theory can be used to generate the output of the control module 805. The control module 805 can be responsible for generating and outputting commands to the controllable components of the host vehicle to implement the desired navigation goals or requirements of the driving strategy module 803.

[0213] Returning to the driving strategy module 803, in some embodiments, the driving strategy module 803 can be implemented using a trained system trained through reinforcement learning. In other embodiments, the driving strategy module 803 can be implemented "manually" by using specified algorithms to resolve various scenarios that may arise during autonomous navigation, without machine learning methods. However, while this approach is feasible, it may lead to an oversimplified driving strategy and may lack the flexibility of a machine learning-based trained system. For example, a trained system may be better equipped to handle complex navigation states and can better determine whether a taxi is parked or stopped to pick up / drop off passengers; determine whether a pedestrian intends to cross the street in front of the main vehicle; defensively balance unexpected behavior of other drivers; negotiate in dense traffic involving target vehicles and / or pedestrians; decide when to suspend certain navigation rules or augment others; anticipate unsensed but anticipated situations (e.g., whether a pedestrian will emerge from behind a car or obstacle); and so on. A reinforcement learning-based trained system is also better equipped to handle continuous and high-dimensional state spaces and continuous action spaces.

[0214] Training a system using reinforcement learning can involve learning a driving policy that maps sensed states to navigation actions. The driving policy is a function π: S → A, where S is a set of states, and... This is the action space (e.g., the desired speed, acceleration, yaw command, etc.). The state space is S = S s ×S p S s It is the sensing state, and S p This is additional information about the state saved through this strategy. Working at discrete time intervals, at time t, the current state s can be observed. t ∈S, and this strategy can be applied to obtain the desired action a. t =π(s) t ).

[0215] The system can be trained by exposing it to various navigation states, enabling it to apply the policy, and providing rewards (based on a reward function designed to reward desired navigational behavior). Based on this reward feedback, the system can "learn" the policy and become trained in generating desired navigational actions. For example, the learning system can observe the current state s. t ∈S, and based on policy To determine action a t ∈A. Based on the determined action (and the execution of that action), the environment moves to the next state s. t+1∈S, to be observed by the learning system. For each action generated in response to the observed state, the feedback to the learning system is a reward signal r1, r2, ...

[0216] The goal of reinforcement learning (RL) is to find a policy π. It is typically assumed that at time t, there exists a reward function r. t Its measurement is in state s t And take action a t The instantaneous mass. However, taking action a at time t. t The environment is affected, and therefore the values ​​of future states are influenced. Therefore, when deciding which action to take, not only current rewards but also future rewards should be considered. In some cases, when the system determines that taking a lower-reward option now may result in a higher reward in the future, the system should take that action even if it is associated with a lower reward than another available option. To formalize this, we observe that the policy π and the initial state s are inductively derived in... The distribution on, where if the actor starts at state s0 = s and follows policy π from there, then the vector (r1, ..., r2) is... T The probability of observing a reward is r1, ..., r. T The probability of . The value of the initial state s can be defined as:

[0217]

[0218] Instead of limiting the time range to T, we can discount future rewards, defining the following for some fixed γ∈(0,1):

[0219]

[0220] In any case, the optimal strategy is the solution to the following equation:

[0221]

[0222] Here, expectation is based on the initial state s.

[0223] There are several possible methods for training driving strategy systems. For example, imitation methods (e.g., behavioral cloning) can be used, in which the system learns from state / action pairs, where actions are those actions a good actor (e.g., a human) would choose in response to a particular observed state. Suppose we observe a human driver. Through this observation, many forms of (s) t a t Examples of ) (where s t It is a state, and at The actions of a human driver can be acquired, observed, and used as the basis for training driving strategy systems. For example, supervised learning can be used to learn policy π, such that π(s) t )≈a t This method has many potential advantages. First, it does not require defining a reward function. Second, the learning is supervised and occurs offline (no actor needs to be applied during the learning process). The disadvantage of this method is that different human drivers, and even the same human driver, are not deterministic in their policy choices. Therefore, for ||π(s) t )-a t Learning very small functions is generally not feasible. Moreover, over time, even small errors can accumulate and become large errors.

[0224] Another approach is policy-based learning. Here, the policy can be expressed parametrically, and a suitable optimization technique (e.g., stochastic gradient descent) can be used to directly optimize the policy. This method directly optimizes the policy... The given problem is solved. Of course, there are many other methods to solve this problem. One advantage of this method is that it directly solves the problem, thus often leading to good practical results. A potential drawback is that it often requires "online policy" training, meaning that the learning of π is an iterative process, where, at iteration j, the policy π has an imperfect quality. j And in order to construct the next strategy π j It must be based on π j It interacts with the environment while performing actions.

[0225] The system can also be trained via value-based learning (learning the Q or V function). It is assumed that a good approximation can learn the optimal value function V*. An optimal policy can be constructed (e.g., relying on the Bellman equation). Some versions of value-based learning can be implemented offline (called "offline policy" training). Some drawbacks of value-based methods may be due to their strong reliance on Markovian assumptions and the need for approximations of complex functions (approximating the value function can be more difficult than directly approximating the policy).

[0226] Another technique may include model-based learning and planning (learning the probabilities of state transitions and solving an optimization problem to find the optimal V). Combinations of these techniques can also be used to train learning systems. In this approach, the dynamics of the process can be learned, i.e., the actions taken (s... t a t ) and in the next state st+1 This is a function that generates a distribution. Once this function is learned, an optimal problem can be solved to find the policy π whose value is optimal. This is called "planning". One advantage of this method may be that the learning part is supervised and can be learned by observing triples (s t a t s t+1 This approach is similar to the "imitation" method, but one drawback is that small errors during the learning process can accumulate and result in under-executed strategies.

[0227] Another approach for training the driving policy module 803 may include decomposing the driving policy function into semantically meaningful components. This allows certain parts of the policy to be implemented manually, ensuring the policy's safety; and reinforcement learning techniques can be used to implement other parts of the policy, adaptively achieving a human-like balance between defensive / aggressive behaviors and human-like negotiation with other drivers across various scenarios. Technically, reinforcement learning methods can combine several approaches and provide an easy-to-operate training procedure, where most training can be performed using recorded data or a self-built simulator.

[0228] In some embodiments, the training of the driving strategy module 803 may rely on an "options" mechanism. For illustration, consider a simple scenario of driving strategy on a two-lane highway. In the direct RL method, policy π maps the state to... In this context, the first component of π(s) is the desired acceleration command, and the second component of π(s) is the yaw rate. In the modified method, the following strategy can be constructed:

[0229] Automatic Cruise Control (ACC) strategy, o ACC S→A: This strategy always outputs a yaw rate of 0 and only changes the speed to achieve smooth and accident-free driving.

[0230] ACC + Left Strategy, o L S→A: The longitudinal command for this strategy is the same as the ACC command. Yaw rate is a direct implementation of centering the vehicle toward the center of the left lane while ensuring safe lateral movement (e.g., not moving to the left if there is a vehicle on the left).

[0231] ACC + right strategy, o R S→A: with o L The same applies, but vehicles can center themselves in the middle of the right lane.

[0232] These strategies can be called "options". Using these "options", we can learn the strategy π that has chosen those options.o S→O, where O is a set of available options. In one case, O={o ACC o L o R Option selector strategy π o By setting each s Let's define the actual strategy π: S→A.

[0233] In practice, the policy function can be decomposed into option graph 901, such as... Figure 9 As shown. Figure 10 Another example, option graph 1000, is shown. An option graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There exists a special node called the root node 903 of the graph. This node has no incoming nodes. The decision-making process traverses the entire graph from the root node until it reaches a "leaf" node, which is a node with no outgoing decision lines. Figure 9 As shown, leaf nodes may include, for example, nodes 905, 907, and 909. Upon encountering a leaf node, the driving strategy module 803 may output acceleration and steering commands associated with a desired navigation action, which is associated with the leaf node.

[0234] Internal nodes (such as nodes 911, 913, and 915) can, for example, trigger the implementation of a policy that selects children from its available options. The set of available children for an internal node includes all nodes associated with a particular internal node via a decision line. For example, Figure 9 The internal node 913 designated as "merge" includes three child nodes 909, 915 and 917, each connected to node 913 via a decision line ("hold", "overtake on the right" and "overtake on the left" respectively).

[0235] Flexibility in decision-making systems can be achieved by enabling nodes to adjust their position within the hierarchy of the option graph. For example, any node could be allowed to declare itself "critical." Each node could implement an "is critical" function, which outputs "true" if the node is in a critical part of its policy implementation. For instance, a node responsible for overtaking could declare itself critical midway through a maneuver. This imposes a constraint on the set of available offspring of node u, which can include all nodes v that are children of node u, and that there exists a path from v to a leaf node passing through all nodes designated as critical. On the one hand, this approach allows the desired path to be declared on the graph at each time step; on the other hand, it preserves the stability of the policy, especially when the critical part of the policy is being implemented.

[0236] By defining an option graph, the problem of learning the driving policy π: S→A can be decomposed into the problem of defining a policy for each node in the graph, where the policy at an internal node should be selected from its available child nodes. For some nodes, the corresponding policy can be implemented manually (e.g., by specifying a set of actions in response to an observed state using an if-then type algorithm), while for others it can be implemented using a trained system built through reinforcement learning. The choice between manual or trained / learned methods can depend on the safety aspects associated with the task and on its relative simplicity. The option graph can be constructed in a way that allows some nodes to implement directly while others can rely on a trained model. This approach ensures the safe operation of the system.

[0237] The following discussion provides information regarding the driving strategy module 803, such as... Figure 9 Further details regarding the role of the option graph in the above description. As mentioned above, the input to the driving strategy module is the "sensed state," which summarizes the environment map, for example, obtained from available sensors. The output of the driving strategy module 803 is a set of expectations (optionally along with a set of hard constraints) that define the trajectory as a solution to the optimization problem.

[0238] As described above, an option graph represents a hierarchical set of decisions organized as a Directed Acyclic Graph (DAG). There exists a special node called the “root” of the graph. The root node is the only node without any incoming edges (e.g., decision lines). The decision-making process traverses the graph from the root node until it reaches a “leaf” node, which is a node without any outgoing edges. Each internal node should implement a strategy of selecting one offspring from its available offspring. Each leaf node should implement a strategy based on a set of expectations (e.g., a set of navigation goals for the main vehicle) defined by the entire path from the root to the leaf. This set of expectations, directly defined based on the sensed state, along with a set of hard constraints, establishes an optimization problem whose solution is the vehicle's trajectory. Hard constraints can be employed to further enhance system safety, and the expectations can be used to provide driving comfort and human-like driving behavior. The trajectory, provided as a solution to the optimization problem, further defines the commands that should be provided to the steering, braking, and / or engine actuators to complete that trajectory.

[0239] return Figure 9 Option diagram 901 represents the option diagram for a two-lane highway, including merging lanes (meaning that at some point, a third lane merges into the right or left lane of the highway). Root node 903 first determines whether the primary vehicle is in a normal road scenario or approaching a lane-merging scenario. This is an example of a decision that can be made based on the sensed state. Normal road node 911 includes three child nodes: hold node 909, left overtaking node 917, and right overtaking node 915. Hold refers to the situation where the primary vehicle wants to continue traveling in the same lane. The hold node is a leaf node (without an outgoing edge / line). Therefore, the hold node defines a set of expectations. The first expectation it defines may include the desired lateral position—e.g., as close as possible to the center of the current lane. Smooth navigation may also be desired (e.g., within a predetermined or allowed maximum acceleration). The hold node may also define how the primary vehicle reacts to other vehicles. For example, the hold node may look at sensed target vehicles and assign a semantic meaning to each vehicle, which can be translated into components of the trajectory.

[0240] Various semantic meanings can be assigned to target vehicles in the environment of the master vehicle. For example, in some embodiments, semantic meanings may include any of the following designations: 1) Irrelevant: Indicates that the sensed vehicle in the scene is currently irrelevant; 2) Next Lane: Indicates that the sensed vehicle is in the adjacent lane and should maintain an appropriate offset relative to that vehicle (the precise offset can be computed in an optimization problem that constructs a trajectory given expectations and hard constraints, and the precise offset may potentially depend on the semantic type of the target vehicle set by the hold-alive leaf of the vehicle's option graph, which defines the expectations relative to the target vehicle); 3) Give Way: The master vehicle will attempt to give way to the sensed target vehicle by, for example, reducing speed (especially if the master vehicle determines that the target vehicle may cut into the master vehicle's lane); 4) Take Way: The master vehicle will attempt to take the right-of-way by, for example, increasing speed; 5) Follow: The master vehicle expects to follow the target vehicle and maintain smooth driving; 6) Left / Right Overtake: This means that the master vehicle wants to initiate a lane change to the left or right lane. Left Overtake node 917 and Left Overtake node 915 are internal nodes whose expectations are not yet defined.

[0241] The next node in option diagram 901 is the gap selection node 919. This node is responsible for selecting the gap between two target vehicles in a specific target lane that the master vehicle wishes to enter. By selecting a node of the form IDj, for some value of j, the master vehicle arrives at a leaf that specifies the desired outcome for the trajectory optimization problem—for example, the master vehicle wishes to maneuver to reach the selected gap. This maneuver might involve first accelerating / braking in the current lane and then moving to the target lane at an appropriate time to enter the selected gap. If the gap selection node 919 cannot find a suitable gap, it moves to the abort node 921, which defines the expectation of returning to the center of the current lane and canceling the overtaking maneuver.

[0242] Returning to lane merging node 913, when the primary vehicle approaches to merge, it has several options that may vary depending on the specific circumstances. For example, such as... Figure 11A As shown, the main vehicle 1105 is traveling along a two-lane road where no other target vehicle is detected in the main lane or merging lane 1111 of the two-lane road. In this case, the driving strategy module 803 can choose to remain in the lane 909 when it reaches the lane merging node 913. That is, it is expected to remain in its current lane when no target vehicle is sensed merging into the road.

[0243] exist Figure 11BIn the latter case, the situation is slightly different. Here, the main vehicle 1105 senses one or more target vehicles 1107 entering the main road 1112 from the merging lane 1111. In this case, once the driving strategy module 803 encounters the lane merging node 913, it can choose to initiate a left overtaking maneuver to avoid the lane merging situation.

[0244] exist Figure 11C In this scenario, the primary vehicle 1105 encounters one or more target vehicles 1107 entering the main road 1112 from merging lane 1111. The primary vehicle 1105 also detects a target vehicle 1109 traveling in a lane adjacent to its own. The primary vehicle also detects one or more target vehicles 1110 traveling in the same lane as the primary vehicle 1105. In this situation, the driving strategy module 803 can decide to adjust the speed of the primary vehicle 1105 to give way to target vehicle 1107 and proceed ahead of target vehicle 1115. This can be achieved, for example, by proceeding to a gap selection node 919, which then selects the gap between ID0 (vehicle 1107) and ID1 (vehicle 1115) as the appropriate lane-merging gap. In this case, the appropriate gap for the lane-merging situation defines the objective of the trajectory planner optimization problem.

[0245] As described above, nodes in the option graph can declare themselves as "critical," ensuring that the selected option passes through critical nodes. Formally, each node can implement the IsCritical function. After performing a forward pass from root to leaf on the option graph and solving the optimization problem for the trajectory planner, a backward pass can be performed from leaf back to root. Along this backward pass, the IsCritical function of all nodes in the pass can be called, and a list of all critical nodes can be maintained. In the forward path corresponding to the next time frame, the driving strategy module 803 may need to select a path from the root node to the leaf that passes through all critical nodes.

[0246] Figures 11A to 11CThis can be used to demonstrate the potential benefits of this method. For example, in the case of initiating an overtaking maneuver and the driving strategy module 803 reaching the leaf corresponding to IDk, such as when the master vehicle is in the middle of an overtaking maneuver, selecting the hold node 909 would be undesirable. To avoid such a jump, the IDj node can designate itself as critical. During the maneuver, the success of the trajectory planner can be monitored, and if the overtaking maneuver proceeds as expected, the IsCritical function will return a "true" value. This method ensures that the overtaking maneuver will continue in the next time frame (instead of jumping to another potentially inconsistent maneuver before completing the initially selected maneuver). On the other hand, if the monitoring of the maneuver indicates that the selected maneuver does not proceed as expected, or if the maneuver becomes unnecessary or impossible, the IsCritical function can return a "false" value. This allows the gap selection node to select a different gap in the next time frame, or to abort the overtaking maneuver entirely. This method, on the one hand, allows the desired path to be declared on the option graph at each time step, while on the other hand, it can help improve the stability of the strategy during the execution of critical parts.

[0247] Hard constraints can differ from navigation expectations, which will be discussed in more detail below. For example, hard constraints can ensure safe driving by applying an additional filtering layer to planned navigation actions. The hard constraints involved can be determined based on sensed states and can be manually programmed and defined rather than using a trained system built on reinforcement learning. However, in some embodiments, the trained system can learn applicable hard constraints to apply and follow. This approach can prompt the driving strategy module 803 to reach a selected action that already conforms to the applicable hard constraints, which can reduce or eliminate the need for subsequent modifications to conform to the applicable hard constraints. Nevertheless, as a redundant safety measure, hard constraints can be applied to the output of the driving strategy module 803 even if it has been trained to handle predetermined hard constraints.

[0248] There are many examples of potential hard constraints. For instance, a hard constraint can be defined in conjunction with guardrails at the edge of a road. Under no circumstances should a primary vehicle be allowed to cross the guardrail. This rule creates a hard lateral constraint on the trajectory of the primary vehicle. Another example of a hard constraint can include road bumps (e.g., rate-controlled bumps), which can impose hard constraints on driving speed both before and while crossing the bump. Hard constraints can be considered safety-critical and therefore can be defined manually rather than solely relying on a trained system that learns the constraints during training.

[0249] Compared to hard constraints, the desired outcome can be to achieve or attain comfortable driving. As discussed above, examples of desired outcomes could include the objective of positioning the primary vehicle within the lane at a lateral position corresponding to the center of the primary vehicle's lane. Another desired outcome might include the ID of a suitable entry gap. Note that the primary vehicle does not need to be precisely centered in the lane; rather, the desired outcome of being as close to it as possible ensures that even if it deviates from the lane center, the primary vehicle tends to migrate to the center of the lane. Desired outcomes may not be safety-critical. In some embodiments, desired outcomes may need to be negotiated with other drivers and pedestrians. One approach to constructing desired outcomes could rely on an option graph, and the policies implemented at at least some nodes of the graph could be based on reinforcement learning.

[0250] For nodes in option graph 901 or 1000 implemented as learning-based training nodes, the training process may include decomposing the problem into a supervised learning phase and a reinforcement learning phase. In the supervised learning phase, the problem can be learned from (s... t a t )arrive Differentiable mappings make This can be similar to "model-based" reinforcement learning. However, in the forward loop of the network, s can be used t+1 The actual value is used to replace This eliminates the problem of error accumulation. The role of prediction is to propagate messages from the future back to the past. In this sense, the algorithm can be a combination of "model-based" reinforcement learning and "policy-based learning".

[0251] In certain scenarios, an important element that can be provided is a differentiable path from future loss / reward back to the decision about the action. Through an option graph structure, the implementation of options involving safety constraints is often non-differentiable. To overcome this problem, the selection of offspring in the learned policy nodes can be randomized. That is, a node can output a probability vector p, which assigns a probability to each offspring selected for a particular node. Assume a node has k offspring, and let a... (1) , ..., a (k) For the action of the path from each child to the leaf, the resulting predicted action is therefore... This could lead to a differentiable path from the action to p. In practice, action a can be chosen as a for i ~ p. (i) And a and The difference between them can be called additive noise.

[0252] For a given s t a t Down Training can combine supervised learning with real-world data. For training node policies, a simulator can be used. Afterward, the policy can be fine-tuned using real data. Two concepts can make the simulation more realistic. First, using imitation, the "behavioral cloning" paradigm can be used to construct an initial policy using a large real-world dataset. In some cases, the resulting actors may be appropriate. In other cases, the resulting actors at least form a very good initial policy for other actors on the road. Second, using self-play, our own policy can be used to reinforce training. For example, given an initial implementation by other actors (cars / pedestrians) who may be experienced, the policy can be trained based on a simulator. Some other actors can be replaced by the new policy, and the process can be repeated. Therefore, the policy can continue to improve because it should respond to a wider variety of other actors with different levels of complexity.

[0253] Furthermore, in some embodiments, the system may implement a multi-agent approach. For example, the system may consider data from various sources and / or images captured from multiple angles. Additionally, some disclosed embodiments may provide energy economy because the anticipation of events that do not directly involve the primary vehicle but may affect it, or even the anticipation of events that may lead to unpredictable situations involving other vehicles, can be considered (e.g., the radar may "see through" vehicles ahead and the unavoidable anticipation, or even the high probability of events that will affect the primary vehicle).

[0254] Trained system with imposed navigation constraints

[0255] In autonomous driving scenarios, a key concern is ensuring the safety of the learned strategies of a trained navigation network. In some embodiments, constraints can be used to train the driving strategy system so that the actions selected by the trained system are accounted for by applicable safety constraints. Additionally, in some embodiments, an extra layer of safety can be provided by passing the selected actions of the trained system through one or more hard constraints related to a specific sensed scenario in the host vehicle's environment. This approach ensures that the host vehicle's actions are limited to those determined to satisfy applicable safety constraints.

[0256] At its core, the navigation system may include a learning algorithm based on a policy function that maps observed states to one or more desired actions. In some implementations, the learning algorithm is a deep learning algorithm. Desired actions may include at least one action expected to maximize the vehicle's anticipated reward. While in some cases the actual action taken by the vehicle may correspond to one of the desired actions, in others the actual action may be determined based on the observed states, one or more desired actions, and non-learning hard constraints (e.g., safety constraints) imposed on the learning navigation engine. These constraints may include no-driving zones around various types of detected objects (e.g., target vehicles, pedestrians, stationary objects on the roadside or road, moving objects on the roadside or road, guardrails, etc.). In some cases, the size of this zone may vary based on the detected motion (e.g., speed and / or direction) of the detected object. Other constraints may include maximum speed when crossing within a pedestrian's influence zone, maximum deceleration rate (to address the distance between the target vehicle and the main vehicle), mandatory stopping at sensed pedestrian crossings or railway crossings, etc.

[0257] Hard constraints, when used in conjunction with systems trained through machine learning, can provide a level of safety in autonomous driving that exceeds the safety achievable solely based on the output of a trained system. For example, a desired set of constraints can be used as training guidance to train a machine learning system, allowing it to select actions in response to sensed navigation states that adhere to applicable navigation constraints. However, trained systems retain some flexibility in selecting navigation actions, and therefore, at least in some cases, actions selected by the trained system may not strictly adhere to the relevant navigation constraints. Therefore, to require selected actions to strictly adhere to the relevant navigation constraints, non-machine learning components, outside the learning / training framework, that guarantee strict application of the relevant navigation constraints, can be used to combine, compare, filter, adjust, and modify the output of the trained system.

[0258] The following discussion provides additional details about trained systems and the potential benefits (particularly from a security perspective) of combining trained systems with algorithmic components outside the trained / learning framework. As previously mentioned, reinforcement learning objectives for policies can be optimized via stochastic gradient ascent. The objective (e.g., expected reward) can be defined as...

[0259] Objectives involving expectations can be used in machine learning scenarios. However, such objectives, when not constrained by navigational constraints, may not return actions strictly bound by those constraints. For example, consider a reward function where, This indicates the trajectory of rare "corner" events (such as accidents) to be avoided, while Representing the remaining trajectories, one goal of the learning system could be to learn to perform overtaking maneuvers. Typically, in accident-free trajectories, Successful, smooth overtaking will be rewarded, while failing to overtake while remaining in the lane will be penalized—therefore, the range is [-1, 1]. If the sequence... If the incident represents an accident, then the reward -r should provide a sufficiently high penalty to prevent such events from occurring. The question is, what should the value of r be to ensure accident-free driving?

[0260] It was observed that the accident had the following impact: The effect is an additional term -pr, where p is the probabilistic quality of the trajectory with the accident event. If this term is negligible, i.e., p << 1 / r, the learning system can more often preferentially execute the strategy that leads to the accident (or generally adopt a reckless driving strategy) to successfully perform overtaking maneuvers, rather than a more defensive strategy at the cost of some overtaking maneuvers failing. In other words, if the probability of the accident is at most p, then r must be set such that r >> 1 / p. It can be expected that p is very small (e.g., at p = 10). -9 (on the order of magnitude of). Therefore, r should be large. In the policy gradient, it can be estimated that... The gradient of the random variable. The following lemma shows that the gradient of the random variable. The variance of pr 2 As the variance increases, it becomes greater than r for the condition r >> 1 / p. Therefore, estimating the objective may be difficult, and estimating its gradient may be even more difficult.

[0261] Lemma: Let π o Let p and r be scalars, such that under probability p, we obtain And obtain with probability 1-p So,

[0262]

[0263] The final approximation applies to the case where r ≥ 1 / p.

[0264] This discussion indicates that the form is The goal may not ensure functional safety without introducing variance problems. Baseline subtraction methods used to reduce variance may not provide sufficient remedy for this problem, as it will stem from... The high variance of the baseline constant is transferred to the same high variance, and the estimate of the baseline constant will also be affected by numerical instability. Furthermore, if the probability of an accident is p, then at least 1 / p sequences should be sampled on average before obtaining the accident event. This means that the methods used to achieve this... The lower bound for the number of samples in the sequence of the minimization learning algorithm is 1 / p. The solution to this problem can be found in the architectural design described in this paper, rather than through numerical tuning techniques. The approach here is based on the idea that hard constraints should be injected outside the learning framework. In other words, the policy function can be decomposed into learnable and non-learnable parts. Formally, the policy function can be constructed as follows: in Mapping the (unknown) state space to a set of expectations (e.g., expected navigation goals, etc.), while π (T) This expectation is mapped to a trajectory (which determines how the car should move over short distances). The function π (T) It is responsible for driving comfort and making strategic decisions, such as which other vehicles should be overtaken or yielded to, and what the desired position of the main vehicle is within its lane. The mapping from the sensed navigation state to the desired outcome is the strategy. It can learn from experience by maximizing expected rewards. The resulting expectation can be transformed into a cost function for the driving trajectory. The function π (T) It is not a learned function, but rather a path that minimizes the cost of being subject to hard constraints in terms of functional safety. This decomposition ensures functional safety while providing a comfortable driving experience.

[0265] like Figure 11D As shown, a dual lane merging navigation scenario provides a further example illustrating these concepts. In dual lane merging, vehicles approach merging area 1130 from both the left and right sides. Vehicles from each side (such as vehicle 1133 or vehicle 1135) can decide whether to merge into the lane on the other side of merging area 1130. Successfully executing dual lane merging in busy traffic can require significant negotiation skills and experience, and may be difficult to perform heuristically or brute-force by enumerating all possible trajectories that all actors in the scenario might take. In this dual lane merging example, a set of expectations suitable for dual lane merging maneuvering can be defined. It can be the Cartesian product of the following sets:

[0266] Where, [0, v maxLet L be the desired target speed of the primary vehicle, L = {1, 1.5, 2, 2.5, 3, 3.5, 4} be the desired lateral position in lane units, where integers represent lane centers and fractions represent lane boundaries, and {g, t, o} are classification labels assigned to each of the n other vehicles. If the primary vehicle is to yield to another vehicle, the other vehicle can be assigned "g"; if the primary vehicle is to occupy the lane relative to another vehicle, the other vehicle can be assigned "t"; or if the primary vehicle is to maintain an offset distance relative to another vehicle, the other vehicle can be assigned "o".

[0267] The following describes a set of expectations. How can this be transformed into a cost function for the driving trajectory? The driving trajectory can be represented by (x1, y1), ..., (x... k y k ) indicates that (x i y i Let be the lateral and longitudinal position of the main vehicle at time τ·i (in egocentric units), where in some experiments τ = 0.1 seconds and k = 10. Other values ​​can, of course. The cost assigned to the trajectory can be a weighted sum of the individual costs assigned to the desired speed, lateral position, and the label assigned to each of the other n vehicles.

[0268] Given the desired rate υ∈[0,υ max The cost of the trajectory associated with the rate is

[0269]

[0270] Given a desired lateral position l∈L, the cost associated with that desired lateral position is:

[0271]

[0272] Where dist(x, y, 1) is the distance from point (x, y) to lane position l. Regarding the cost due to other vehicles, for any other vehicle, (x′1, y′1), ..., (x′1, y′1) = 1. k y′ k Let be the other vehicles in the egocentric unit of the main vehicle, and i can be the earliest point, for which there exists j such that (x) i y i ) and (x' j y' jThe distance between them is very small. If there is no such point, then i can be set to i = ∞. If the other vehicle is classified as "giving way", then it can be expected that τi > τj + 0.5, which means that the main vehicle will arrive at the trajectory intersection point at least 0.5 seconds after the other vehicle arrives at the same point. The formula for converting the above constraint into a cost can be [τ(ji) + 0.5]. + .

[0273] Similarly, if another car is classified as "obstructing traffic," we can expect τj > τi + 0.5, which can be converted into a cost [τ(ij) + 0.5]. + If another car is classified as "offset," then we can expect i = ∞, meaning the trajectory of the main vehicle and the trajectory of the offset car do not intersect. This situation can be converted into a cost by penalizing the distance between the trajectories.

[0274] Assigning weights to each of these costs provides a single objective function π for the trajectory planner. (T) The cost of encouraging smooth driving can be increased to achieve this goal. Furthermore, to ensure the functional safety of the trajectory, hard constraints can be added to achieve this goal. For example, (x) can be prohibited. i y i If |ij| is small, then for any trajectory point (x') of any other vehicle, the following holds true: j y' j ), prohibited (x) i y i Approaching (x') j y' j ).

[0275] In summary, strategy π θ It can be decomposed into a mapping from the unknown state to a set of expectations and a mapping from the expectations to the actual trajectory. The latter mapping is not based on learning and can be achieved by solving an optimization problem whose cost depends on the expectations, and whose hard constraints guarantee the functional safety of the policy.

[0276] The following discussion describes the mapping from the unknown state to the expected value of the set. As mentioned above, to meet functional safety requirements, systems relying solely on reinforcement learning may suffer from issues related to rewards. The high and inconvenient variance of the problem can be avoided by decomposing the problem into a mapping from an (unknown) state space to a set of desired paths using policy gradient iteration, and then mapping to the actual trajectories without involving a machine learning-based system.

[0277] For various reasons, decision-making can be further broken down into semantically meaningful components. For example, The size can be large or even continuous. Regarding Figure 11D In the dual lane merging scenario described above, Additionally, gradient estimators may involve terms In such an expression, the variance can increase with the time interval T. In some cases, the value of T can be approximately 250, which may be sufficient to produce significant variance. Assuming a sampling rate in the range of 10 Hz and a lane merging zone of 1130 meters, preparation for merging can begin approximately 300 meters before the merging zone. If the main vehicle is traveling at a speed of 16 meters per second (approximately 60 kilometers per hour), the value of T for an episode can be approximately 250.

[0278] Returning to the concept of option diagrams, Figure 11E The text shows what can be represented. Figure 11D The text describes an option graph for a dual-lane scenario. As previously mentioned, an option graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). Within this graph, there may exist a special node called the "root" node 1140, which can be the only node without any incoming edges (e.g., decision lines). The decision-making process can traverse the graph starting from the root node until it reaches a "leaf" node, i.e., a node without any outgoing edges. Each internal node can implement a policy function that selects one offspring from its available offspring. It is possible to find a set of expectations from a set of traversals on the option graph. The predefined mapping. In other words, traversal on the option graph can be automatically converted to... The expectation in the graph. Given a node v in the graph, the parameter vector θ v You can specify a strategy for selecting the offspring of v. If θ is all θ v If a concatenation exists, then it can be defined by traversing from the root of the graph to the leaves. At the same time, at each node v, use θ v Define a strategy to select child nodes.

[0279] exist Figure 11E In the dual lane merging option diagram 1139, the root node 1140 can first determine whether the main vehicle is within the lane merging area (e.g., Figure 11DIn area 1130, the primary vehicle may be approaching a lane-merging area and needs to prepare for a possible lane change. In both cases, the primary vehicle may need to decide whether to change lanes (e.g., to the left or right) or to remain in its current lane. If the primary vehicle has decided to change lanes, it may need to determine whether conditions are suitable for continuing and perform a lane-change maneuver (e.g., at “Continue” node 1142). If changing lanes is not possible, the primary vehicle may attempt to “advance” towards the desired lane by aiming at a lane marking (e.g., at node 1144, as part of negotiation with vehicles in the desired lane). Alternatively, the primary vehicle may choose to “remain” in the same lane (e.g., at node 1146). This process can naturally determine the lateral position of the primary vehicle. For example,

[0280] This allows for the natural determination of the desired lateral position. For example, if the master vehicle changes lane from lane 2 to lane 3, the "Continue" node can set the desired lateral position to 3, the "Maintain" node to 2, and the "Advance" node to 2.5. Next, the master vehicle can decide whether to maintain the "same" rate (node ​​1148), "accelerate" (node ​​1150), or "decelerate" (node ​​1152). Then, the master vehicle can enter a "chain" structure 1154 that overtakes other vehicles and sets their semantic meaning to values ​​in the set {g, t, o}. This process sets the expectation relative to other vehicles. Parameters can be shared across all nodes in this chain (similar to a recurrent neural network).

[0281] A potential benefit of the option is the interpretability of the results. Another potential benefit is the ability to rely on sets. The structure is decomposable, and therefore, the policy at each node can be selected from a small number of probabilities. Furthermore, this structure allows for a reduction in the variance of the policy gradient estimator.

[0282] As mentioned above, the length of a segment in a dual lane-changing scenario can be approximately T = 250 steps. Such a value (or any other suitable value depending on the specific navigation scenario) provides sufficient time to see the consequences of the lead vehicle's actions (e.g., if the lead vehicle decides to change lanes in preparation for a lane change, the lead vehicle can only see the benefit after successfully completing the lane change). On the other hand, due to the dynamics of driving, the lead vehicle must make decisions at a sufficiently high frequency (e.g., 10 Hz in the above case).

[0283] Option graphs can reduce the effective value of T in at least two ways. First, given higher-level decisions, rewards can be defined for lower-level decisions while considering shorter segments. For example, when the master vehicle has already selected the "lane change" and "continue" nodes, by watching a 2-3 second segment (meaning T becomes 20-30 instead of 250), a policy for assigning semantic meaning to the vehicle can be learned. Second, for higher-level decisions (such as whether to change lanes or stay in the same lane), the master vehicle may not need to make a decision every 0.1 seconds. Instead, the master vehicle can make decisions at a lower frequency (e.g., per second), or implement an "option termination" function, then the gradient can be computed only after each option termination. In both cases, the effective value of T may be an order of magnitude smaller than its original value. In summary, the estimator for each node can depend on the value of T, which is an order of magnitude smaller than the original 250 steps, which may immediately result in a smaller variance.

[0284] As mentioned above, hard constraints can promote safer driving, and several different types of constraints can exist. For example, static hard constraints can be defined directly based on the sensing state. These hard constraints can include speed bumps, speed limits, road curves, intersections, etc., within the host vehicle's environment, and may involve one or more constraints on vehicle speed, heading, acceleration, braking (deceleration), etc. Static hard constraints may also include semantic free space, where, for example, the host vehicle is prohibited from driving out of free space and from driving too close to physical obstacles. Static hard constraints can also restrict (e.g., prohibit) maneuvers that do not conform to various aspects of the vehicle's kinematics; for example, static hard constraints can be used to prohibit maneuvers that may cause the host vehicle to roll over, slide, or otherwise lose control.

[0285] Hard constraints can also be associated with vehicles. For example, a constraint could be used requiring a vehicle to maintain a longitudinal distance of at least one meter and a lateral distance of at least 0.5 meters from other vehicles. Constraints can also be applied such that the primary vehicle will avoid maintaining a collision path with one or more other vehicles. For example, time τ can be a time metric based on a specific scenario. Predicted trajectories of the primary vehicle and one or more other vehicles can be considered from the current time to time τ. In the case where two trajectories intersect, This can represent the arrival and departure times of vehicle i at the intersection point. That is, each car will arrive at the intersection point when the first part of the car passes through it, and a certain amount of time is required before the last part of the car passes through it. This amount of time separates the arrival time from the departure time. Assume... (That is, if vehicle 1 arrives at a later time than vehicle 2), then we want to ensure that vehicle 1 has left the intersection point before vehicle 2 arrives. Otherwise, a collision will occur. Therefore, it is possible to implement a mechanism that... The hard constraints. Furthermore, to ensure that vehicles 1 and 2 are staggered from each other by a minimum, an additional safety margin can be obtained by including a buffer time (e.g., 0.5 seconds or another appropriate value) in the constraints. The hard constraints related to the predicted intersection trajectories of the two vehicles can be expressed as...

[0286] The amount of time τ during which the trajectories of the primary vehicle and one or more other vehicles are tracked can vary. However, in an intersection scenario, the speed may be lower, τ may be longer, and τ can be defined as the time during which the primary vehicle will enter and leave the intersection in less than τ seconds.

[0287] Of course, applying hard constraints to vehicle trajectories requires predicting the trajectories of those vehicles. For the primary vehicle, trajectory prediction may be relatively straightforward, as it typically already understands and is, in fact, planning the expected trajectory at any given time. Predicting the trajectories of other vehicles may be less straightforward. For other vehicles, the baseline calculations used to determine the predicted trajectories may depend on their current speed and heading, for example, as determined by analysis of an image stream captured by one or more cameras and / or other sensors (radar, lidar, acoustic, etc.) on the primary vehicle.

[0288] However, there may be exceptions that can simplify the problem or at least provide increased confidence in the predicted trajectory for another vehicle. For example, in structured roads with lane markings and the possibility of yield rules, the trajectory of other vehicles can be based at least in part on the position of the other vehicles relative to the lanes and on applicable yield rules. Therefore, in some cases, when lane structure is observed, it can be assumed that vehicles in the next lane will adhere to lane boundaries. That is, the primary vehicle can assume that vehicles in the next lane will remain in their lane unless evidence indicating that a vehicle in the next lane will cut into the primary vehicle's lane is observed (e.g., traffic lights, strong lateral movement, movement across lane boundaries).

[0289] Other situations can also provide clues about the expected trajectories of other vehicles. For example, at stop signs, traffic lights, roundabouts, etc., where the primary vehicle has the right-of-way, it can be assumed that other vehicles will obey that right-of-way. Therefore, unless evidence of a violation is observed, it can be assumed that other vehicles will continue along the trajectory of obeying the primary vehicle's right-of-way.

[0290] Hard constraints can also be applied relative to pedestrians in the host vehicle's environment. For example, a buffer distance can be established relative to pedestrians, prohibiting the host vehicle from driving closer than a predetermined buffer distance relative to any observed pedestrian. The pedestrian buffer distance can be any suitable distance. In some embodiments, the buffer distance can be at least one meter relative to an observed pedestrian.

[0291] Similar to the case with vehicles, hard constraints can also be applied relative to the relative motion between the pedestrian and the main vehicle. For example, the pedestrian's trajectory (based on heading and speed) can be monitored relative to the expected trajectory of the main vehicle. Given a specific pedestrian trajectory, where for each point p on the trajectory, t(p) can represent the time required for the pedestrian to reach point p. To maintain the required buffer distance of at least 1 meter from the pedestrian, t(p) must be greater than the time it takes for the main vehicle to reach point p (with a sufficient time difference so that the main vehicle passes in front of the pedestrian at least 1 meter ahead), or t(p) must be less than the time it takes for the main vehicle to reach point p (e.g., if the main vehicle brakes to give way to the pedestrian). However, in the latter example, the hard constraint might require the main vehicle to arrive at point p sufficiently later than the pedestrian so that the main vehicle can pass behind the pedestrian and maintain the required buffer distance of at least 1 meter. Of course, there may be exceptions to the hard constraints for pedestrians. For example, if the primary vehicle has the right-of-way or is traveling at a very slow speed, and there is no evidence that the pedestrian will refuse to give way to the primary vehicle or will otherwise navigate toward the primary vehicle, the pedestrian hard constraint can be relaxed (e.g., to a smaller buffer of at least 0.75 meters or 0.50 meters).

[0292] In some examples, constraints may be relaxed when it is determined that not all constraints can be satisfied. For example, if a road is too narrow to allow for two curbs or the required clearance (e.g., 0.5 meters) between a curb and a parked vehicle, one or more constraints may be relaxed if mitigation is possible. For example, if there are no pedestrians (or other objects) on the sidewalk, vehicles may drive slowly at a distance of 0.1 meters from the curb. In some embodiments, constraints may be relaxed if relaxing them improves the user experience. For example, to avoid potholes, constraints may be relaxed to allow vehicles to get closer to the edge of the lane, curb, or pedestrians than would normally be permissible. Furthermore, in determining which constraints to relax, in some embodiments, one or more constraints to relax are those considered to have the least available negative impact on safety. For example, a constraint regarding how close a vehicle can get to a curb or concrete barrier may be relaxed before relaxing a constraint concerning proximity to other vehicles. In some embodiments, pedestrian constraints may be the last to be relaxed, or in some cases may never be relaxed.

[0293] Figure 12 Examples of scenes that can be captured and analyzed during navigation of the host vehicle are shown. For example, the host vehicle may include a navigation system (e.g., system 100) as described above, which can receive multiple images representing the host vehicle's environment from cameras (e.g., at least one of image capture device 122, image capture device 124, and image capture device 126) associated with the host vehicle. Figure 12 The scene shown is an example of one of the images that can be captured at time t from the environment of a master vehicle traveling along a predicted trajectory 1212 in lane 1210. The navigation system may include at least one processing device (e.g., any EyeQ processor or other device described above) specifically programmed to receive multiple images and analyze them to determine actions in response to the scene. Specifically, as... Figure 8 As shown, at least one processing device can implement a sensing module 801, a driving strategy module 803, and a control module 805. The sensing module 801 can collect and output image information collected from a camera, and provide this information to the driving strategy module 803 in the form of a recognized navigation state, which can constitute a trained navigation system trained using machine learning techniques (such as supervised learning, reinforcement learning, etc.). Based on the navigation state information provided to the driving strategy module 803 by the sensing module 801, the driving strategy module 803 (e.g., by implementing the option map method described above) can generate the desired navigation actions to be performed by the main vehicle in response to the recognized navigation state.

[0294] In some embodiments, at least one processing device may use, for example, control module 805, to directly translate the desired navigation action into a navigation command. However, in other embodiments, hard constraints may be applied to test the desired navigation action provided by driving strategy module 803 against various predetermined navigation constraints that may be involved in the scene and the desired navigation action. For example, if driving strategy module 803 outputs a desired navigation action that would cause the master vehicle to follow trajectory 1212, the navigation action may be tested against one or more hard constraints associated with various aspects of the master vehicle's environment. For example, captured image 1201 may show curb 1213, pedestrian 1215, target vehicle 1217, and stationary objects present in the scene (e.g., an overturned box). Each of these may be associated with one or more hard constraints. For example, curb 1213 may be associated with a static constraint that prevents the master vehicle from navigating onto or past the curb and onto pedestrian crossing 1214. The curb 1213 can also be associated with a barrier envelope that defines a distance (e.g., a buffer zone) extending away from the curb (e.g., 0.1 m, 0.25 m, 0.5 m, 1 m, etc.) and along the curb, defining a no-navigation zone for primary vehicles. Of course, static constraints can also be associated with other types of roadside boundaries (e.g., guardrails, concrete posts, traffic cones, bridge towers, or any other type of roadside barrier).

[0295] It should be noted that distance and range can be determined by any suitable method. For example, in some embodiments, distance information can be provided by an onboard radar and / or lidar system. Alternatively or additionally, distance information can be derived from the analysis of one or more images captured from the environment of the host vehicle. For example, multiple pixels representing an identified object in an image can be determined and compared with the known field of view and focal length geometry of the image capturing device to determine scale and distance. For example, velocity and acceleration can be determined by observing the change in scale between objects in one image and another over a known time interval. This analysis can indicate the direction of motion of the object toward or away from the host vehicle, and at what speed the object is moving away from or toward the host vehicle. Crossing speed can be determined by analyzing the change in the X-coordinate position of the object from one image to another over a known time interval.

[0296] Pedestrian 1215 can be associated with a pedestrian envelope defining a buffer zone 1216. In some cases, the imposed hard constraints can prohibit the host vehicle from navigating within one meter of pedestrian 1215 (in any direction relative to pedestrian 1215). Pedestrian 1215 can also define the location of a pedestrian influence zone 1220. This influence zone can be associated with constraints limiting the speed of the host vehicle within the influence zone. The influence zone can extend from pedestrian 1215 to 5 meters, 10 meters, 20 meters, etc. Each gradient of the influence zone can be associated with a different speed limit. For example, within a range of one to five meters from pedestrian 1215, the host vehicle can be limited to a first speed (e.g., 10 mph, 20 mph, etc.), which can be less than the speed limit in a pedestrian influence zone extending from 5 meters to 10 meters. Any gradient can be used for each gradient of the influence zone. In some embodiments, the first gradient can be narrower than the range from one meter to five meters and can extend only from one meter to two meters. In other embodiments, the first level of the influence zone may extend from one meter (the boundary of the pedestrian-only navigation zone) to a distance of at least 10 meters. The second level may further extend from ten meters to at least approximately twenty meters. The second level may be associated with the maximum speed of the main vehicle, which is greater than the maximum speed associated with the first level of the pedestrian influence zone.

[0297] One or more stationary object constraints may also be associated with a scene detected in the environment of the host vehicle. For example, in image 1201, at least one processing device may detect a stationary object, such as a box 1219 present in the road. The detected stationary object may include at least one of various objects, such as a tree, pole, road sign, or other objects in the road. One or more predetermined navigation constraints may be associated with the detected stationary object. For example, such constraints may include a stationary object envelope that defines a buffer zone about the object, which may prevent the host vehicle from navigating into the buffer zone. At least a portion of the buffer zone may extend a predetermined distance from the edge of the detected stationary object. For example, in the scene represented by image 1201, a buffer zone of at least 0.1 meters, 0.25 meters, 0.5 meters, or more may be associated with the box 1219, such that the host vehicle will pass the right or left side of the box by at least a distance (e.g., a buffer zone distance) to avoid a collision with the detected stationary object.

[0298] The predetermined hard constraints may also include one or more target vehicle constraints. For example, a target vehicle 1217 may be detected in image 1201. To ensure that the primary vehicle does not collide with the target vehicle 1217, one or more hard constraints may be employed. In some cases, the target vehicle envelope may be associated with a single buffer distance. For example, a buffer may be defined by a distance of 1 meter around the target vehicle in all directions. The buffer may define an area extending at least one meter from the target vehicle, prohibiting the primary vehicle from navigating into that area.

[0299] However, the envelope surrounding the target vehicle 1217 does not need to be defined by a fixed buffer distance. In some cases, the predetermined hard constraints associated with the target vehicle (or any other movable object detected in the environment of the master vehicle) may depend on the orientation of the master vehicle relative to the detected target vehicle. For example, in some cases, the longitudinal buffer distance (e.g., the distance extending from the target vehicle toward the front or rear of the master vehicle—such as when the master vehicle is traveling toward the target vehicle) may be at least one meter. The lateral buffer distance (e.g., the distance extending from the target vehicle toward either side of the master vehicle—such as when the master vehicle is traveling in the same or opposite direction as the target vehicle, such that one side of the master vehicle passes the side adjacent to the target vehicle) may be at least 0.5 meters.

[0300] As mentioned above, other constraints may also be involved in detecting target vehicles or pedestrians in the environment of the main vehicle. For example, the predicted trajectories of the main vehicle and target vehicle 1217 can be considered, and hard constraints may be required if the two trajectories intersect (e.g., at intersection point 1230). or The primary vehicle is vehicle 1, and the target vehicle 1217 is vehicle 2. Similarly, the trajectory of pedestrian 1215 can be monitored relative to the expected trajectory of the primary vehicle (based on direction of travel and speed). Given a specific pedestrian trajectory, for each point p on the trajectory, t(p) will represent the pedestrian's arrival at point p (i.e., Figure 12 The time required for the main vehicle to reach point p (points 1, 2, 3, and 1) is as follows. To maintain the required buffer distance of at least 1 meter from the pedestrian, t(p) must be greater than the time it takes for the main vehicle to reach point p (with a sufficient time difference so that the main vehicle passes in front of the pedestrian at least 1 meter ahead), or t(p) must be less than the time it takes for the main vehicle to reach point p (e.g., if the main vehicle brakes to give way to the pedestrian). Nevertheless, in the latter example, the hard constraint would require the main vehicle to arrive at point p a sufficient time later than the pedestrian so that the main vehicle can pass behind the pedestrian and maintain the required buffer distance of at least 1 meter.

[0301] Other hard constraints may also be employed. For example, in at least some cases, a maximum deceleration rate of the main vehicle may be used. This maximum deceleration rate can be determined based on the detected distance to the target vehicle following the main vehicle (e.g., using images collected from a rear-view camera). Hard constraints may include mandatory stops at sensed pedestrian crossings or railway crossings, or other applicable constraints.

[0302] In cases where the analysis of a scenario within the host vehicle's environment indicates that one or more predetermined navigation constraints may be involved, these constraints can be imposed relative to one or more planned navigation actions of the host vehicle. For example, if the analysis of a scenario causes the driving strategy module 803 to return a desired navigation action, that desired navigation action can be tested against one or more of the involved constraints. If the desired navigation action is determined to violate any aspect of the involved constraints (e.g., if the desired navigation action would drive the host vehicle within 0.7 meters of pedestrian 1215, where a predetermined hard constraint requires the host vehicle to maintain a distance of at least 1.0 meter from pedestrian 1215), at least one modification to the desired navigation action can be made based on one or more predetermined navigation constraints. Adjusting the desired navigation action in this way can provide the host vehicle with actual navigation actions by following the constraints involved in the specific scenario detected in the host vehicle's environment.

[0303] After determining the actual navigation action of the master vehicle, the navigation action can be achieved by adjusting at least one navigation actuator of the master vehicle in response to the determined actual navigation action. Such navigation actuator may include at least one of the master vehicle's steering mechanism, brake, or accelerator.

[0304] Priority constraints

[0305] As mentioned above, navigation systems can employ various hard constraints to ensure the safe operation of the primary vehicle. Constraints may include minimum safe driving distances relative to pedestrians, target vehicles, roadblocks, or detected objects; maximum driving speed when passing within the influence zone of a detected pedestrian; or the maximum deceleration rate of the primary vehicle, etc. These constraints can be imposed on trained systems based on machine learning (supervised, reinforcement, or combined) training, but they can also be useful for untrained systems (e.g., those that employ algorithms to directly handle expected situations occurring in a scenario derived from the primary vehicle's environment).

[0306] In any situation, a hierarchy of constraints may exist. In other words, some navigation constraints take precedence over others. Therefore, if a situation arises where a navigation action that would satisfy all relevant constraints is unavailable, the navigation system can first determine the available navigation action that fulfills the highest priority constraint. For example, the system could also instruct the vehicle to avoid the pedestrian first, even if avoiding the pedestrian would result in a collision with another vehicle or object detected in the road. In another example, the system could instruct the vehicle to ride on the curb to avoid the pedestrian.

[0307] Figure 13 A flowchart illustrating an algorithm for implementing a hierarchy of relevant constraints determined based on analysis of a scene in the host vehicle's environment is provided. For example, in step 1301, at least one processing device associated with the navigation system (e.g., an EyeQ processor, etc.) can receive multiple images representing the host vehicle's environment from a camera mounted on the host vehicle. Through analysis of the images representing the scene in the host vehicle's environment in step 1303, a navigation state associated with the host vehicle can be identified. For example, the navigation state might indicate that the host vehicle is traveling along a two-lane road 1210, such as... Figure 12 As shown, target vehicle 1217 is moving through the intersection in front of the main vehicle, pedestrian 1215 is waiting to cross the road the main vehicle is traveling on, object 1219 exists in front of the main vehicle's lane, and various other attributes of the scene.

[0308] In step 1305, one or more navigation constraints related to the navigation state of the primary vehicle can be determined. For example, after analyzing a scene in the environment of the primary vehicle represented by one or more captured images, at least one processing device can determine one or more navigation constraints related to objects, vehicles, pedestrians, etc., identified through image analysis of the captured images. In some embodiments, at least one processing device can determine at least a first predetermined navigation constraint and a second predetermined navigation constraint related to the navigation state, and the first predetermined navigation constraint may be different from the second predetermined navigation constraint. For example, the first navigation constraint may relate to one or more target vehicles detected in the environment of the primary vehicle, while the second navigation constraint may relate to pedestrians detected in the environment of the primary vehicle.

[0309] In step 1307, at least one processing device may determine the priority associated with the constraints identified in step 1305. In the described example, a second predetermined navigation constraint involving a pedestrian may have a higher priority than a first predetermined navigation constraint involving a target vehicle. While priorities associated with navigation constraints can be determined or assigned based on various factors, in some embodiments, the priority of a navigation constraint may be related to its relative importance from a safety perspective. For example, while it may be important to comply with or satisfy all implemented navigation constraints as many times as possible, some constraints may be associated with a higher safety risk compared to others, and therefore, these constraints may be assigned a higher priority. For example, a navigation constraint requiring the primary vehicle to maintain a distance of at least 1 meter from a pedestrian may have a higher priority than a constraint requiring the primary vehicle to maintain a distance of at least 1 meter from a target vehicle. This may be because a collision with a pedestrian would have more serious consequences than a collision with another vehicle. Similarly, maintaining a distance between the primary vehicle and the target vehicle may have a higher priority than constraints requiring the primary vehicle to avoid boxes on the road, drive below a certain speed on speed bumps, or expose the primary vehicle occupant to no more than the maximum acceleration level.

[0310] Although the driving strategy module 803 is designed to maximize safety by satisfying navigation constraints involved in a specific scenario or navigation state, in some cases it is practically impossible to satisfy every one of the involved constraints. In such cases, as shown in step 1309, the priority of each involved constraint can be used to determine which involved constraint should be satisfied first. Continuing the example above, in cases where it is impossible to satisfy both the pedestrian clearance constraint and the target vehicle clearance constraint simultaneously, and only one of the constraints can be satisfied, the higher-priority pedestrian clearance constraint may cause that constraint to be satisfied before attempting to maintain clearance to the target vehicle. Therefore, under normal circumstances, as shown in step 1311, where both the first predetermined navigation constraint and the second predetermined navigation constraint can be satisfied, at least one processing device can determine a first navigation action of the main vehicle that satisfies both the first and second predetermined navigation constraints based on the identified navigation state of the main vehicle. However, in other cases, where not all the constraints involved can be satisfied, such as in step 1313, where the first predetermined navigation constraint and the second predetermined navigation constraint cannot be satisfied simultaneously, at least one processing device can determine a second navigation action of the master vehicle that satisfies the second predetermined navigation constraint (i.e., the higher priority constraint) but does not satisfy the first predetermined navigation constraint (which has a lower priority than the second navigation constraint) based on the identified navigation state.

[0311] Next, in step 1315, in order to implement the determined navigation action of the master vehicle, at least one processing device may, in response to the determined first navigation action or the determined second navigation action of the master vehicle, cause at least one adjustment to the navigation actuator of the master vehicle. As described in the preceding examples, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0312] Constraint relaxation

[0313] As mentioned above, navigation constraints can be imposed for safety purposes. Constraints may include a minimum safe driving distance relative to a pedestrian, target vehicle, roadblock, or detected object; a maximum driving speed when passing within the influence zone of a detected pedestrian; or a maximum deceleration rate for the primary vehicle, etc. These constraints can be imposed on learning or non-learning navigation systems. In some cases, these constraints can be relaxed. For example, if the primary vehicle slows down or stops near a pedestrian and then slowly moves forward to convey its intention to pass the pedestrian, the pedestrian's response can be detected from the acquired images. If the pedestrian's response is to remain still or stop moving (and / or if eye contact with the pedestrian is sensed), it can be understood that the pedestrian recognizes the navigation system's intention to pass the pedestrian. In this case, the system can relax one or more predetermined constraints and enforce less stringent constraints (e.g., allowing the vehicle to navigate within 0.5 meters of the pedestrian, rather than within a more stringent 1-meter boundary).

[0314] Figure 14 A flowchart is provided for implementing control of a primary vehicle based on the relaxation of one or more navigation constraints. In step 1401, at least one processing device may receive multiple images representing the environment of the primary vehicle from a camera associated with the primary vehicle. Analysis of the images in step 1403 enables the identification of a navigation state associated with the primary vehicle. In step 1405, at least one processor may determine navigation constraints associated with the navigation state of the primary vehicle. Navigation constraints may include a first predetermined navigation constraint relating to at least one aspect of the navigation state. In step 1407, analysis of the multiple images may reveal the presence of at least one navigation constraint relaxation factor.

[0315] Navigation constraint slack factors may include any suitable indicator that one or more navigation constraints can be suspended, altered, or otherwise relaxed in at least one respect. In some embodiments, at least one navigation constraint slack factor may include (based on image analysis) the determination that a pedestrian's eyes are looking in the direction of the main vehicle. In this case, it can be more safely assumed that the pedestrian is aware of the main vehicle. Therefore, the confidence that the pedestrian will not be involved in an unintended action that would cause the pedestrian to move into the main vehicle's path may be higher. Other constraint slack factors may also be used. For example, at least one navigation constraint slack factor may include: a pedestrian determined not to be moving (e.g., a pedestrian assumed to be unlikely to enter the main vehicle's path); or a pedestrian whose movement is determined to be slowing down. Navigation constraint slack factors may also include more complex actions, such as a pedestrian determined not to be moving after the main vehicle stops and then resumes movement. In this case, it can be assumed that the pedestrian knows the main vehicle has the right-of-way, and that the pedestrian's stopping may indicate the pedestrian's intention to yield to the main vehicle. Other situations that may lead to the slack of one or more constraints include the type of curb (e.g., low curbs or curbs with a gradual slope may allow for slack distance constraints), the absence of pedestrians or other objects on the sidewalk, vehicles whose engines are not running may have slack distances, or situations where pedestrians are moving away from and / or away from the area that the main vehicle is traveling towards.

[0316] Upon identification of a navigation constraint slack factor (e.g., in step 1407), a second navigation constraint can be determined or generated in response to the detection of the constraint slack factor. This second navigation constraint may differ from the first navigation constraint, and may include at least one characteristic that is relaxed relative to the first navigation constraint. The second navigation constraint may include a newly generated constraint based on the first constraint, wherein the newly generated constraint includes at least one modification that relaxes the first constraint in at least one aspect. Alternatively, the second constraint may constitute a predetermined constraint that is less stringent than the first navigation constraint in at least one aspect. In some embodiments, such a second constraint may be retained only for cases where constraint slack factors are identified in the environment of the primary vehicle. Whether the second constraint is newly generated or selected from a set of fully or partially available predetermined constraints, applying the second navigation constraint in place of the more stringent first navigation constraint (which may be applied where no relevant navigation constraint slack factor is detected) can be referred to as constraint slack and can be completed in step 1409.

[0317] If at least one constraint relaxation factor is detected in step 1407, and at least one constraint has been relaxed in step 1409, the navigation action of the master vehicle can be determined in step 1411. The navigation action of the master vehicle can be based on the identified navigation state and can satisfy a second navigation constraint. The navigation action can be implemented in step 1413 by causing at least one adjustment to the navigation actuator of the master vehicle in response to the determined navigation action.

[0318] As described above, the use of navigation constraints and relaxed navigation constraints can be applied to navigation systems that are trained (e.g., through machine learning) or untrained (e.g., a navigation system programmed to respond with a predetermined action in response to a specific navigation state). In the case of a trained navigation system, the availability of relaxed navigation constraints for certain navigation situations can represent a mode switch from trained system response to untrained system response. For example, a trained navigation network can determine the initial navigation action of the host vehicle based on a first navigation constraint. However, the action taken by the vehicle can be different from the navigation action that satisfies the first navigation constraint. Instead, the action taken can satisfy a more relaxed second navigation constraint and can be an action generated by an untrained system (e.g., as a response to detecting a specific condition in the host vehicle's environment, such as the presence of a navigation constraint relaxation factor).

[0319] There are numerous examples of navigation constraints that can be relaxed in response to the detection of constraint relaxation factors in the environment of the master vehicle. For instance, where the predetermined navigation constraints include a buffer associated with a detected pedestrian, and at least a portion of the buffer extends a certain distance from the detected pedestrian, the relaxed navigation constraints (newly generated, retrieved from memory from a predetermined set, or generated as a relaxed version of a previously existing constraint) can include different or modified buffers. For example, the different or modified buffers may have a smaller distance relative to the pedestrian than the original or unmodified buffer relative to the detected pedestrian. Thus, if an appropriate constraint relaxation factor is detected in the environment of the master vehicle, the master vehicle can be allowed to navigate closer to the detected pedestrian, given the relaxed constraints.

[0320] As described above, the relaxed characteristics of navigation constraints may include a reduction in the width of the buffer associated with at least one pedestrian. However, the relaxed characteristics may also include a reduction in the width of the buffer associated with the target vehicle, a detected object, a roadside obstacle, or any other object detected in the environment of the primary vehicle.

[0321] At least one relaxed characteristic may also include other types of modifications to the navigation constraint characteristics. For example, a relaxed characteristic may include an increase in the rate associated with at least one predetermined navigation constraint. A relaxed characteristic may also include an increase in the maximum permissible deceleration / acceleration associated with at least one predetermined navigation constraint.

[0322] While constraints can be relaxed in some cases, as mentioned above, navigation constraints can be augmented in others. For example, in some situations, the navigation system can determine the conditions under which an enhancement to a set of normal navigation constraints is warranted. Such enhancement may include adding new constraints to a set of predetermined constraints or adjusting one or more aspects of predetermined constraints. Additions or adjustments may result in more conservative navigation relative to the set of predetermined constraints applicable under normal driving conditions. Conditions under which constraint enhancement can be warranted may include sensor malfunctions, adverse environmental conditions (rain, snow, fog, or other conditions associated with reduced visibility or reduced vehicle traction), etc.

[0323] Figure 15 A flowchart is provided for implementing control of a primary vehicle based on enhancements to one or more navigation constraints. In step 1501, at least one processing device may receive multiple images representing the environment of the primary vehicle from a camera associated with the primary vehicle. In step 1503, analysis of the images enables the identification of a navigation state associated with the primary vehicle. In step 1505, at least one processor may determine navigation constraints associated with the navigation state of the primary vehicle. The navigation constraints may include a first predetermined navigation constraint relating to at least one aspect of the navigation state. In step 1507, analysis of the multiple images may reveal the presence of at least one navigation constraint enhancement factor.

[0324] The navigation constraints involved may include those mentioned above (e.g., regarding...). Figure 12Any navigation constraint or any other suitable navigation constraint. Navigation constraint enhancement factors may include any indicator that one or more navigation constraints can be supplemented / enhanced in at least one aspect. Supplementation or enhancement of navigation constraints may be performed on a per-group basis (e.g., by adding a new navigation constraint to a pre-defined set of constraints) or on a per-constraint basis (e.g., by modifying a particular constraint such that the modified constraint is more restrictive than the original constraint, or by adding a new constraint corresponding to a pre-defined constraint, wherein the new constraint is more restrictive than the corresponding constraint in at least one aspect). Alternatively or additionally, supplementation or enhancement of navigation constraints may refer to selection from a set of pre-defined constraints based on hierarchy. For example, a set of enhanced constraints may be used for selection based on whether a navigation enhancement factor is detected in or relative to the host vehicle's environment. In the normal case where no enhancement factor is detected, the relevant navigation constraints may be extracted from constraints applicable to the normal case. On the other hand, in the case where one or more constraint enhancement factors are detected, the relevant constraints may be extracted from enhanced constraints that are generated or pre-defined relative to one or more enhancement factors. The enhanced constraints may be more restrictive in at least one aspect compared to the corresponding constraints applicable in the normal case.

[0325] In some embodiments, at least one navigation constraint enhancement factor may include detecting (e.g., based on image analysis) the presence of ice, snow, or water on a road surface in the environment of the primary vehicle. For example, such determination may be based on the detection of: areas with higher reflectivity than expected on a dry road (e.g., indicating ice or water on the road); white areas on the road indicating the presence of snow; shadows on the road consistent with the presence of longitudinal grooves on the road (e.g., tire tracks in snow); water droplets or ice / snow particles on the primary vehicle's windshield; or any other suitable indicator of the presence of water or ice / snow on the road surface.

[0326] At least one navigation constraint enhancement factor may also include particles detected on the outer surface of the windshield of the host vehicle. Such particles may impair the image quality of one or more image-capturing devices associated with the host vehicle. While the description pertains to the windshield of the host vehicle in relation to a camera mounted behind the windshield, the detection of particles on other surfaces (e.g., camera lenses or lens caps, headlight lenses, rear windshields, taillight lenses, or any other surface of the host vehicle visible to (or detected by) image-capturing devices associated with the host vehicle) may also indicate the presence of a navigation constraint enhancement factor.

[0327] Navigation constraint enhancement factors can also be detected as attributes of one or more image acquisition devices. For example, a detected reduction in image quality of one or more images captured by an image capture device (e.g., a camera) associated with the primary vehicle can also constitute a navigation constraint enhancement factor. The degradation in image quality may be associated with hardware malfunctions or partial hardware malfunctions related to the image capture device or components associated with it. This degradation in image quality can also be caused by environmental conditions. For example, the presence of smoke, fog, rain, snow, etc., in the air surrounding the primary vehicle can also lead to reduced image quality relative to roads, pedestrians, target vehicles, etc., that may be present in the environment of the primary vehicle.

[0328] Navigation constraint enhancement factors can also involve other aspects of the primary vehicle. For example, in some cases, navigation constraint enhancement factors may include detected faults or partial faults in systems or sensors associated with the primary vehicle. Such enhancement factors may include, for example, detected faults or partial faults in speed sensors, GPS receivers, accelerometers, cameras, radar, lidar, brakes, tires, or any other system associated with the primary vehicle that could affect the primary vehicle's ability to navigate relative to navigation constraints associated with the primary vehicle's navigation state.

[0329] Upon identifying the presence of a navigation constraint enhancement factor (e.g., in step 1507), a second navigation constraint can be determined or generated in response to the detection of the constraint enhancement factor. This second navigation constraint may differ from the first navigation constraint, and may include at least one characteristic that is enhanced relative to the first navigation constraint. The second navigation constraint may be more restrictive than the first navigation constraint because detecting constraint enhancement factors in or associated with the primary vehicle's environment may indicate that the primary vehicle may have at least one reduced navigation capability relative to normal operating conditions. Such reduced capability may include lower road traction (e.g., ice, snow, or water on the road; reduced tire pressure, etc.); impaired vision (e.g., rain, snow, dust, smoke, fog, etc., reducing the quality of captured images); impaired detection capabilities (e.g., sensor malfunction or partial malfunction, reduced sensor performance, etc.), or any other reduction in the primary vehicle's ability to navigate in response to a detected navigation state.

[0330] If at least one constraint enhancement factor is detected in step 1507, and at least one constraint is enhanced in step 1509, a navigation action for the primary vehicle can be determined in step 1511. The navigation action for the primary vehicle can be based on the identified navigation state and can satisfy a second navigation (i.e., enhanced) constraint. The navigation action can be implemented in step 1513 by inducing at least one adjustment to the navigation actuator of the primary vehicle in response to the determined navigation action.

[0331] As discussed, the use of navigation constraints and enhanced navigation constraints can be employed for both trained (e.g., through machine learning) and untrained (e.g., systems programmed to respond with predetermined actions in response to specific navigation states) navigation systems. In the case of a trained navigation system, the availability of enhanced navigation constraints for certain navigation situations can represent a mode switch from trained to untrained system responses. For example, a trained navigation network can determine the initial navigation action of the primary vehicle based on a first navigation constraint. However, the action taken by the vehicle can be different from the navigation action that satisfies the first navigation constraint. Instead, the action taken can satisfy an enhanced second navigation constraint and can be an action generated by an untrained system (e.g., as a response to the detection of specific conditions in the primary vehicle's environment, such as the presence of navigation constraint enhancement factors).

[0332] There are numerous examples of navigation constraints that can be generated, supplemented, or enhanced in response to the detection of constraint enhancement factors in the environment of the primary vehicle. For instance, where the predetermined navigation constraints include buffers associated with detected pedestrians, objects, vehicles, etc., and at least a portion of the buffers extends a certain distance from the detected pedestrian / object / vehicle, the enhanced navigation constraints (newly generated, retrieved from memory from a predetermined set, or generated as an enhanced version of a previously existing constraint) may include different or modified buffers. For example, the different or modified buffers may have a larger distance relative to the pedestrian / object / vehicle than the original or unmodified buffer relative to the detected pedestrian / object / vehicle. Therefore, if an appropriate constraint enhancement factor is detected in or relative to the primary vehicle's environment, the enhanced constraint can force the primary vehicle to navigate further away from the detected pedestrian / object / vehicle.

[0333] At least one enhanced characteristic may also include other types of modifications to the navigation constraint characteristics. For example, the enhanced characteristic may include a reduction in velocity associated with at least one predetermined navigation constraint. The enhanced characteristic may also include a reduction in the maximum permissible deceleration / acceleration associated with at least one predetermined navigation constraint.

[0334] Navigation based on long-term planning

[0335] In some embodiments, the disclosed navigation system can not only respond to a navigation state detected in the environment of the host vehicle, but also determine one or more navigation actions based on long-range planning. For example, the system can consider the potential impact of one or more navigation actions, which are options for navigating relative to the detected navigation state, on future navigation states. Considering the impact of available actions on future states allows the navigation system to determine navigation actions not only based on the currently detected navigation state, but also based on long-range planning. Navigation using long-range planning techniques may be particularly suitable where the navigation system employs one or more reward functions as a technique for selecting navigation actions from available options. Potential rewards can be analyzed relative to available navigation actions that can be taken in response to the detected current navigation state of the host vehicle. However, further, potential rewards can also be analyzed relative to actions that can be taken in response to future navigation states, which are expected to result from available actions in response to the current navigation state. Therefore, the disclosed navigation system can, in certain circumstances, select a navigation action from available actions that can be taken in response to the current navigation state in response to the detected navigation state, even if the selected navigation action may not produce the highest reward. This is especially true when the system determines that the selected action may lead to a future navigation state that gives rise to one or more potential navigation actions, where the potential actions offer a higher reward than the selected action, or in some cases, than any action available relative to the current navigation state. This principle can be expressed more simply as: taking a less advantageous action now in order to generate a higher reward option in the future. Therefore, the disclosed navigation system capable of long-term planning can select actions that are not optimal in the short term, where a short-term loss of reward indicated by long-term prediction leads to an increase in long-term reward.

[0336] Typically, autonomous driving applications involve a series of planning problems where the navigation system decides immediate actions to optimize a longer-term goal. For example, when a vehicle encounters a lane-changing situation at a roundabout, the navigation system can decide on an immediate acceleration or braking command to initiate navigation into the roundabout. While the immediate action at the roundabout in response to a detected navigation state might involve an acceleration or braking command in response to the detected state, the long-term goal is a successful lane change, and the long-term effect of the chosen command is the success or failure of the lane change. The planning problem can be addressed by decomposing it into two phases. First, supervised learning can be applied to predict the near future based on the present (assuming the predictor's representation with respect to the present will be differentiable). Second, a recurrent neural network can be used to model the complete trajectory of the actors, where unexplained factors are modeled as (additive) input nodes. This allows the use of supervised learning techniques and direct optimization of the recurrent neural network to determine solutions to the long-term planning problem. This approach also enables the learning of robust policies by incorporating adversarial elements into the environment.

[0337] The two most fundamental elements of an autonomous driving system are sensing and planning. Sensing deals with finding a compact representation of the current state of the environment, while planning deals with deciding what actions to take to optimize future goals. Supervised machine learning techniques are useful for solving the sensing problem. Machine learning algorithm frameworks can also be used for the planning part, especially reinforcement learning (RL) frameworks, such as those mentioned above.

[0338] RL can be executed sequentially in several rounds. In round t, the planner (also known as the actor or driving strategy module 803) can observe the state s. t ∈S, which represents the actor and the environment. Then it should determine the action a. t ∈A. After performing this action, the actor receives an immediate reward. And move to the new state s t+1 As an example, the primary vehicle may include an adaptive cruise control (ACC) system, where the vehicle should autonomously apply acceleration / braking to maintain an appropriate distance from the vehicle in front while maintaining smooth driving. This state can be modeled as a pair Where x t It's the distance to the car in front, v t It is the speed of the main vehicle relative to the vehicle in front. The action... It will be an acceleration command (if a) t (If <0, the main vehicle decelerates). The reward may depend on |a t |(reflects driving smoothness) and s t(A function reflecting the safe distance maintained between the lead vehicle and the vehicle in front). The planner's goal is to maximize the cumulative reward (either up to a timeframe for future rewards or the sum of discounted rewards). To do this, the planner can rely on policy π: S→A, which maps states to actions.

[0339] Supervised learning (SL) can be viewed as a special case of RL, where s t It is sampled from some distribution on S, and the reward function can have In the form of, It is the loss function, and the learner observes y. t The value of y t To view status s t The optimal action to take (which may be noisy). There may be several differences between the typical RL model and the special case SL, and these differences make the typical RL problem more challenging.

[0340] In some cases of SolidWorks (SL), the actions (or predictions) taken by the learner may have no impact on the environment. In other words, s t+1 and a t They are independent. This has two important implications. First, in SL, samples (s1, y1), ..., (s m y m The states (s) can be collected in advance before the search for a strategy (or predictor) with good accuracy relative to the samples can begin. Conversely, in RL, the states (s) are collected before the search for a strategy (or predictor) with good accuracy relative to the samples can begin. t+1 Typically, the choice depends on the action taken (and the previous state), which in turn depends on the policy used to generate that action. This binds the data generation process to the policy learning process. Secondly, because actions in SL do not affect the environment, the choice of a... t The contribution to the representation of π is local. Specifically, a t It only affects the value of the immediate reward. In contrast, in RL, an action taken in round t may have a long-term impact on the reward value in future rounds.

[0341] In SL, the knowledge of the "correct" answer is y t Along with rewards The shape together can provide for a t Having complete knowledge of the rewards of all possible choices allows us to calculate the reward relative to a. tThe derivative of the reward. Conversely, in RL, the "one-off" value of the reward may be all that is observable for the specific choice of action taken. This can be called "bandit" feedback. This is one of the most important reasons why "exploration" is needed as part of long-term navigation planning, because in RL-based systems, if only "bandit" feedback is available, the system may not always know whether the action taken is the best action to take.

[0342] Many RL algorithms rely at least in part on the mathematical optimization model of Markov Decision Processes (MDPs). The Markov assumption is: given s... t and a t s t+1 The distribution of the policy is completely determined. Based on the stable distribution of the MDP states, a closed-form expression for the cumulative reward of a given policy is given. The stable distribution of the policy can be represented as a solution to a linear programming problem. This leads to two classes of algorithms: 1) optimization relative to the primal problem, known as policy search; and 2) optimization relative to the dual problem, whose variables are called the value function V. π If the MDP starts from an initial state s and selects actions from there based on π, then the value function determines the expected cumulative reward. The relevant quantity is the state-action value function Q. π (s, a), assuming starting from state s, immediately choosing action a, and from there choosing actions according to π, the state-action value function determines the cumulative reward. The Q-function may lead to a characterization of the optimal policy (using the Bellman equation). In particular, the Q-function may indicate that the optimal policy is a deterministic function from S to the end (in fact, it may be characterized as a "greedy" policy relative to the optimal Q-function).

[0343] One potential advantage of the MDP model is that it allows coupling the future to the present using the Q function. For example, suppose the main vehicle is currently in state s, Q... π The value of (s, a) can indicate the impact of performing action a on the future. Therefore, the Q function can provide a local measure of the quality of action a, making the RL problem more similar to the SL scenario.

[0344] Many RL algorithms approximate the V or Q function in some way. Value iterative algorithms (such as Q-learning algorithms) can rely on the fact that the V and Q functions of the optimal policy can be fixed points of some operators derived from the Bellman equations. Actor-critic policy iterative algorithms aim to learn the policy in an iterative manner, where at iteration t, the "criterionist" estimates... And based on this estimate, the "actors" improved their strategies.

[0345] Despite the mathematical advantages of MDPs and the convenience of switching to Q-function representations, this approach may have several limitations. For example, the approximate concept of Markov behavioral states may be all that can be found in some cases. Furthermore, state transitions depend not only on the actions of the actor but also on the actions of other actors in the environment. For instance, in the ACC example mentioned above, although the dynamics of the autonomous vehicle may be Markovian, the next state may depend on the actions of the driver of another vehicle, which are not necessarily Markovian. One possible solution to this problem is to use a partially observed MDP, where it is assumed that Markovian states exist, but what is observed is a distribution based on the hidden states.

[0346] A more direct approach could be to consider game theory generalizations of MDPs (e.g., stochastic game frameworks). In fact, algorithms used for MDPs can be generalized to multi-player games (e.g., min-max-Q learning or Nash-Q learning). Other approaches could include explicit modeling of other players and vanishing-regret learning algorithms. Learning in multi-player scenarios can be more complex than in single-player scenarios.

[0347] A second constraint on the Q-function representation can arise by deviating from the tabular setting. The tabular setting occurs when the number of states and actions is small, so Q can be represented as a table with |S| rows and |A| columns. However, if the natural representations of S and A involve Euclidean space, and the state and action spaces are discrete, the number of states / actions can be exponential in dimensionality. In this case, adopting the tabular setting may not be practical. Instead, the Q-function can be approximated by a function derived from the category of parameter assumptions (e.g., a neural network of a certain architecture). For example, the Deep Q-Network (DQN) learning algorithm can be used. In DQN, the state space can be continuous, while the action space can still be a small, discrete set. Methods for handling continuous action spaces may exist, but they may rely on approximations of the Q-function. In any case, the Q-function can be complex and sensitive to noise, thus potentially posing a challenge to learning.

[0348] A different approach could be to use recurrent neural networks (RNNs) to solve RL problems. In some cases, RNNs can be combined with the concept of multi-player games and robustness to adversarial environments from game theory. Furthermore, this approach may not explicitly rely on any Markov assumptions.

[0349] The following describes in more detail the method of navigation through prediction-based planning. In this method, it can be assumed that the state space S is... A subset of, and the action space A is A subset of. This may be a natural behavior in many applications. As mentioned above, there may be two key differences between RL and SL: (1) because past actions affect future rewards, information from the future may need to be propagated back to the past; (2) the “robbery” nature of rewards may blur the dependency between state, action and reward, which can complicate the learning process.

[0350] As a first step in this method, an interesting problem can be observed where the robbery nature of the reward is not the issue. For example, the reward value of the ACC application (discussed in more detail below) may be differentiable with respect to the current state and action. In fact, even if the reward is given in a "robbery" manner, learning a differentiable function... Make The problem can be a relatively straightforward SL problem (e.g., a one-dimensional regression problem). Therefore, the first step of this method could be to define the reward as a function. The function is differentiable with respect to s and a, or the first step of this method could be to use a regression learning algorithm to learn the differentiable function. This function minimizes at least some regression loss on the samples, where the instance vector is... And the target scalar is r t In some cases, exploratory elements can be used to create training sets.

[0351] A similar idea can be used to address the connection between the past and the future. For example, suppose we can learn a differentiable function. Make Learning such a function can be characterized as a learning problem (SL). It can be considered a near-future predictor. Next, the parametric function π can be used. θ Let S→A be used to describe the strategy of mapping from S to A. Let π... θ Representing it as a neural network allows the use of recurrent neural networks (RNNs) to represent segments of an actor running for T rounds, where the next state is defined as... Here, It can be defined by the environment and can express unpredictable aspects of the near future. t+1 Dependent on s in a differentiable manner t and a t The fact enables a link between future rewards and past behavior. The policy function π θThe parameter vector can be learned by backpropagation on the obtained RNN. Note that it is not necessary to learn the parameter vector of v. t Explicit probability assumptions are imposed on it. In particular, no Markov relation requirement is needed. Instead, it can rely on the recurrent network to propagate “sufficient” information between the past and future. Intuitively, It can describe the predictable part of the near future, while v t It can express unpredictable aspects that may arise due to the behavior of other participants in the environment. The learning system should learn strategies that are robust to the behavior of other participants. If ||ν t The relationship between past actions and future rewards can be too noisy to learn meaningful policies. Explicitly expressing the system's dynamics in a transparent manner makes it easier to incorporate prior knowledge. For example, prior knowledge can be simplified in definition. The problem.

[0352] As mentioned above, the learning system can benefit from robustness relative to adversarial environments, such as the environment of the primary vehicle, which may include multiple other drivers who might act in unexpected ways. Without considering v t In models that impose probabilistic assumptions, it can be considered that the choice of v is adversarial. t The environment. In some cases, it is possible to target μ. t Imposing constraints is necessary, otherwise the opposing side might make the planning problem difficult or even impossible. One natural constraint might be requiring a constant to limit ||μ. t ||.

[0353] Robustness to adversarial environments could be useful in autonomous driving applications. Selecting μ in an adversarial manner... t It can even accelerate the learning process because it focuses the learning system on robust, optimal strategies. This concept can be illustrated using a simple game theory approach. The state is... Action as The instantaneous loss function is 0.1|a t |+[|s t |-2] + , where [x] + =max{x, 0} is the ReLU (corrected linear unit) function. The next state is s. t+1 =s t +a t +v t , where v t ∈[-0.5, 0.5] represents the adversarial choice made against the environment. Here, the optimal policy can be written as a two-layer network with ReLU: a t =[s t-1.5] + + [-s t -1.5] + It was observed that when |s t When |∈(1.5, 2], the optimal action may have a larger immediate loss than the action a=0. Therefore, the system can plan for the future and can rely on more than just immediate loss. It is observed that the loss is related to a t The derivative is 0.1sign(a) t ), and about s t The derivative is 1[|s t |>2]sign(s t ). In s t In the case of ∈(1.5, 2], v t The adversarial choice will set v t =0.5, therefore, whenever a t >1.5-s t In round t+1, there can be a non-zero loss. In this case, the derivative of the loss can be directly propagated back to a. t Therefore, in a t In the case that the choice is not optimal, v t Adversarial choices can help navigation systems obtain non-zero backpropagation messages. This relationship can help navigation systems choose current actions based on the expectation that this current action (even if it leads to suboptimal rewards or even losses) will provide opportunities for more optimal actions that lead to higher rewards in the future.

[0354] This approach can be applied to virtually any navigation situation that might arise. The following description applies to an example approach: Adaptive Cruise Control (ACC). In the ACC problem, the primary vehicle might attempt to maintain an appropriate distance from a target vehicle ahead (e.g., 1.5 seconds from the target vehicle). Another objective might be to drive as smoothly as possible while maintaining the desired clearance. A model representing this situation can be defined as follows. The state space is... Action space is The first coordinate of the state is the speed of the target car, the second coordinate is the speed of the master vehicle, and the last coordinate is the distance between the master vehicle and the target vehicle (e.g., the position of the master vehicle along the road curve minus the position of the target vehicle). The action taken by the master vehicle is to accelerate, and this can be expressed as 'a'. t Let τ represent the time difference between consecutive rounds. While τ can be set to any suitable quantity, in one example, τ could be 0.1 seconds. Position s t It can be represented as And the (unknown) acceleration of the target vehicle can be expressed as

[0355] The complete dynamics of the system can be described as follows:

[0356]

[0357]

[0358]

[0359] This can be described as the sum of two vectors:

[0360]

[0361] The first vector represents the predictable portion, while the second vector represents the unpredictable portion. The reward for round t is defined as follows:

[0362] in

[0363] The first term results in a penalty for non-zero acceleration, thus encouraging smooth driving. The second term depends on the distance x to the target car. t Distance from expectations The ratio between the desired distance and the distance traveled is defined as the maximum value between a distance of 1 meter and a braking distance of 1.5 seconds. In some cases, this ratio may be exactly 1, but as long as the ratio is within the range of [0.7, 1.3], the strategy may waive any penalty, which may allow the primary vehicle to relax its navigation—a feature that may be important in achieving smooth driving.

[0364] By implementing the methods outlined above, the navigation system of the main vehicle (e.g., through the operation of the driving strategy module 803 within the processing unit 110 of the navigation system) can select an action in response to the observed state. The selected action can be based not only on the analysis of rewards associated with responsive actions available relative to the sensed navigation state, but also on consideration and analysis of future states, potential actions in response to future states, and rewards associated with potential actions.

[0365] Figure 16 An algorithmic approach to navigation based on detection and long-term planning is illustrated. For example, in step 1601, at least one processing device 110 of the main vehicle's navigation system may receive multiple images. These images may capture scenes representing the environment of the main vehicle and may be provided by any of the image capture devices described above (e.g., cameras, sensors, etc.). Analyzing one or more of these images in step 1603 enables at least one processing device 110 to identify the current navigation state associated with the main vehicle (as described above).

[0366] In steps 1605, 1607, and 1609, various potential navigation actions in response to the sensed navigation state can be determined. These potential navigation actions (e.g., first navigation action to Nth available navigation action) can be determined based on the sensed state and the long-term goals of the navigation system (e.g., completing a merge, smoothly following the vehicle in front, overtaking a target vehicle, avoiding objects in the road, decelerating in response to a detected stop sign, avoiding an intruding target vehicle, or any other navigation action that can advance the system's navigation goals).

[0367] For each of the identified potential navigation actions, the system can determine an expected reward. The expected reward can be determined according to any of the techniques described above and may include analysis of one or more reward functions for a particular potential action. For each potential navigation action (e.g., first, second, and Nth) identified in steps 1605, 1607, and 1609, expected rewards 1606, 1608, and 1610 can be determined, respectively.

[0368] In some cases, the primary vehicle's navigation system may select from available potential actions based on values ​​associated with expected rewards 1606, 1608, and 1610 (or any other type of indicator of expected reward). For example, in some situations, the action that produces the highest expected reward may be selected.

[0369] In other cases, particularly when the navigation system is performing long-term planning to determine the primary vehicle's navigation actions, the system may not select the potential action that yields the highest expected reward. Instead, the system can look to the future to analyze whether there is an opportunity to achieve a higher reward later if a lower-reward action is selected in response to the current navigation state. For example, for any or all of the potential actions determined in steps 1605, 1607, and 1609, future states can be determined. Each future state determined in steps 1613, 1615, and 1617 can represent a future navigation state expected to be obtained based on the current navigation state, modified by the corresponding potential action (e.g., the potential actions determined in steps 1605, 1607, and 1609).

[0370] For each of the future states predicted in steps 1613, 1615, and 1617, one or more future actions (as navigation options available in response to the determined future state) can be identified and evaluated. At steps 1619, 1621, and 1623, for example, values ​​of expected rewards or any other type of indicator associated with the one or more future actions can be generated (e.g., based on one or more reward functions). The expected reward associated with the one or more future actions can be evaluated by comparing the values ​​of the reward functions associated with each future action or by comparing any other indicator associated with the expected reward.

[0371] In step 1625, the primary vehicle's navigation system may select a navigation action based on a comparison of anticipated rewards, not only based on potential actions identified relative to the current navigation state (e.g., in steps 1605, 1607, and 1609), but also based on anticipated rewards determined as a result of potential future actions available in response to a predicted future state (e.g., determined at steps 1613, 1615, and 1617). The selection at step 1625 may be based on the option and reward analysis performed at steps 1619, 1621, and 1623.

[0372] The selection of a navigation action at step 1625 can be based solely on a comparison of the expected rewards associated with future action options. In this case, the navigation system can select an action for the current state based solely on a comparison of the expected rewards derived from actions to potential future navigation states. For example, the system could select a potential action identified at steps 1650, 1610, and 1609 that is associated with the highest future reward value determined through the analysis at steps 1619, 1621, and 1623.

[0373] The selection of a navigation action in step 1625 can also be based solely on a comparison of current action options (as described above). In this case, the navigation system can select the potential action identified in steps 1605, 1607, or 1609 that is associated with the highest expected reward 1606, 1608, or 1610. This selection can be performed with little or no consideration of future navigation states or the future expected reward of available navigation actions in response to expected future navigation states.

[0374] On the other hand, in some cases, the selection of a navigation action in step 1625 can be based on a comparison of the expected rewards associated with both future action options and current action options. In fact, this could be based on one of the navigation principles of long-term planning. For example, the expected reward for a future action can be analyzed to determine if there are any expected rewards that authorize a selection of a lower-reward action in response to the current navigation state in order to achieve a potentially higher reward in response to a subsequent navigation action that is expected to be available in response to a future navigation state. As an example, the value of expected reward 1606 or other indicators could indicate the highest expected reward among rewards 1606, 1608, and 1610. On the other hand, expected reward 1608 could indicate the lowest expected reward among rewards 1606, 1608, and 1610. Instead of simply selecting the potential action determined in step 1605 (i.e., the action that results in the highest expected reward 1606), the analysis of future states, potential future actions, and future rewards can be used to select a navigation action in step 1625. In one example, it can be determined that the reward identified in step 1621 (in response to at least one future action in response to a future state determined in step 1615, which is based on a second potential action determined in step 1607) may be higher than the expected reward 1606. Based on this comparison, the second potential action determined in step 1607 can be selected instead of the first potential action determined in step 1605, even though the expected reward 1606 is higher than the expected reward 1608. In one example, the potential navigation action determined in step 1605 may include merging lanes in front of the detected target vehicle, while the potential navigation action determined in step 1607 may include merging lanes behind the target vehicle. Although the expected reward 1606 for merging lanes in front of the target vehicle may be higher than the expected reward 1608 associated with merging lanes behind the target vehicle, it can be determined that merging lanes behind the target vehicle may result in a future state for which there may be action options that produce potential rewards higher than the expected rewards 1606, 1608, or other available actions based on the current, sensed navigation state.

[0375] The selection from potential actions at step 1625 can be based on any suitable comparison of anticipated rewards (or any other measure or indicator of the benefit associated with a potential action compared to each other). In some cases, as described above, if a second potential action is expected to provide at least one future action associated with an anticipated reward higher than that associated with the first potential action, the second potential action can be selected over the first. In other cases, more complex comparisons can be employed. For example, the reward associated with an action option in response to a anticipated future state can be compared with more than one anticipated reward associated with a identified potential action.

[0376] In some cases, if at least one future action is expected to produce a reward higher than any reward expected as a result of a potential action to the current state (e.g., expected rewards 1606, 1608, 1610, etc.), then the action based on the expected future state and the expected reward can influence the selection of a potential action to the current state. In some cases, the future action option that produces the highest expected reward (e.g., from the expected reward associated with a potential action to the sensed current state, and from the expected reward associated with a potential future action option relative to a potential future navigation state) can be used as guidance for selecting a potential action to the current navigation state. That is, after identifying the future action option that produces the highest expected reward (or a reward above a predetermined threshold, etc.), step 1625 can select a potential action that will lead to a future state associated with the identified future action that produces the highest expected reward.

[0377] In other cases, the selection of available actions can be based on the difference determined between expected rewards. For example, if the difference between the expected reward associated with the future action determined in step 1621 and expected reward 1606 is greater than the difference between expected reward 1608 and expected reward 1606 (assuming a difference in positive signs), then the second potential action determined in step 1607 can be selected. In another example, if the difference between the expected reward associated with the future action determined in step 1621 and the expected reward associated with the future action determined in step 1619 is greater than the difference between expected reward 1608 and expected reward 1606, then the second potential action determined in step 1607 can be selected.

[0378] Several examples of selecting from potential actions for the current navigation state have been described. However, any other suitable comparison technique or criterion can be used to select available actions through long-term planning based on action and reward analysis extending to anticipated future states. Furthermore, although Figure 16This represents two layers in long-term planning analysis (e.g., the first layer considers the rewards from potential actions in response to the current state, while the second layer considers the rewards from future action options in response to the anticipated future state). However, analysis based on more layers is possible. For example, instead of basing long-term planning analysis on one or two layers, three, four, or more layers of analysis can be used to select from available potential actions in response to the current navigation state.

[0379] After selecting from potential actions in response to the sensed navigation state, in step 1627, at least one processor may cause at least one adjustment action on the navigation actuator of the host vehicle in response to the selected potential navigation. The navigation actuator may include any suitable means for controlling at least one aspect of the host vehicle. For example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0380] Navigation based on inference of other parties' aggression

[0381] Target vehicles can be monitored by analyzing acquired image streams to identify indicators of driving aggression. This paper describes aggression as a qualitative or quantitative parameter, but other characteristics can be used: perceived levels of attention (potential driver injury, distraction from a mobile phone, drowsiness, etc.). In some cases, a target vehicle can be perceived as having a defensive posture, and in others, a more aggressive posture can be identified. Navigation actions can be selected or generated based on aggression indicators. For example, in some cases, relative speed, relative acceleration, increase in relative acceleration, following distance, etc., relative to the lead vehicle can be tracked to determine whether the target vehicle is aggressive or defensive. For instance, if a target vehicle is identified as having an aggression level exceeding a threshold, the lead vehicle may be inclined to yield to the target vehicle. The level of aggression of a target vehicle can also be identified based on its behavior relative to one or more obstacles in or near its path (e.g., vehicles ahead, road obstacles, traffic lights, etc.).

[0382] As an introduction to this concept, an exemplary experiment will be described relative to a primary vehicle merging into a roundabout, where the navigation objective is to cross and exit the roundabout. This scenario can begin with the primary vehicle approaching the entrance to the roundabout and end with the primary vehicle reaching the exit (e.g., a second exit). Success can be measured based on whether the primary vehicle consistently maintains a safe distance from all other vehicles, whether the primary vehicle completes the route as quickly as possible, and whether the primary vehicle follows a smooth acceleration strategy. In this example, N TA target vehicle can be randomly placed on a roundabout. To model a mixture of adversarial and typical behaviors, at probability p, the target vehicle can be modeled as having an "aggressive" driving strategy, whereby the aggressive target vehicle accelerates when the master vehicle attempts to merge in front of it. At probability 1-p, the target vehicle can be modeled as having a "defensive" driving strategy, whereby the target vehicle decelerates and allows the master vehicle to merge. In this experiment, p = 0.5, and the master vehicle's navigation system may not provide information about the types of other drivers. The types of other drivers can be randomly selected at the beginning of the segment.

[0383] Navigation state can be represented as the velocity and position of the primary vehicle (actor), and the position, velocity, and acceleration of the target vehicles. Maintaining target acceleration observation may be important to differentiate between aggressive and defensive drivers based on the current state. All target vehicles can potentially move along a one-dimensional curve with a circular path as its contour. The primary vehicle can move along its own one-dimensional curve, which intersects with the target vehicle's curve at a merging point, and this point is the starting point of both curves. To model rational driving, the absolute values ​​of acceleration for all vehicles can be bounded by constants. Since reverse driving is not allowed, velocity can also be conveyed via ReLU. Note that by disallowing reverse driving, long-term planning may become necessary, as the actor will not regret their past actions.

[0384] As mentioned above, the next state s t+1 It can be decomposed into predictable parts and unpredictable parts ν t The sum. Expression It can represent the dynamics of vehicle position and velocity (which can be well defined in a differentiable way), while ν t It can represent the acceleration of the target vehicle. This can be verified. It can be represented as a combination of ReLU functions over affine transformations, therefore its relation to s t and a t It is differentiable. The vector ν t It can be defined by the simulator in a non-differentiable manner, and can exhibit aggressive behavior towards some targets and defensive behavior towards others. Figure 17A and 17B The image shows two frames from this simulator. In this exemplary experiment, the master vehicle 1701 learns to slow down as it approaches the entrance to the roundabout. It also learns to give way to aggressive vehicles (e.g., vehicles 1703 and 1705) and to safely continue driving when merging ahead of defensive vehicles (e.g., vehicles 1706, 1708, and 1710). Figure 17A and 17BIn the example shown, the navigation system of the primary vehicle 1701 does not provide the type of the target vehicle. Instead, it determines whether a particular vehicle is identified as aggressive or defensive by inference based on observed (e.g., the target vehicle's) position and acceleration. Figure 17A Based on position, speed, and / or relative acceleration, the primary vehicle 1701 can determine that the target vehicle 1703 has an aggressive tendency, and therefore the primary vehicle 1701 can stop and wait for the target vehicle 1703 to pass rather than attempting to merge in front of the target vehicle 1703. However, in Figure 17B In the process, target vehicle 1701 identifies that target vehicle 1710, which is traveling behind vehicle 1703, exhibits a defensive tendency (also based on the observed position, speed, and / or relative acceleration of vehicle 1710), and thus successfully merges into the lane in front of target vehicle 1710 and behind target vehicle 1703.

[0385] Figure 18 A flowchart is provided representing an exemplary algorithm for navigating a master vehicle based on predicted aggression from other vehicles. Figure 18 In the example, the level of aggression associated with at least one target vehicle can be inferred based on the behavior of the target vehicle as observed by objects in the environment relative to the target vehicle. For example, in step 1801, at least one processing device of the main vehicle navigation system (e.g., processing device 110) can receive multiple images representing the environment of the main vehicle from a camera associated with the main vehicle. In step 1803, analysis of one or more of the received images enables at least one processor to identify a target vehicle (e.g., vehicle 1703) in the environment of the main vehicle 1701. In step 1805, analysis of one or more of the received images enables at least one processing device to identify at least one obstacle to the target vehicle in the environment of the main vehicle. The object may include debris in the road, parking lights / traffic lights, pedestrians, another vehicle (e.g., a vehicle traveling in front of the target vehicle, a parked vehicle, etc.), boxes in the road, roadblocks, curbs, or any other type of object that may be encountered in the environment of the main vehicle. In step 1807, analysis of one or more of the received images enables at least one processing device to determine at least one navigation characteristic of the target vehicle relative to at least one identified obstacle to the target vehicle.

[0386] Various navigation characteristics can be used to infer the aggression level of a detected target vehicle in order to generate an appropriate navigation response to the target vehicle. For example, such navigation characteristics may include the relative acceleration between the target vehicle and at least one identified obstacle, the distance of the target vehicle from the obstacle (e.g., the following distance of the target vehicle behind another vehicle), and / or the relative speed between the target vehicle and the obstacle.

[0387] In some embodiments, the navigation characteristics of a target vehicle can be determined based on the outputs from sensors associated with the host vehicle (e.g., radar, speed sensors, GPS, etc.). However, in some cases, the navigation characteristics of a target vehicle can be determined partly or entirely based on the analysis of images of the host vehicle's environment. For example, image analysis techniques described above and in, for example, U.S. Patent No. 9,168,868, incorporated herein by reference, can be used to identify a target vehicle within the host vehicle's environment. Furthermore, monitoring the position of the target vehicle in the captured images over time and / or monitoring the position of one or more features associated with the target vehicle (e.g., taillights, headlights, bumpers, wheels, etc.) in the captured images can make it possible to determine the relative distance, velocity, and / or acceleration between the target vehicle and the host vehicle or between the target vehicle and one or more other objects in the host vehicle's environment.

[0388] The level of aggression of an identified target vehicle can be inferred from any suitable observed navigation characteristics of the target vehicle, or any combination of observed navigation characteristics. For example, a determination of aggression can be made based on any observed characteristic and one or more predetermined threshold levels or any other suitable qualitative or quantitative analysis. In some embodiments, a target vehicle may be considered aggressive if it is observed following a host vehicle or another vehicle at a distance less than a predetermined aggressive distance threshold. Conversely, a target vehicle observed following a host vehicle or another vehicle at a distance greater than a predetermined defensive distance threshold may be considered defensive. The predetermined aggressive distance threshold does not need to be the same as the predetermined defensive distance threshold. Furthermore, either or both of the predetermined aggressive distance threshold and the predetermined defensive distance threshold may include a range of values, rather than a bright line value. Moreover, neither the predetermined aggressive distance threshold nor the predetermined defensive distance threshold needs to be fixed. Instead, these values ​​or ranges of values ​​can change over time, and different thresholds / ranges of thresholds can be applied based on the observed characteristics of the target vehicle. For example, the applied threshold may depend on one or more other characteristics of the target vehicle. Higher observed relative speeds and / or accelerations may authorize the application of larger thresholds / ranges. Conversely, lower relative velocity and / or acceleration (including zero relative velocity and / or acceleration) can authorize the application of smaller distance thresholds / ranges when making aggressive / defensive inferences.

[0389] Aggression / defensivity inferences can also be based on relative speed and / or relative acceleration thresholds. A target vehicle may be considered aggressive if its relative speed and / or relative acceleration relative to another vehicle exceeds a predetermined level or range. Conversely, a target vehicle may be considered defensive if its relative speed and / or relative acceleration relative to another vehicle is below a predetermined level or range.

[0390] While aggression / defensiveness can be determined solely based on any observed navigational characteristics, the determination can also depend on any combination of observed characteristics. For example, as mentioned above, in some cases, a target vehicle may be considered aggressive simply because it is observed following another vehicle at a distance below a certain threshold or range. However, in other cases, a target vehicle may be considered aggressive if it is following another vehicle at a relative speed and / or relative acceleration greater than the predetermined amount or range (which may be the same as or different from the threshold applied when determining based solely on distance). Similarly, a target vehicle may be considered defensive simply because it is observed following another vehicle at a distance greater than a certain threshold or range. However, in other cases, a target vehicle may be considered defensive if it is following another vehicle at a relative speed and / or relative acceleration less than the predetermined amount or range (which may be the same as or different from the threshold applied when determining based solely on distance). If, for example, a vehicle accelerates or decelerates at a speed exceeding 0.5G (e.g., a rate of change of acceleration (jerk) of 5 m / s³), a vehicle has a lateral acceleration of 0.5G while changing lanes or on a curve, a vehicle causes another vehicle to do any of the above, a vehicle changes lanes and causes another vehicle to give way with a deceleration of 0.3G or a rate of change of acceleration of 3 m / s³ or more, and / or a vehicle changes two lanes without stopping, then system 100 can be considered aggressive / defensive.

[0391] It should be understood that a reference to a quantity exceeding a range can indicate that the quantity exceeds or falls within that range. Similarly, a reference to a quantity falling below a range can indicate that the quantity falls below or falls within that range. Furthermore, while examples for making aggressive / defensive inferences are described relative to distance, relative acceleration, and relative velocity, any other suitable quantity can be used. For example, calculations of the time of collision, or any indirect indicator of the target vehicle's distance, acceleration, and / or velocity, can be used. It should also be noted that while the examples above focus on a target vehicle relative to other vehicles, aggressive / defensive inferences can be made by observing the target vehicle's navigational characteristics relative to any other type of obstacle (e.g., pedestrians, roadblocks, traffic lights, debris, etc.).

[0392] Back Figure 17A and 17B In the example shown, when the primary vehicle 1701 approaches a roundabout, the navigation system (including at least one of its processing devices) can receive an image stream from a camera associated with the primary vehicle. Based on the analysis of one or more received images, any one of the target vehicles 1703, 1705, 1706, 1708, and 1710 can be identified. Furthermore, the navigation system can analyze the navigation characteristics of one or more identified target vehicles. The navigation system can recognize that the gap between target vehicles 1703 and 1705 represents a potential first opportunity to merge into the roundabout. The navigation system can analyze target vehicle 1703 to determine an aggression indicator associated with target vehicle 1703. If target vehicle 1703 is deemed aggressive, the primary vehicle navigation system may choose to yield to vehicle 1703 instead of merging in front of vehicle 1703. On the other hand, if target vehicle 1703 is deemed defensive, the primary vehicle navigation system may attempt to complete the lane merge in front of vehicle 1703.

[0393] When the primary vehicle 1701 approaches the roundabout, at least one processing device of the navigation system can analyze the captured images to determine navigation characteristics associated with the target vehicle 1703. For example, based on the images, it can determine that vehicle 1703 is following vehicle 1705 at a distance that provides sufficient clearance for the primary vehicle 1701 to enter safely. In fact, it can be determined that vehicle 1703 is following vehicle 1705 at a distance exceeding an aggressive distance threshold, and therefore, based on this information, the primary vehicle navigation system may tend to identify the target vehicle 1703 as defensive. However, in some cases, as described above, more than one navigation characteristic of the target vehicle can be analyzed when making an aggressive / defensive determination. Further analysis may allow the primary vehicle navigation system to determine that when the target vehicle 1703 is following behind the target vehicle 1705 at a non-aggressive distance, vehicle 1703 has a relative velocity and / or relative acceleration relative to vehicle 1705 that exceeds one or more thresholds associated with aggressive behavior. In fact, the primary vehicle 1701 can determine that the target vehicle 1703 is accelerating relative to vehicle 1705 and approaching the gap existing between vehicles 1703 and 1705. Based on further analysis of relative speed, acceleration, and distance (and even the rate at which the gap between vehicles 1703 and 1705 is closing), the primary vehicle 1701 can determine that the target vehicle 1703 is behaving aggressively. Therefore, although there may be a sufficient gap that the primary vehicle can safely navigate into, the primary vehicle 1701 can anticipate that a lane change in front of the target vehicle 1703 will result in an aggressively navigating vehicle immediately behind the primary vehicle. Furthermore, based on behavior observed through image analysis or other sensor outputs, the target vehicle 1703 can be expected to either continue accelerating toward the primary vehicle 1701 or continue moving toward the primary vehicle 1701 at a non-zero relative speed if the primary vehicle 1701 were to change lanes in front of vehicle 1703. From a safety perspective, this situation may be undesirable and could also cause discomfort to the passengers of the primary vehicle. For this reason, as Figure 17B As shown, the main vehicle 1701 may choose to give way to vehicle 1703 and merge into the roundabout behind vehicle 1703 and in front of vehicle 1710, which is considered defensive based on the analysis of one or more of its navigation characteristics.

[0394] Back Figure 18In step 1809, at least one processing device of the main vehicle's navigation system can determine a navigation action of the main vehicle (e.g., merging ahead of vehicle 1710 and behind vehicle 1703) based on at least one navigation characteristic of the identified target vehicle relative to the identified obstacle. To implement this navigation action (in step 1811), at least one processing device can cause at least one adjustment to the main vehicle's navigation actuator in response to the determined navigation action. For example, braking can be applied to give way to... Figure 17A Vehicle 1703, and the accelerator can be applied in conjunction with the steering of the master vehicle's wheels to allow the master vehicle to enter the roundabout behind vehicle 1703, such as Figure 17B As shown.

[0395] As illustrated in the examples above, the navigation of the primary vehicle can be based on the navigation characteristics of the target vehicle relative to another vehicle or object. Alternatively, the navigation of the primary vehicle can be based solely on the navigation characteristics of the target vehicle without specifically referencing another vehicle or object. For example, in... Figure 18 At step 1807, analysis of multiple images captured from the environment of the primary vehicle enables the determination of at least one navigation characteristic of the identified target vehicle, indicating the level of aggression associated with the target vehicle. Navigation characteristics may include speed, acceleration, etc., which do not require reference to another object or target vehicle for determining aggression / defensiveness. For example, observed acceleration and / or speed associated with the target vehicle exceeding a predetermined threshold or falling within or exceeding a numerical range may indicate aggressive behavior. Conversely, observed acceleration and / or speed associated with the target vehicle falling below a predetermined threshold or falling within or exceeding a numerical range may indicate defensive behavior.

[0396] Of course, in some cases, to determine aggression / defense, observed navigation characteristics (e.g., position, distance, acceleration, etc.) can be referenced relative to the host vehicle. For example, navigation characteristics that indicate the level of aggression associated with the target vehicle may include an increase in relative acceleration between the target vehicle and the host vehicle, the following distance of the target vehicle behind the host vehicle, and the relative speed between the target vehicle and the host vehicle.

[0397] Navigation based on liability constraints

[0398] As described in the preceding sections, planned navigation actions can be tested against predetermined constraints to ensure compliance with certain rules. In some embodiments, this concept can be extended to consideration of potential accident liability. As explained below, the primary objective of autonomous navigation is safety. Since absolute safety may be impossible (e.g., at least because a particular master vehicle under autonomous control cannot control other vehicles around it—it can only control its own actions), using potential accident liability as a consideration in autonomous navigation, and effectively as a constraint on planned actions, can help ensure that a particular autonomous vehicle does not take any actions deemed unsafe—e.g., those actions for which potential accident liability might be attached to the master vehicle. If the master vehicle only takes safe actions that are determined not to cause an accident due to the master vehicle's own fault or liability, then a desired level of accident avoidance (e.g., less than 10 per hour) can be achieved. -9 ).

[0399] Most current autonomous driving approaches face challenges including a lack of safety assurance (or at least an inability to provide the desired level of safety) and a lack of scalability. Consider the problem of ensuring safe driving in multi-agent situations. Since society is unlikely to tolerate deaths in road accidents caused by machines, an acceptable level of safety is crucial for the acceptance of autonomous vehicles. While the goal might be to achieve zero accidents, this may be impossible because accidents typically involve multiple actors, and situations can be conceived where the accident is entirely due to the fault of other actors. For example, as... Figure 19 As shown, the master vehicle 1901 is driving on a multi-lane highway. While the master vehicle 1901 can control its own actions relative to target vehicles 1903, 1905, 1907, and 1909, it cannot control the actions of the target vehicles around it. As a result, if vehicle 1905 suddenly cuts into the master vehicle's lane, for example, along the collision path, the master vehicle 1901 may be unable to avoid an accident with at least one target vehicle. To address this challenge, a typical response from autonomous vehicle practitioners is to employ a statistically driven approach, in which safety verification becomes more rigorous as more mileage data is collected.

[0400] However, to understand the nature of the problem with data-driven approaches to safety, the first thing to consider is that the probability of death from an accident involving human driving per hour is known to be 10. -6 A reasonable assumption is that, in order for society to accept machines replacing humans in driving tasks, the mortality rate should be reduced by three orders of magnitude, i.e., to 10 per hour. -9 The probability. This estimate is similar to the assumed airbag mortality rate and comes from aviation standards. For example, 10 -9It's the probability of a wing spontaneously detaching from the aircraft in mid-air. However, attempting to guarantee safety using data-driven statistical methods that provide additional confidence through accumulated flight mileage is impractical. Guaranteeing 10 [units of something] per hour... -9 The amount of data required to determine the probability of death is its reciprocal (i.e., 10). 9 This is proportional to the amount of data collected over hours, which is on the order of approximately 30 billion miles. Furthermore, multi-actuator systems interact with their environment and may not be verifiable offline (unless a realistic simulator is available that simulates real human driving (such as reckless driving) with its full richness and complexity—but validating such a simulator is even more difficult than creating a safe autonomous vehicle actor). Any changes to the planning and control software would require the same amount of new data collection, which is clearly impractical and unrealistic. Moreover, developing systems through data always suffers from a lack of interpretability and explainability of the actions taken—if an autonomous vehicle (AV) accident results in death, we need to know the cause. Therefore, a model-based approach to safety is needed, but existing “functional safety” and ASIL requirements in the automotive industry are not designed for multi-actuator environments.

[0401] The second major challenge in developing safe driving models for autonomous vehicles is the need for scalability. The premise of AVs is not merely "building a better world," but rather the assumption that driverless mobility can be sustained at a lower cost than drivered mobility. This premise is always intertwined with the concept of scalability—that is, supporting the mass production of AVs (in the millions), and more importantly, supporting negligible incremental costs to enable them to drive in new cities. Therefore, the costs of computing and sensing are indeed significant, and the costs of validation and the ability to drive "everywhere," rather than in a select few cities, are also essential requirements for sustaining the business if AVs are to be manufactured at scale.

[0402] The problem with most current approaches lies in a brute-force mentality along three axes: (i) the required computational density, (ii) how high-definition maps are defined and created, and (iii) the required sensor specifications. This brute-force approach runs counter to scalability and shifts the focus to a future where unrestricted in-vehicle computing is ubiquitous, where the cost of building and maintaining HD maps becomes negligible and scalable, and exotic, ultra-advanced sensors are developed, manufactured to automotive-grade levels at negligible cost. While a future where any one of these scenarios is indeed possible, making all of them a reality is likely a low-probability event. Therefore, it is necessary to provide a formal model that combines safety and scalability into a socially acceptable and scalable AV program that supports millions of vehicles driving anywhere in developed countries.

[0403] The disclosed embodiments represent a solution that can provide a target level of safety (or even exceed that target) and can be scaled up to systems comprising millions (or more) of autonomous vehicles. In terms of safety, a model known as “Responsibility Sensitive Safety (RSS)” is introduced, which formalizes the concept of “accident attribution” in a way that is interpretable and explainable, and incorporates the concept of “responsibility” into the actions of the robotic actor. The definition of RSS is agnostic about how it is implemented—a key feature that facilitates the creation of a compelling global safety model. The motivation for RSS is that observations (such as…) Figure 19 (As shown) the actors play asymmetrical roles in an accident, where typically only one actor is responsible for and therefore liable for the accident. RSS models also include formal treatment of "cautious driving" under limited sensing conditions, where not all actors are always visible (e.g., due to occlusion). A primary goal of RSS models is to guarantee that an actor will never be "attributed" to or responsible for an accident. A model is only useful if it incorporates effective strategies consistent with RSS (e.g., a function that maps "sensing state" to actions). For example, an action that seems innocent at the present moment could lead to a catastrophic event in the distant future ("butterfly effect"). RSS can help construct a set of local constraints in the near future that guarantee (or at least practically guarantee) that future accidents will not occur due to the actions of the primary vehicle.

[0404] Another contribution revolves around the introduction of a “semantic” language, which includes units, measurements, and action spaces, and how to incorporate them into the specifications for AV planning, sensing, and actuation. To gain a semantic understanding, in this context, consider how someone taking a driving course is instructed to think about “driving strategies.” These instructions are not geometric—they don’t take things like, “drive 13.7 meters at the current speed, then at 0.8 m / s…” 2 Instead of the form of "acceleration at a certain rate," these instructions are semantic in nature—"follow the car in front of you" or "overtake that car on your left." The typical language of human driving strategies is about longitudinal and lateral targets, not geometric units of acceleration vectors. Formal semantic language can be useful in several ways, including the computational complexity of planning that does not increase exponentially with time and the number of actors, the way safe and comfortable interactions are handled, how computations of sensing are defined, the specifications of sensor modalities, and how they interact in fusion methods. (Semantic language-based) fusion methods can ensure that RSS models only work on orders of magnitude 10. 5 By performing offline validation on a dataset of hours of driving data, the required 10 hours of driving data can be achieved. -9 The probability of death.

[0405] For example, in a reinforcement learning setting, a Q-function can be defined in the semantic space (e.g., a function that evaluates the long-term quality of an action a∈A when the actor is in state s∈S; given such a Q-function, the natural selection of actions could be to choose the action with the highest quality π(s) = argmax). a Q(s,a)), in this semantic space, the number of trajectories to be examined at any given time is in the order of 10. 4The space is defined regardless of the time frame used for planning. The signal-to-noise ratio in this space can be high, allowing effective machine learning methods to successfully model the Q-function. In the case of computation on sensing, semantics can allow for the differentiation between errors affecting safety and errors affecting driving comfort. We define a PAC model (Probably Approximate Correct, borrowing Valiant's PAC learning terminology) for sensing, fit this model to the Q-function, and demonstrate how measurement errors can be incorporated into planning in a way that conforms to RSS while allowing for optimization of driving comfort. Semantic language may be important for the success of some aspects of this model, as other standard measurements of errors (such as errors relative to the global coordinate system) may not be compatible with the PAC sensing model. Furthermore, semantic language may be a significant contributing factor to the definition of HD maps, which can be constructed using low-bandwidth sensing data, thus enabling crowdsourcing and supporting scalability.

[0406] In summary, the disclosed embodiments may include a formal model covering the following key components of AV: sensing, planning, and action. This model can help ensure that, from a planning perspective, incidents of AV's own fault do not occur. Furthermore, with the PAC sensing model, even with sensing errors, the described fusion method may require only a very reasonable amount of offline data collection to conform to the described safety model. Moreover, this model can combine security and scalability through semantic language, thus providing a complete approach to secure and scalable AV. Finally, it is worth noting that developing an accepted safety model that will be adopted by industry and regulatory bodies may be a prerequisite for the success of AV.

[0407] RSS models typically follow the classic sense-plan-action robot control approach. The sensing system is responsible for understanding the current state of the host vehicle's environment. The planning component, often referred to as the "driving strategy," can be implemented by a set of hard-coded instructions, through a trained system (e.g., a neural network), or a combination thereof. This planning component is responsible for determining the optimal next movement based on available options for achieving a driving objective (e.g., how to move from the left lane to the right lane to exit the highway). The action component is responsible for implementing the plan (e.g., a system of actuators and one or more controllers for steering, accelerating, and / or braking the vehicle to perform the selected navigation actions). The embodiments described below focus primarily on the sensing and planning components.

[0408] Accidents can stem from sensing or planning errors. Planning is a multi-agent effort, as other road users (people and machines) react to the actions of the AV. The described RSS model is designed to address issues such as safety in the planning component. This can be termed multi-agent safety. In statistical methods, the probability of planning errors can be estimated “online.” That is, after each software update, billions of miles must be driven with the new version to provide an estimate of an acceptable level of frequency of planning errors. This is clearly impractical. As an alternative, the RSS model can provide a 100% (or nearly 100%) guarantee that the planning module will not commit errors attributable to the AV (formally defining the concept of “attribution”). The RSS model also provides an efficient method for its validation that does not rely on online testing.

[0409] Errors in sensing systems are often easier to verify because sensing can be independent of vehicle movement, and therefore we can use "offline" data to verify the probability of severe sensing errors. However, even collecting more than 10 9 Offline data from hours of driving is also challenging. As part of the description of the disclosed sensing system, a fusion method is described that can be validated using significantly less data.

[0410] The described RSS system can also be scaled to millions of vehicles. For example, the described semantic driving strategy and the applied safety constraints can be aligned with sensing and mapping requirements that can be scaled to millions of vehicles even with today's technology.

[0411] The fundamental building block of such a system is a thorough safety definition, i.e., the minimum standards that the AV system may need to comply with. In the following technical lemma, statistical methods used to verify AV systems are proven infeasible, even for verifying simple statements such as "the system experiences N incidents per hour." This means that model-based safety definitions are the only feasible tool for verifying AV systems.

[0412] Lemma 1: Let X be a probability space, and A be the event Pr(A) = p1 < 0.1. Suppose we sample from X... (iid) samples, and let So

[0413] Pr(Z=0)≥e -2 .

[0414] We will use the inequality 1 - x ≥ e to prove that... -2x (Completeness is demonstrated in Appendix A.1), thus obtaining

[0415]

[0416] Corollary 1: Assume that AV system AV1 experiences an accident with a small but insufficient probability p1. Any deterministic verification procedure given 1 / p1 samples will, with a constant probability, be unable to distinguish AV1 from the different AV system AV0 that never experiences an accident.

[0417] To gain insight into typical values ​​for these probabilities, let's assume we want the probability of an accident to be 10 per hour. -9 However, a certain AV system only provides 10 -8 The probability. Even if the system obtains 10 8 Even after hours of driving, there is still a constant probability that the verification process will fail to indicate that the system is dangerous.

[0418] Finally, note that the difficulty lies in invalidating a single, specific, and potentially dangerous AV system. A complete solution cannot be viewed as a single system because new versions, bug fixes, and updates are required. From the validator's perspective, every change, even a single line of code, generates a new system. Therefore, a statistically validated solution must be validated online with a new sample after each small correction or change to account for shifts in the state distribution observed and derived from the new system. Repeatedly and systematically obtaining such a large sample size (and even then, with a constant probability of failing to validate the system) is impractical.

[0419] Furthermore, any statistical claim must be formalized for measurement. A claim about the statistical number of accidents occurring in a system is far weaker than a claim that "it is driven safely." To illustrate this, what constitutes safety must be formally defined.

[0420] Absolute safety is impossible

[0421] If no accident-following action a occurs at some future time, then the action a taken by car c can be considered absolutely safe. This can be demonstrated by observing, for example,... Figure 19The simple driving scenario depicted demonstrates that absolute safety is impossible. From vehicle 1901's perspective, no action can guarantee that surrounding vehicles won't collide with it. Solving this problem by prohibiting autonomous vehicles from being in this situation...

Claims

1. A system for a host vehicle, the system comprising: At least one processor programmed to: receiving at least one image representative of an environment of a host vehicle from an image capture device; detecting at least one obstacle in an environment of the host vehicle based on an analysis of the at least one image; determining a current speed of the host vehicle and a predicted path of the host vehicle; monitoring a driver input to at least one of a throttle control, a brake control, or a steering control associated with the host vehicle; Determining a proximity buffer zone relative to the at least one obstacle, the at least one obstacle comprising the target vehicle, the proximity buffer zone being determined based on a sum of at least: a host vehicle acceleration distance, the host vehicle acceleration distance being determined as the distance the host vehicle would travel if accelerating from the current speed at a maximum acceleration capability of the host vehicle within a reaction time associated with the host vehicle, and a host vehicle stopping distance, the host vehicle stopping distance being determined as a distance required to reduce a maximum speed of the host vehicle to zero at a maximum braking rate capability of the host vehicle, the maximum speed corresponding to a speed of the host vehicle when accelerating at the maximum acceleration capability of the host vehicle within the reaction time associated with the host vehicle; a target vehicle stopping distance, the target vehicle stopping distance being determined as a distance required to reduce a current speed of the target vehicle to zero at an assumed maximum braking rate capability of the target vehicle; as well as determining, based on the predicted path, whether the driver input will cause the host vehicle to navigate within the proximity buffer zone relative to at least one obstacle, Wherein, the at least one processor is further programmed to: if at least one processing device determines that the driver input will not result in the host vehicle navigating within a proximity buffer zone relative to at least one obstacle, allowing the driver input to cause a corresponding change in one or more host vehicle motion control systems; as well as If at least one processing device determines that the driver input would cause the host vehicle to navigate within a proximity buffer zone relative to at least one obstacle, preventing the driver input from causing a or corresponding changes in multiple host vehicle motion control systems.

2. The system according to claim 1, wherein: The approach buffer zone is further determined based on a maximum distance traveled during a reaction time and a distance traveled during braking.

3. The system according to claim 2, wherein: The maximum distance traveled during the reaction time comprises the sum of the host vehicle's speed multiplied by the reaction time, and the reaction time squared multiplied by half of the host vehicle's maximum acceleration capability.

4. The system according to claim 1, wherein: The at least one processor is further programmed to alert the driver if the at least one processing device determines that the driver input will cause the host vehicle to navigate within a proximity buffer zone relative to at least one obstacle.

5. The system according to claim 1, wherein: The approach buffer zone is further determined based on a predetermined sub-maximum braking rate that is less than a maximum braking rate capability of the host vehicle.

6. The system according to claim 1, wherein: The at least one obstacle includes a pedestrian or an object on the roadway, and the approach buffer zone relative to the at least one obstacle includes at least a minimum distance to be maintained between the host vehicle and the at least one obstacle.

7. The system according to claim 1, wherein: Determining whether the driver input will cause the host vehicle to navigate within a proximity buffer zone relative to at least one obstacle includes determining an acceleration vector of the host vehicle due to the driver input.

8. The system according to claim 1, wherein: At least one obstacle is one of a pedestrian or a target vehicle.

9. The system according to claim 8, wherein: The at least one processor is further programmed to: determining an obstacle speed and an obstacle predicted path of one of the pedestrian or target vehicle; and The approach buffer zone is determined using the determined obstacle speed and the obstacle prediction path.

10. The system according to claim 9, wherein: The approach buffer zone is determined based on a maximum distance traveled during a reaction time and a distance traveled during braking, and wherein the distance traveled during braking comprises a difference between: The square of the sum of the host vehicle's speed and reaction time multiplied by its maximum acceleration capability, and the square of the obstacle's speed, and The distance traveled during braking also includes dividing the difference by twice the maximum braking capacity value.

11. The system according to claim 9, wherein: The approach buffer zone is further determined by an assumed maximum braking rate capability of the target vehicle.

12. The system according to claim 9, wherein: The target vehicle is determined to be in front of the host vehicle, and wherein the at least one processor is further programmed to determine that the driver input will result in a change in a longitudinal distance between the target vehicle and the host vehicle.

13. The system according to claim 9, wherein: The target vehicle is determined to be in a different lane than the host vehicle, and wherein the at least one processor is further programmed to determine that the driver input will result in lateral movement of the host vehicle such that after the lateral movement, the target vehicle will be in front of the host vehicle.

14. The system according to claim 9, wherein: The target vehicle is determined to be in a different lane than the host vehicle, and wherein the at least one processor is further programmed to determine that the driver input will result in lateral movement of the host vehicle such that after the lateral movement, the host vehicle will be in front of the target vehicle.

15. A method for navigating a host vehicle, the method comprising: receiving at least one image representative of an environment of a host vehicle from an image capture device; detecting at least one obstacle in an environment of the host vehicle based on an analysis of the at least one image; determining a current speed of the host vehicle and a predicted path of the host vehicle; monitoring a driver input to at least one of a throttle control, a brake control, or a steering control associated with the host vehicle; Determining a proximity buffer zone relative to the at least one obstacle, the at least one obstacle comprising the target vehicle, the proximity buffer zone being determined based on a sum of at least: a host vehicle acceleration distance, the host vehicle acceleration distance being determined as the distance the host vehicle would travel if accelerating from the current speed at a maximum acceleration capability of the host vehicle within a reaction time associated with the host vehicle, and a host vehicle stopping distance, the host vehicle stopping distance being determined as a distance required to reduce a maximum speed of the host vehicle to zero at a maximum braking rate capability of the host vehicle, the maximum speed corresponding to a speed of the host vehicle when accelerating at the maximum acceleration capability of the host vehicle within the reaction time associated with the host vehicle; a target vehicle stopping distance, the target vehicle stopping distance being determined as a distance required to reduce a current speed of the target vehicle to zero at an assumed maximum braking rate capability of the target vehicle; as well as determining, based on the predicted path, whether the driver input will cause the host vehicle to navigate within the proximity buffer zone relative to at least one obstacle, The method further comprises: if it is determined that the driver input will not result in the host vehicle navigating within a proximity buffer zone relative to at least one obstacle, allowing the driver input to cause a corresponding change in one or more host vehicle motion control systems; as well as If it is determined that the driver input would cause the host vehicle to navigate within a proximity buffer zone relative to at least one obstacle, the driver input is prevented from causing a corresponding change in one or more host vehicle motion control systems.

16. The method according to claim 15, wherein: The approach buffer zone is determined based on a maximum distance traveled during the reaction time and a distance traveled during braking.

17. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising: receiving at least one image representative of an environment of a host vehicle from an image capture device; detecting at least one obstacle in an environment of the host vehicle based on an analysis of the at least one image; determining a current speed of the host vehicle and a predicted path of the host vehicle; monitoring a driver input to at least one of a throttle control, a brake control, or a steering control associated with the host vehicle; and determining a proximity buffer zone relative to the at least one obstacle, the at least one obstacle comprising the target vehicle, the proximity buffer zone being determined based on a sum of at least: a host vehicle acceleration distance, the host vehicle acceleration distance being determined as the distance the host vehicle would travel if accelerating from the current speed at a maximum acceleration capability of the host vehicle within a reaction time associated with the host vehicle, and a host vehicle stopping distance, the host vehicle stopping distance being determined as a distance required to reduce a maximum speed of the host vehicle to zero at a maximum braking rate capability of the host vehicle, the maximum speed corresponding to a speed of the host vehicle when accelerating at the maximum acceleration capability of the host vehicle within the reaction time associated with the host vehicle; a target vehicle stopping distance, the target vehicle stopping distance being determined as a distance required to reduce a current speed of the target vehicle to zero at an assumed maximum braking rate capability of the target vehicle; as well as determining, based on the predicted path, whether the driver input will cause the host vehicle to navigate within the proximity buffer zone relative to at least one obstacle, The operation further includes: If the processor determines that the driver input will not result in the host vehicle navigating within a proximity buffer zone relative to at least one obstacle, allowing the driver input to cause a corresponding change in one or more host vehicle motion control systems; and If the processor determines that the driver input would cause the host vehicle to navigate within a proximity buffer zone relative to at least one obstacle, the driver input is prevented from causing a corresponding change in one or more host vehicle motion control systems.

Images