System and method for navigation with secure distance

Abstract

Systems and methods for navigating a host vehicle are disclosed. In one embodiment, at least one processing device may be programmed to receive an image representative of an environment of a host vehicle, determine a planned navigation action of the host vehicle, analyze the image to identify a target vehicle having a direction of travel toward the host vehicle, and transmit the target vehicle to the host vehicle. And determining a next state distance between the host vehicle and the target vehicle that would result from the planned navigation action being taken. The at least one processing device may further determine a stop distance of the host vehicle based on a braking rate, a maximum acceleration capability, and a current speed of the host vehicle, determine a stop distance of the target vehicle based on a braking rate, a maximum acceleration capability, and a current speed of the target vehicle, and performing a planned navigation action if the determined next state distance is greater than the sum of the stopping distances of the host vehicle and the target vehicle.

Description

[0001] This application is a divisional application, the parent application of which is entitled "System and Method for Navigation with Safe Distance", filed on August 11, 2019, with application number 201980043159.2.

[0002] Cross-references to related applications

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 718,554, filed August 14, 2018; U.S. Provisional Patent Application No. 62 / 724,355, filed August 29, 2018; U.S. Provisional Patent Application No. 62 / 772,366, filed November 28, 2018; and U.S. Provisional Patent Application No. 62 / 777,914, filed December 11, 2018. All of the above applications are incorporated herein by reference in their entirety. Technical Field

[0004] This disclosure generally relates to autonomous vehicle navigation. Furthermore, this disclosure relates to systems and methods for navigation based on potential accident liability constraints. Background Technology

[0005] With continuous technological advancements, the goal of fully autonomous vehicles capable of navigating on roads is imminent. Autonomous vehicles may need to consider a wide variety of factors and make appropriate decisions based on those factors to safely and accurately reach their desired destination. For example, an autonomous vehicle may need to process and interpret visual information (e.g., information captured from cameras), information from radar or lidar, and may also use information from other sources (e.g., from GPS devices, speed sensors, accelerometers, suspension sensors, etc.). Simultaneously, to navigate to its destination, an autonomous vehicle may also need to identify its position within a specific roadway (e.g., a specific lane in a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, observe traffic signals and signs, move from one road to another at appropriate intersections or junctions, and respond to any other situations that occur or progress during vehicle operation. Furthermore, navigation systems may need to comply with certain imposed constraints. In some cases, these constraints may relate to the interaction between the primary vehicle and one or more other objects (such as other vehicles, pedestrians, etc.). In other cases, constraints may relate to rules of responsibility to be followed when performing one or more navigation actions for the primary vehicle.

[0006] In the field of autonomous driving, there are two important considerations for feasible autonomous vehicle systems. The first is the standardization of safety assurance, including the requirements that every autonomous vehicle must meet to ensure safety, and how to verify those requirements. The second is scalability, because engineering solutions that lead to cost-effectiveness cannot scale to millions of vehicles and may hinder the widespread adoption of autonomous vehicles, or even their less widespread adoption. Therefore, an interpretable mathematical model for safety assurance and a system design that complies with safety assurance requirements while scalable to millions of vehicles are needed. Summary of the Invention

[0007] Embodiments consistent with this disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation features. For example, consistent with the disclosed embodiments, the disclosed system may include one, two, or more cameras monitoring the vehicle's environment. The disclosed system may provide a navigation response based, for example, analysis of images captured by one or more cameras. The navigation response may also take into account other data, including, for example, global positioning system (GPS) data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data.

[0008] In one embodiment, a system for navigating a master vehicle is disclosed. The system may include at least one processing device programmed to receive at least one image representing the environment of the master vehicle from an image capture device; determine a planned navigation action for achieving a navigation objective of the master vehicle based on at least one driving strategy; analyze the at least one image to identify a target vehicle in the master vehicle's environment, wherein the target vehicle's direction of travel is toward the master vehicle; determine the next-state distance between the master vehicle and the target vehicle that would result if the planned navigation action were taken; determine the master vehicle's braking rate, maximum acceleration capability, and current speed; determine the master vehicle's stopping distance based on the master vehicle's braking rate, maximum acceleration capability, and current speed; determine the target vehicle's current speed, maximum acceleration capability, and braking rate; determine the target vehicle's stopping distance based on the target vehicle's braking rate, maximum acceleration capability, and current speed; and perform the planned navigation action if the determined next-state distance is greater than the sum of the master vehicle's stopping distance and the target vehicle's stopping distance.

[0009] In one embodiment, a method for navigating a master vehicle is disclosed. The method may include: receiving from an image capture device at least one image representing the environment of the master vehicle; determining a planned navigation action for achieving a navigation objective of the master vehicle based on at least one driving strategy; analyzing the at least one image to identify a target vehicle in the environment of the master vehicle, wherein the target vehicle's direction of travel is toward the master vehicle; determining a next-state distance between the master vehicle and the target vehicle that would result if the planned navigation action were taken; determining the master vehicle's braking rate, maximum acceleration capability, and current speed; determining a stopping distance of the master vehicle based on the master vehicle's braking rate, maximum acceleration capability, and current speed; determining the target vehicle's current speed, maximum acceleration capability, and braking rate; determining a stopping distance of the target vehicle based on the target vehicle's braking rate, maximum acceleration capability, and current speed; and performing the planned navigation action if the determined next-state distance is greater than the sum of the master vehicle's stopping distance and the target vehicle's stopping distance.

[0010] In one embodiment, a system for navigating a master vehicle is disclosed. The system may include at least one processing device programmed to receive at least one image representing the environment of the master vehicle from an image capture device; determine a planned navigation action for achieving a navigation objective of the master vehicle based on at least one driving strategy; analyze the at least one image to identify a target vehicle in the master vehicle's environment; determine the next-state lateral distance between the master vehicle and the target vehicle if the planned navigation action is taken; determine the master vehicle's maximum yaw rate capability, maximum turning radius capability, and current lateral speed; determine the master vehicle's lateral braking distance based on the master vehicle's maximum yaw rate capability, maximum turning radius capability, and current lateral speed; determine the target vehicle's current lateral speed, maximum yaw rate capability, and maximum turning radius capability; determine the target vehicle's lateral braking distance based on the target vehicle's current lateral speed, maximum yaw rate capability, and maximum turning radius capability; and perform the planned navigation action if the determined next-state lateral distance is greater than the sum of the master vehicle's lateral braking distance and the target vehicle's lateral braking distance.

[0011] In one embodiment, a method for navigating a master vehicle is disclosed. The method may include: receiving at least one image representing the environment of the master vehicle from an image capture device; determining a planned navigation action for achieving a navigation objective of the master vehicle based on at least one driving strategy; analyzing the at least one image to identify a target vehicle in the master vehicle's environment; determining a next-state lateral distance between the master vehicle and the target vehicle that would result if the planned navigation action were taken; determining the master vehicle's maximum yaw rate capability, maximum turning radius capability, and current lateral speed; determining a lateral braking distance of the master vehicle based on its maximum yaw rate capability, maximum turning radius capability, and current lateral speed; determining the target vehicle's current lateral speed, maximum yaw rate capability, and maximum turning radius capability; determining a lateral braking distance of the target vehicle based on its current lateral speed, maximum yaw rate capability, and maximum turning radius capability; and performing the planned navigation action if the determined next-state lateral distance is greater than the sum of the master vehicle's lateral braking distance and the target vehicle's lateral braking distance.

[0012] In one embodiment, a system for navigating a primary vehicle near a pedestrian crossing is disclosed. The system may include at least one processing device programmed to: receive from an image capture device at least one image representing the environment of the primary vehicle; detect a representation of a pedestrian crossing in the at least one image based on analysis of the at least one image; determine, based on analysis of the at least one image, whether a representation of a pedestrian is present in the at least one image; detect the presence of a traffic light in the environment of the primary vehicle; determine whether the detected traffic light is related to the primary vehicle and the pedestrian crossing; determine the state of the detected traffic light; determine the proximity of a pedestrian relative to the detected pedestrian crossing when a representation of a pedestrian appears in the at least one image; determine a planned navigation action based on at least one driving strategy for navigating the primary vehicle relative to the detected pedestrian crossing, wherein the determination of the planned navigation action is further based on the determined state of the detected traffic light and the determined proximity of the pedestrian relative to the detected pedestrian crossing; and cause one or more actuator systems of the primary vehicle to implement the planned navigation action.

[0013] In one embodiment, a method for navigating a primary vehicle near a pedestrian crossing is disclosed. The method may include: receiving from an image capture device at least one image representing the environment of the primary vehicle; detecting a representation of a pedestrian crossing in the at least one image based on analysis of the at least one image; determining, based on analysis of the at least one image, whether a representation of a pedestrian appears in the at least one image; detecting the presence of a traffic light in the environment of the primary vehicle; determining whether the detected traffic light is related to the primary vehicle and the pedestrian crossing; determining the state of the detected traffic light; determining the proximity of a pedestrian relative to the detected pedestrian crossing when a representation of a pedestrian appears in the at least one image; determining a planned navigation action based on at least one driving strategy for navigating the primary vehicle relative to the detected pedestrian crossing, wherein the determination of the planned navigation action is further based on the determined state of the detected traffic light and the determined proximity of the pedestrian relative to the detected pedestrian crossing; and causing one or more actuator systems of the primary vehicle to implement the planned navigation action.

[0014] In one embodiment, a method for navigating a primary vehicle near a pedestrian crossing is disclosed. The method may include: receiving from an image capture device at least one image representing the environment of the primary vehicle; detecting the start and end positions of the crossing; determining, based on analysis of the at least one image, whether a pedestrian is present near the crossing; detecting the presence of a traffic light in the environment of the primary vehicle; determining whether the traffic light is associated with the primary vehicle and the crossing; determining the state of the traffic light; determining a navigation action for the primary vehicle near the detected crossing based on: the association of the traffic light; the determined state of the traffic light; the presence of a pedestrian near the crossing; the shortest distance between the pedestrian and one of the start or end positions of the crossing; and the pedestrian's motion vector; and causing one or more actuator systems of the primary vehicle to perform the navigation action.

[0015] Consistent with other disclosed embodiments, a non-transitory computer-readable storage medium may store program instructions that can be executed by at least one processing device and perform any of the steps and / or methods described herein.

[0016] The foregoing general description and the following detailed description are merely exemplary and explanatory, and not intended to limit the scope of the claims. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various embodiments disclosed.

[0018] In the attached diagram:

[0019] Figure 1 It is a schematic representation of an example system consistent with the disclosed embodiments.

[0020] Figure 2AIt is an illustrative side view representation of an example vehicle that includes a system consistent with the disclosed embodiments.

[0021] Figure 2B It is consistent with the disclosed embodiments. Figure 2A The illustration shows a top view of the vehicle and system.

[0022] Figure 2C This is an illustrative top view representation of another embodiment of a vehicle that includes a system consistent with the disclosed embodiments.

[0023] Figure 2D This is an illustrative top view representation of yet another embodiment of a vehicle that includes a system consistent with the disclosed embodiments.

[0024] Figure 2E This is an illustrative top view representation of yet another embodiment of a vehicle that includes a system consistent with the disclosed embodiments.

[0025] Figure 2F This is a schematic representation of an example vehicle control system consistent with the disclosed embodiments.

[0026] Figure 3A It is an illustrative representation of the interior of a vehicle, including a rearview mirror and a user interface for a vehicle imaging system, consistent with the disclosed embodiments.

[0027] Figure 3B This is an illustration of an example of a camera mount configured to be positioned behind the rearview mirror and against the vehicle's windshield, consistent with the disclosed embodiments.

[0028] Figure 3C It is consistent with the disclosed embodiments. Figure 3B The diagram shows the camera installation from different perspectives.

[0029] Figure 3D This is an illustration of an example of a camera mounted behind a rearview mirror and against a vehicle windshield, consistent with the disclosed embodiments.

[0030] Figure 4 This is an exemplary block diagram of a memory configured to store instructions for performing one or more operations, consistent with the disclosed embodiments.

[0031] Figure 5A This is a flowchart illustrating an exemplary process consistent with the disclosed embodiments for evoking one or more navigation responses based on monocular image analysis.

[0032] Figure 5B This is a flowchart illustrating an exemplary process for detecting one or more vehicles and / or pedestrians in a set of images, consistent with the disclosed embodiments.

[0033] Figure 5C This is a flowchart illustrating an exemplary process for detecting road markings and / or lane geometry information in a set of images, consistent with the disclosed embodiments.

[0034] Figure 5D This is a flowchart illustrating an exemplary process for detecting traffic lights in a set of images, consistent with the disclosed embodiments.

[0035] Figure 5E This is a flowchart illustrating an example line procedure for inducing one or more navigation responses based on a vehicle path, consistent with the disclosed embodiments.

[0036] Figure 5F This is a flowchart illustrating an exemplary process for determining whether a vehicle ahead is changing lanes, consistent with the disclosed embodiments.

[0037] Figure 6 This is a flowchart illustrating an exemplary process consistent with the disclosed embodiments for evoking one or more navigation responses based on stereo image analysis.

[0038] Figure 7 This is a flowchart illustrating an exemplary process consistent with the disclosed embodiments for causing one or more navigation responses based on the analysis of three sets of images.

[0039] Figure 8 It is a block diagram representation of a module that is consistent with the disclosed embodiments and can be implemented by one or more specially programmed processing devices of the navigation system of an autonomous vehicle.

[0040] Figure 9 It is a navigation options diagram consistent with the disclosed embodiments.

[0041] Figure 10 It is a navigation options diagram consistent with the disclosed embodiments.

[0042] Figure 11A , Figure 11B and Figure 11C A schematic representation of navigation options for the primary vehicle in a merge zone, consistent with the disclosed embodiments, is provided.

[0043] Figure 11D An illustrated description of a dual-lane merging scenario consistent with the disclosed embodiments is provided.

[0044] Figure 11E An option diagram, consistent with the disclosed embodiments, is provided that is potentially useful in dual-lane merging scenarios.

[0045] Figure 12A representational image capturing the environment of the master vehicle and a graph of potential navigation constraints, consistent with the disclosed embodiments, is provided.

[0046] Figure 13 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0047] Figure 14 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0048] Figure 15 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0049] Figure 16 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0050] Figure 17A and 17B An illustrative diagram of a main vehicle navigating into a roundabout, consistent with the disclosed embodiments, is provided.

[0051] Figure 18 An algorithm flowchart for navigating vehicles, consistent with the disclosed embodiments, is provided.

[0052] Figure 19 An example of a main vehicle traveling on a multi-lane highway, consistent with the disclosed embodiments, is shown.

[0053] Figure 20A and 20B An example of a vehicle cutting in front of another vehicle, consistent with the disclosed embodiments, is shown.

[0054] Figure 21 An example of a vehicle following another vehicle, consistent with the disclosed embodiments, is shown.

[0055] Figure 22 An example of a vehicle leaving a parking lot and merging into a potentially busy road, consistent with the disclosed embodiments, is shown.

[0056] Figure 23 A vehicle traveling on a road, consistent with the disclosed embodiments, is shown.

[0057] Figures 24A-24D Four example scenarios consistent with the disclosed embodiments are shown.

[0058] Figure 25 An example scenario consistent with the disclosed embodiments is shown.

[0059] Figure 26An example scenario consistent with the disclosed embodiments is shown.

[0060] Figure 27 An example scenario consistent with the disclosed embodiments is shown.

[0061] Figure 28A and 28B An example of a scenario consistent with the disclosed embodiments, in which a vehicle is following another vehicle, is shown.

[0062] Figure 29A and 29B Example blame is shown in a scenario consistent with the disclosed embodiments.

[0063] Figure 30A and 30B Example attribution is shown in a scenario consistent with the disclosed embodiments.

[0064] Figures 31A-31D An example attribution in a drift scenario consistent with the disclosed embodiments is shown.

[0065] Figure 32A and 32B An example attribution is shown in a two-way traffic scenario consistent with the disclosed embodiments.

[0066] Figure 33A and 33B An example attribution is shown in a two-way traffic scenario consistent with the disclosed embodiments.

[0067] Figure 34A and 34B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0068] Figure 35A and 35B An example attribution is shown in a route prioritization scenario consistent with the disclosed embodiments.

[0069] Figure 36A and 36B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0070] Figure 37A and 37B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0071] Figure 38A and 38B An example attribution in a route prioritization scenario consistent with the disclosed embodiments is shown.

[0072] Figure 39A and39B An example attribution is shown in a route prioritization scenario consistent with the disclosed embodiments.

[0073] Figure 40A and 40B An example attribution is shown in a traffic light scenario consistent with the disclosed embodiments.

[0074] Figure 41A and 41B An example attribution is shown in a traffic light scenario consistent with the disclosed embodiments.

[0075] Figure 42A and 42B An example attribution is shown in a traffic light scenario consistent with the disclosed embodiments.

[0076] Figures 43A-43C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is shown.

[0077] Figures 44A-44C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is shown.

[0078] Figure 45A-45C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is shown.

[0079] Figures 46A-46D An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is shown.

[0080] Figure 47A This is a diagram illustrating the blame time and appropriate response consistent with the disclosed embodiments.

[0081] Figure 47B This is a diagram illustrating route priorities for routes with different geometries consistent with the disclosed embodiments.

[0082] Figure 47C It is an illustration of the longitudinal arrangement of routes of different geometries, consistent with the disclosed embodiments.

[0083] Figure 47D This is a diagram illustrating the safe longitudinal distance between vehicles consistent with the disclosed embodiments.

[0084] Figure 47E This is an illustration consistent with the disclosed embodiments, depicting a situation where a vehicle cannot predict the path of another vehicle.

[0085] Figure 47F This is a diagram illustrating route priority at a traffic light, consistent with the disclosed embodiments.

[0086] Figure 47G This is an illustration of an exemplary unstructured route consistent with the disclosed embodiments.

[0087] Figure 47H This is an illustration of exemplary lateral behavior in an unstructured situation consistent with the disclosed embodiments.

[0088] Figure 47I This is a diagram illustrating the exposure time and attribution time of the shielded area, consistent with the disclosed embodiments.

[0089] Figure 48A An example scenario is shown, consistent with the disclosed embodiments, in which two vehicles travel in opposite directions.

[0090] Figure 48B An example of a target vehicle traveling toward a host vehicle, consistent with the disclosed embodiments, is shown.

[0091] Figure 49 An example of a main vehicle maintaining a safe longitudinal distance, consistent with the disclosed embodiments, is shown.

[0092] Figure 50A and 50B A flowchart is provided that depicts an exemplary process for maintaining a safe longitudinal distance, consistent with the disclosed embodiments.

[0093] Figure 51A An example scenario is shown where two vehicles are laterally spaced apart from each other, consistent with the disclosed embodiments.

[0094] Figure 51B An example of a main vehicle maintaining a safe lateral distance, consistent with the disclosed embodiments, is shown.

[0095] Figure 52A An example of a master vehicle performing planned navigation actions consistent with the disclosed embodiments is shown.

[0096] Figure 52B An example is shown where the primary vehicle determines whether to perform a navigation action.

[0097] Figure 53A and 53B A flowchart is provided that depicts an example process for maintaining a safe lateral distance, consistent with the disclosed embodiments.

[0098] Figure 54 It is a schematic illustration of a roadway including a crosswalk, consistent with the disclosed embodiments.

[0099] Figure 55A and 55BIt is a schematic illustration of possible navigation actions performed by a vehicle traveling along a roadway, consistent with the disclosed embodiments.

[0100] Figure 55C and 55D An example of estimating the distance from a pedestrian to a crosswalk, consistent with the disclosed embodiments, is shown.

[0101] Figure 56 It is a flowchart describing a process for navigating a vehicle near a crosswalk, consistent with the disclosed embodiments. Detailed Implementation

[0102] The following detailed description refers to the accompanying drawings. Wherever possible, the same reference numerals are used in the drawings and the following description to refer to the same or similar parts. Although several illustrative embodiments are described herein, modifications, adjustments, and other implementations are possible. For example, components shown in the drawings may be replaced, added, or modified, and the illustrative methods described herein may be modified by replacing, reordering, removing, or adding steps to the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. Rather, the appropriate scope is defined by the appended claims.

[0103] Overview of autonomous vehicles

[0104] As used throughout this disclosure, the term "autonomous vehicle" means a vehicle capable of performing at least one navigation change without driver input. "Navigation change" refers to one or more of the vehicle's steering, braking, or acceleration / deceleration. "Autonomous" means that the vehicle does not need to be fully automatic (e.g., fully operable without a driver or driver input). Rather, autonomous vehicles include those capable of operating under driver control during certain time periods and without driver control during other time periods. Autonomous vehicles may also include those that control only certain aspects of vehicle navigation (such as steering (e.g., maintaining vehicle line between lane constraints) or certain steering operations in certain situations (but not all situations)) but leave other aspects to the driver (e.g., braking or braking in certain situations). In some cases, an autonomous vehicle may handle some or all aspects of the vehicle's braking, speed control, and / or steering.

[0105] Transportation infrastructure, featuring lane markings, traffic signs, and traffic lights designed to provide visual information to drivers, has been constructed because human drivers typically rely on visual cues and observation to control vehicles. Given these design characteristics of transportation infrastructure, autonomous vehicles can include cameras and processing units that analyze visual information captured from the vehicle's environment. Visual information can include, for example, images representing components of the transportation infrastructure that can be observed by the driver (e.g., lane markings, traffic signs, traffic lights, etc.) and other obstacles (e.g., other vehicles, pedestrians, debris, etc.). Furthermore, autonomous vehicles can utilize stored information, such as information that provides a model of the vehicle's environment during navigation. For example, a vehicle can use GPS data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data to provide information about its environment while the vehicle is in motion, and the vehicle (and other vehicles) can use this information to locate itself on the model. Some vehicles are also capable of communicating with each other, sharing information, and warning peer vehicles of hazards or changes in their surroundings.

[0106] System Overview

[0107] Figure 1 This is a block diagram representation of system 100 consistent with the disclosed example embodiments. System 100 may include various components depending on the requirements of a particular implementation. In some embodiments, system 100 may include a processing unit 110, an image acquisition unit 120, a position sensor 130, one or more memory units 140, 150, a map database 160, a user interface 170, and a wireless transceiver 172. Processing unit 110 may include one or more processing devices. In some embodiments, processing unit 110 may include an application processor 180, an image processor 190, or any other suitable processing device. Similarly, depending on the requirements of a particular application, image acquisition unit 120 may include any number of image acquisition devices and components. In some embodiments, image acquisition unit 120 may include one or more image capture devices (e.g., a camera, CCD, or any other type of image sensor), such as image capture device 122, image capture device 124, and image capture device 126. System 100 may also include a data interface 128 that communicatively connects processing unit 110 to image acquisition unit 120. For example, data interface 128 may include one or more wired and / or wireless links for transmitting image data acquired by image acquisition unit 120 to processing unit 110.

[0108] Wireless transceiver 172 may include one or more devices configured to exchange transmissions over an air interface to one or more networks (e.g., cellular, Internet, etc.) using radio frequency, infrared frequency, magnetic field, or electric field. Wireless transceiver 172 may use any known standard to transmit and / or receive data (e.g., Wi-Fi, etc.). (Bluetooth Smart, 802.15.4, ZigBee, etc.). Such transmissions can include communication from the master vehicle to one or more remotely located servers. Such transmissions can also include (one-way or two-way) communication between the master vehicle and one or more target vehicles in the master vehicle's environment (e.g., to coordinate the master vehicle's navigation in consideration of or in conjunction with target vehicles in the master vehicle's environment), or even broadcast transmissions to unspecified receivers near the transmitting vehicle.

[0109] Both application processor 180 and image processor 190 can include various types of hardware-based processing devices. For example, either or both of application processor 180 and image processor 190 can include a microprocessor, a preprocessor (such as an image preprocessor), a graphics processor, a central processing unit (CPU), supporting circuitry, a digital signal processor, an integrated circuit, memory, or any other type of device suitable for running applications and for image processing and analysis. In some embodiments, application processor 180 and / or image processor 190 can include any type of single-core or multi-core processor, mobile device microcontroller, central processing unit, etc. Various processing devices can be used, including, for example, those available from, such as… Processors obtained from manufacturers, and can include various architectures (e.g., x86 processors, ...). wait).

[0110] In some embodiments, application processor 180 and / or image processor 190 may include components that can be accessed from... Any EyeQ series processor chip obtained. These processor designs all include multiple processing units with local memory and instruction sets. Such a processor may include video input for receiving image data from multiple image sensors, and may also include video output capability. In one example, It uses 90nm-micron technology that operates at 332 MHz. The architecture consists of two floating-point hyper-threaded 32-bit RISC CPUs ( (core), five Visual Computing Engines (VCE), and three Vector Microcode Processors The system consists of a Denali 64-bit mobile DDR controller, a 128-bit internal Sonics Interconnect, dual 16-bit video input and 18-bit video output controllers, a 16-channel DMA, and several peripheral devices. The MIPS34K CPU manages five VCEs and three VMPs. TM And DMA, a second MIPS34K CPU and multi-channel DMA, and other peripherals. Five VCEs, three The MIPS34K CPU can perform the intensive vision computations required for versatile bundled applications. In another example, it is a third-generation processor and is Six times stronger It can be used in the disclosed embodiments. In other examples, it can be used in the disclosed embodiments. and / or Of course, any newer or future EyeQ processing devices can also be used with the disclosed embodiments.

[0111] Any processing device disclosed herein can be configured to perform certain functions. Configuring a processing device (such as any described EyeQ processor or other controller or microprocessor) to perform certain functions may include programming computer-executable instructions and making those instructions available to the processing device for execution during operation of the processing device. In some embodiments, configuring the processing device may include programming the processing device directly using architectural instructions. In other embodiments, configuring the processing device may include storing executable instructions on memory accessible to the processing device during operation. For example, the processing device may access the memory during operation to obtain and execute the stored instructions. In any case, a processing device configured to perform the sensing, image analysis, and / or navigation functions disclosed herein represents a dedicated hardware-based system in the control of multiple hardware-based components of a master vehicle.

[0112] although Figure 1 Two separate processing devices included in processing unit 110 are depicted, but more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to perform the tasks of application processor 180 and image processor 190. In other embodiments, these tasks may be performed by more than two processing devices. Additionally, in some embodiments, system 100 may include one or more processing units 110 without other components such as image acquisition unit 120.

[0113] Processing unit 110 may include various types of devices. For example, processing unit 110 may include various devices such as controllers, image preprocessors, central processing units (CPUs), support circuitry, digital signal processors, integrated circuits, memory, or any other type of device for image processing and analysis. Image preprocessors may include video processors for capturing, digitizing, and processing images from image sensors. CPUs may include any number of microcontrollers or microprocessors. Support circuitry may be any number of circuits known in the art, including caches, power supplies, clocks, and input / output circuits. Memory may store software that controls the operation of the system when executed by the processor. Memory may include databases and image processing software. Memory may include any number of random access memories, read-only memories, flash memory, disk drives, optical storage devices, magnetic tape storage devices, removable storage devices, and other types of storage devices. In one instance, the memory may be separate from processing unit 110. In another instance, the memory may be integrated into processing unit 110.

[0114] Each memory unit 140, 150 may include software instructions that, when executed by a processor (e.g., application processor 180 and / or image processor 190), can control the operation of various aspects of system 100. For example, these memory units may include various database and image processing software, as well as trained systems such as neural networks and deep neural networks. Memory units may include random access memory, read-only memory, flash memory, disk drives, optical storage devices, magnetic tape storage devices, removable storage devices, and / or any other type of storage device. In some embodiments, memory units 140, 150 may be decoupled from application processor 180 and / or image processor 190. In other embodiments, these memory units may be integrated into application processor 180 and / or image processor 190.

[0115] The position sensor 130 may include any type of device suitable for determining the location associated with at least one component of the system 100. In some embodiments, the position sensor 130 may include a GPS receiver. Such a receiver can determine the user's location and speed by processing signals broadcast by Global Positioning System satellites. The location information from the position sensor 130 may be made available to the application processor 180 and / or the image processor 190.

[0116] In some embodiments, system 100 may include components such as a speed sensor (e.g., a speedometer) for measuring the speed of vehicle 200. System 100 may also include one or more accelerometers (single-axis or multi-axis) for measuring the acceleration of vehicle 200 along one or more axes.

[0117] Memory units 140 and 150 may include a database, or data organized in any other form indicating the location of known landmarks. Sensory information about the environment (such as images from lidar or stereo processing of two or more images, radar signals, depth information) can be processed together with location information (such as GPS coordinates, vehicle self-motion, etc.) to determine the vehicle's current position relative to known landmarks and refine the vehicle's position. Certain aspects of this technology are included in what is known as REM... TM In the positioning technology, the technology is sold by the assignee of this application.

[0118] User interface 170 may include any device suitable for providing information to or receiving input from one or more users of system 100. In some embodiments, user interface 170 may include user input devices, including, for example, a touchscreen, microphone, keyboard, pointer device, tracking wheel, camera, knob, button, etc. Using such input devices, users can provide information input or commands to system 100 by typing instructions or information, providing voice commands, using buttons, pointers or eye-tracking capabilities to select menu options on the screen, or by any other suitable technology for transmitting information to system 100.

[0119] User interface 170 may be equipped with one or more processing devices configured to provide and receive information from the user, and process that information for use by, for example, application processor 180. In some embodiments, such processing devices may execute instructions to recognize and track eye movements, receive and interpret voice commands, recognize and interpret touch and / or gestures made on a touchscreen, respond to keyboard input or menu selection, etc. In some embodiments, user interface 170 may include a display, speakers, haptic devices, and / or any other devices for providing output information to the user.

[0120] Map database 160 may include any type of database for storing map data useful to system 100. In some embodiments, map database 160 may include data relating to the location of various items in a reference coordinate system, including roads, water features, geographic features, businesses, points of interest, restaurants, gas stations, etc. Map database 160 may not only store the locations of these items but also descriptors associated with them, including, for example, names associated with any stored features. In some embodiments, map database 160 may be physically located together with other components of system 100. Alternatively or additionally, map database 160 or a portion thereof may be located remotely relative to other components of system 100 (e.g., processing unit 110). In such embodiments, information from map database 160 may be downloaded to a network via wired or wireless data connections (e.g., via cellular networks and / or the Internet, etc.). In some cases, map database 160 may store a sparse data model comprising a polynomial representation of certain road features (e.g., lane markings) or the target trajectory of a primary vehicle. Map database 160 may also include stored representations of various identified landmarks, which can be used to determine or update the known position of the master vehicle relative to the target trajectory. Among other potential identifiers, landmark representations may also include data fields such as landmark type and landmark location.

[0121] Image capture devices 122, 124, and 126 may all include any type of device suitable for capturing at least one image from the environment. Furthermore, any number of image capture devices can be used to acquire images for input to the image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. Reference will be made below. Figures 2B to 2E Image capture devices 122, 124 and 126 are further described.

[0122] One or more cameras (e.g., image capture devices 122, 124, and 126) may be part of a sensing block included on the vehicle. The sensing block may include a variety of other sensors, and any or all of these sensors may be relied upon to develop a sensed navigational state of the vehicle. In addition to cameras (front, side, rear, etc.), other sensors (such as RADAR, LIDAR, and acoustic sensors) may be included in the sensing block. Furthermore, the sensing block may include one or more components configured to transmit and receive information relating to the vehicle's environment. For example, such components may include a wireless transceiver (RF, etc.) that can receive sensor-based information or any other type of information relating to the environment of the main vehicle from a source located at a distance relative to the main vehicle. This information may include sensor output information or related information received from vehicle systems other than the main vehicle. In some embodiments, this information may include information received from a remote computing device, a central server, etc. Moreover, cameras may be configured in many different ways: a single camera unit, multiple cameras, a camera cluster, long FOV, short FOV, wide-angle, fisheye, etc.

[0123] System 100 or its various components can be integrated into a variety of different platforms. In some embodiments, system 100 may be included on vehicle 200, such as... Figure 2A As shown. For example, vehicle 200 may be equipped with the above-mentioned features. Figure 1 The system 100 described includes the processing unit 110 and any other components. While in some embodiments the vehicle 200 may be equipped with only a single image capture device (e.g., a camera), in other embodiments, such as combining... Figures 2B to 2E The embodiments discussed can use multiple image capture devices. For example, Figure 2A Either of the image capture devices 122 and 124 of the vehicle 200 shown may be part of an ADAS (Advanced Driver Assistance System) imaging set.

[0124] The image capture device, included on the vehicle 200 as part of the image acquisition unit 120, can be located at any suitable location. In some embodiments, such as Figures 2A to 2E ,as well as Figures 3A to 3C As shown, the image capture device 122 can be located near the rearview mirror. This location provides a line of sight similar to that of the driver of vehicle 200, which can help determine what is visible and invisible to the driver. The image capture device 122 can be placed anywhere near the rearview mirror, and placing the image capture device 122 on the driver's side of the mirror can also help obtain an image representing the driver's field of vision and / or line of sight.

[0125] Other locations for the image capturing device of the image acquisition unit 120 can also be used. For example, the image capturing device 124 can be located on or within the bumper of the vehicle 200. This location is particularly suitable for image capturing devices with a wide field of view. The line of sight of the image capturing device located on the bumper may be different from that of the driver, and therefore, the bumper image capturing device and the driver may not always see the same object. The image capturing devices (e.g., image capturing devices 122, 124, and 126) can also be located in other locations. For example, the image capturing device can be located on or within one or both of the side mirrors of the vehicle 200, on the roof of the vehicle 200, on the hood of the vehicle 200, on the trunk of the vehicle 200, on the side of the vehicle 200, mounted on any window of the vehicle 200, behind any window of the vehicle 200, or in front of any window of the vehicle 200, and in or near lighting equipment mounted on the front and / or rear of the vehicle 200, etc.

[0126] In addition to the image capture equipment, vehicle 200 may also include various other components of system 100. For example, processing unit 110 may be included on vehicle 200, integrated with or separate from the vehicle's engine control unit (ECU). Vehicle 200 may also be equipped with position sensor 130 such as a GPS receiver, and may also include map database 160 and memory units 140 and 150.

[0127] As previously discussed, the wireless transceiver 172 can receive data via one or more networks (e.g., cellular networks, the Internet, etc.) and / or. For example, the wireless transceiver 172 can upload data collected by the system 100 to one or more servers and download data from one or more servers. Via the wireless transceiver 172, the system 100 can receive, for example, periodic or on-demand updates to data stored in map database 160, memory 140, and / or memory 150. Similarly, the wireless transceiver 172 can upload any data from the system 100 (e.g., images captured by image acquisition unit 120, data received by position sensor 130 or other sensors, vehicle control system, etc.) and / or any data processed by processing unit 110 to one or more servers.

[0128] System 100 can upload data to a server (e.g., to the cloud) based on privacy level settings. For example, System 100 can implement privacy level settings to specify or restrict the types of data (including metadata) that can uniquely identify a vehicle and / or its driver / owner that are sent to the server. Such settings can be configured by a user via, for example, a wireless transceiver 172, can be initialized by factory default settings, or by data received by the wireless transceiver 172.

[0129] In some embodiments, system 100 may upload data according to a “high” privacy level, and, if configured, system 100 may transmit data (e.g., route-related location information, captured images, etc.) without any details about a specific vehicle and / or driver / owner. For example, when uploading data according to a “high” privacy setting, system 100 may not include the vehicle identification number (VIN) or the name of the vehicle’s driver or owner, and may instead transmit data (such as captured images and / or restricted location information related to the route).

[0130] Other privacy levels may also be considered. For example, system 100 may transmit data to the server at an "intermediate" privacy level and may include additional information not included at a "high" privacy level, such as the vehicle's brand and / or model and / or vehicle type (e.g., passenger vehicle, SUV, truck, etc.). In some embodiments, system 100 may upload data at a "low" privacy level. At a "low" privacy level setting, system 100 may upload data that includes information sufficient to uniquely identify a particular vehicle, owner / driver, and / or a portion or the entire route traveled by the vehicle. For example, such "low" privacy level data may include one or more of the following: VIN, driver / owner name, vehicle's origin point before departure, vehicle's intended destination, vehicle's brand and / or model, vehicle type, etc.

[0131] Figure 2A It is an illustrative side view representation of an exemplary vehicle imaging system consistent with the disclosed embodiments. Figure 2B yes Figure 2A The illustrated top view of the embodiment shown is exemplary. Figure 2B As shown, the disclosed embodiments may include a vehicle 200, which includes a system 100 in its body, the system 100 having a first image capture device 122 located near and / or close to the driver's side mirror of the vehicle 200, a second image capture device 124 located on or therein in a bumper area of ​​the vehicle 200 (e.g., one of the bumper areas 210), and a processing unit 110.

[0132] like Figure 2C As shown, both image capture devices 122 and 124 can be located near the rearview mirror of vehicle 200 and / or close to the driver. Furthermore, although... Figure 2B and Figure 2C Two image capture devices 122 and 124 are shown; it should be understood that other embodiments may include more than two image capture devices. For example, in Figure 2D and Figure 2E In the embodiment shown, the first image capture device 122, the second image capture device 124, and the third image capture device 126 are included in the system 100 of the vehicle 200.

[0133] like Figure 2D As shown, image capture device 122 may be located near and / or close to the rearview mirror of vehicle 200, and image capture devices 124 and 126 may be located on or within the bumper area of ​​vehicle 200 (e.g., one of bumper areas 210). Figure 2E As shown, image capture devices 122, 124, and 126 may be located near the rearview mirror and / or close to the driver's seat of vehicle 200. The disclosed embodiments are not limited to any particular number and configuration of image capture devices, and the image capture devices may be located in any suitable location within and / or on vehicle 200.

[0134] It should be understood that the disclosed embodiments are not limited to vehicles and can be applied to other scenarios. It should also be understood that the disclosed embodiments are not limited to a specific type of vehicle 200 and can be applied to all types of vehicles, including cars, trucks, trailers, and other types of vehicles.

[0135] The first image capture device 122 may include any suitable type of image capture device. The image capture device 122 may include an optical axis. In one example, the image capture device 122 may include an Aptina M9V024WVGA sensor with a global shutter. In other embodiments, the image capture device 122 may provide a resolution of 1280 × 960 pixels and may include a rolling shutter. The image capture device 122 may include various optical elements. In some embodiments, it may include one or more lenses, for example, for providing the desired focal length and field of view for the image capture device. In some embodiments, the image capture device 122 may be associated with a 6mm lens or a 12mm lens. In some embodiments, such as Figure 2DAs shown, image capture device 122 can be configured to capture images with a desired field of view (FOV) 202. For example, image capture device 122 can be configured to have a conventional FOV, such as 46 degrees, 50 degrees, 52 degrees, or larger, within the range of 40 to 56 degrees. Alternatively, image capture device 122 can be configured to have a narrow FOV, such as 28 degrees or 36 degrees, within the range of 23 to 40 degrees. Furthermore, image capture device 122 can be configured to have a wide FOV within the range of 100 to 180 degrees. In some embodiments, image capture device 122 can include a wide-angle bumper camera or a camera with an FOV of up to 180 degrees. In some embodiments, image capture device 122 can be a 7.2M (megapixel) image capture device with an aspect ratio of approximately 2:1 (e.g., H×V = 3800×1900 pixels) and a horizontal FOV of approximately 100 degrees. Such an image capture device can be used instead of a three-image capture device configuration. Due to significant lens distortion, in embodiments of image capture devices using radially symmetrical lenses, the vertical FOV of such image capture devices can be significantly less than 50 degrees. For example, such lenses may not be radially symmetrical, which would allow a vertical FOV greater than 50 degrees with a horizontal FOV of 100 degrees.

[0136] The first image capturing device 122 can acquire multiple first images of a scene associated with the vehicle 200. Each of the multiple first images can be acquired as a series of image scan lines, which can be captured using a rolling shutter. Each scan line can include multiple pixels.

[0137] The first image capturing device 122 may have a scan rate associated with the acquisition of each of the first series of image scan lines. The scan rate may refer to the rate at which the image sensor can acquire image data associated with each pixel included in a particular scan line.

[0138] Image capturing devices 122, 124, and 126 may include any suitable type and number of image sensors, such as CCD sensors or CMOS sensors. In one embodiment, a CMOS image sensor and a rolling shutter may be employed, such that each pixel in a row is read one at a time, and scanning of each row continues on a line-by-line basis until the entire image frame has been captured. In some embodiments, the rows may be captured sequentially from top to bottom relative to the frame.

[0139] In some embodiments, one or more of the image capture devices disclosed herein (e.g., image capture devices 122, 124, and 126) may constitute a high-resolution imager and may have a resolution greater than 5M pixels, 7M pixels, 10M pixels, or more pixels.

[0140] The use of a rolling shutter can cause pixels in different rows to be exposed and captured at different times, which can cause distortion and other image artifacts in the captured image frame. On the other hand, when the image capture device 122 is configured to utilize global or synchronous shutter operation, all pixels can be exposed for the same amount of time and during a common exposure period. As a result, the image data in the frame collected from a system using a global shutter represents a snapshot of the entire FOV (such as FOV 202) at a specific time. In contrast, in a rolling shutter application, each row in the frame is exposed and data is captured at different times. Therefore, in an image capture device with a rolling shutter, moving objects may appear distorted. This phenomenon will be described in more detail below.

[0141] The second image capture device 124 and the third image capture device 126 can be any type of image capture device. Similar to the first image capture device 122, each of image capture devices 124 and 126 may include an optical axis. In one embodiment, each of image capture devices 124 and 126 may include an Aptina M9V024 WVGA sensor with a global shutter. Alternatively, each of image capture devices 124 and 126 may include a rolling shutter. Similar to image capture device 122, image capture devices 124 and 126 can be configured to include various lenses and optics. In some embodiments, the lenses associated with image capture devices 124 and 126 may provide a field of view (FOV) (such as FOV 204 and 206) that is equal to or narrower than the FOV associated with image capture device 122 (such as FOV 202). For example, image capture devices 124 and 126 may have an FOV of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less.

[0142] Image capture devices 124 and 126 can acquire multiple second and third images of a scene associated with vehicle 200. Each of these multiple second and third images can be acquired as a second series of image scan lines and a third series of image scan lines, which can be captured using a rolling shutter. Each scan line or line can have multiple pixels. Image capture devices 124 and 126 can have a second scan rate and a third scan rate associated with the acquisition of each image scan line included in the second and third series.

[0143] Each image capture device 122, 124, and 126 can be positioned at any suitable location and orientation relative to the vehicle 200. The relative positions of the image capture devices 122, 124, and 126 can be selected to facilitate the fusion of information acquired from the image capture devices. For example, in some embodiments, the field of view (FOV) associated with image capture device 124 (such as FOV 204) may partially or completely overlap with the FOV associated with image capture device 122 (such as FOV 202) and the FOV associated with image capture device 126 (such as FOV 206).

[0144] Image capture devices 122, 124, and 126 can be located on vehicle 200 at any suitable relative height. In one example, a height difference may exist between image capture devices 122, 124, and 126, which can provide sufficient parallax information to enable stereoscopic analysis. For example, as Figure 2A As shown, the two image capture devices 122 and 124 are at different heights. For example, a lateral displacement difference may also exist between image capture devices 122, 124, and 126, providing additional parallax information for the stereo analysis of processing unit 110. Figure 2C and Figure 2D As shown, the difference in lateral displacement can be expressed by d x In some embodiments, there may be forward or backward displacement (e.g., range displacement) between image capture devices 122, 124, and 126. For example, image capture device 122 may be located 0.5 to 2 meters or more behind image capture device 124 and / or image capture device 126. This type of displacement allows one of the image capture devices to cover potential blind spots of the other(one or more) image capture devices(s).

[0145] Image capture device 122 may have any suitable resolution capability (e.g., the number of pixels associated with an image sensor), and the resolution of one or more image sensors associated with image capture device 122 may be higher, lower, or the same as the resolution of one or more image sensors associated with image capture devices 124 and 126. In some embodiments, one or more image sensors associated with image capture device 122 and / or image capture devices 124 and 126 may have a resolution of 640×480, 1024×768, 1280×960, or any other suitable resolution.

[0146] The frame rate (e.g., the rate at which an image capture device acquires a set of pixel data for an image frame before continuing to capture pixel data associated with the next image frame) can be controllable. The frame rate associated with image capture device 122 can be higher, lower, or the same as the frame rates associated with image capture devices 124 and 126. The frame rates associated with image capture devices 122, 124, and 126 can depend on various factors that may affect the timing of the frame rate. For example, one or more of image capture devices 122, 124, and 126 may include selectable pixel delay periods that are applied before or after acquiring image data associated with one or more pixels of the image sensors in image capture devices 122, 124, and / or 126. Typically, image data corresponding to each pixel can be acquired based on the clock rate used for the device (e.g., one pixel per clock cycle). Additionally, in embodiments including a rolling shutter, one or more of the image capture devices 122, 124, and 126 may include a selectable horizontal blanking period applied before or after acquiring image data associated with a row of pixels of the image sensors in the image capture devices 122, 124, and / or 126. Furthermore, one or more of the image capture devices 122, 124, and 126 may include a selectable vertical blanking period applied before or after acquiring image data associated with image frames of the image capture devices 122, 124, and 126.

[0147] These timing controls enable synchronization of frame rates associated with image capture devices 122, 124, and 126, even if each has a different line scan rate. Furthermore, as will be discussed in more detail below, among other factors (e.g., image sensor resolution, maximum line scan rate, etc.), these selectable timing controls enable synchronization of image capture from areas where the field of view (FOV) of image capture device 122 overlaps with one or more FOVs of image capture devices 124 and 126, even if the field of view of image capture device 122 differs from the FOV of image capture devices 124 and 126.

[0148] The frame rate timing in image capture devices 122, 124, and 126 can depend on the resolution of the associated image sensor. For example, assuming that the line scan rates are similar for two devices, if one device includes an image sensor with a resolution of 640×480 and the other device includes an image sensor with a resolution of 1280×960, more time will be needed to acquire image data frames from the sensor with the higher resolution.

[0149] Another factor that may affect the timing of image data acquisition in image capture devices 122, 124, and 126 is the maximum line scan rate. For example, acquiring one line of image data from an image sensor included in image capture devices 122, 124, and 126 will require a certain minimum amount of time. Assuming no pixel delay period is added, this minimum amount of time for acquiring one line of image data will be related to the maximum line scan rate for a particular device. Devices offering a higher maximum line scan rate have the potential to offer a higher frame rate than devices with a lower maximum line scan rate. In some embodiments, one or more of image capture devices 124 and 126 may have a maximum line scan rate higher than the maximum line scan rate associated with image capture device 122. In some embodiments, the maximum line scan rate of image capture devices 124 and / or 126 may be 1.25, 1.5, 1.75, or 2 times or more of the maximum line scan rate of image capture device 122.

[0150] In another embodiment, image capture devices 122, 124, and 126 may have the same maximum line scan rate, but image capture device 122 may operate at a scan rate less than or equal to its maximum scan rate. The system may be configured such that one or more of image capture devices 124 and 126 operate at a line scan rate equal to the line scan rate of image capture device 122. In other instances, the system may be configured such that the line scan rate of image capture device 124 and / or image capture device 126 may be 1.25, 1.5, 1.75, or 2 or more times the line scan rate of image capture device 122.

[0151] In some embodiments, image capturing devices 122, 124, and 126 may be asymmetrical. That is, they may include cameras with different fields of view (FOV) and focal lengths. For example, the fields of view of image capturing devices 122, 124, and 126 may include any desired area of ​​the environment of vehicle 200. In some embodiments, one or more of image capturing devices 122, 124, and 126 may be configured to acquire image data from the environment in front of vehicle 200, behind vehicle 200, to the side of vehicle 200, or a combination thereof.

[0152] Furthermore, the focal length associated with each image capture device 122, 124, and / or 126 can be selectable (e.g., by including a suitable lens, etc.) so that each device acquires images of objects at a desired distance range relative to the vehicle 200. For example, in some embodiments, image capture devices 122, 124, and 126 can acquire images of approaching objects within a few meters of the vehicle. Image capture devices 122, 124, and 126 can also be configured to acquire images of objects at a greater distance from the vehicle (e.g., 25m, 50m, 100m, 150m, or more). Furthermore, the focal lengths of image capturing devices 122, 124, and 126 can be selected such that one image capturing device (e.g., image capturing device 122) can acquire images of objects relatively close to the vehicle (e.g., within 10m or 20m), while other image capturing devices (e.g., image capturing devices 124 and 126) can acquire images of objects farther away from the vehicle 200 (e.g., greater than 20m, 50m, 100m, 150m, etc.).

[0153] According to some embodiments, the field of view (FOV) of one or more image capture devices 122, 124, and 126 may have a wide angle. For example, an FOV of 140 degrees may be advantageous, especially for image capture devices 122, 124, and 126 that can be used to capture images of areas near vehicle 200. For example, image capture device 122 may be used to capture images of areas to the right or left of vehicle 200, and in these embodiments, it may be desirable for image capture device 122 to have a wide FOV (e.g., at least 140 degrees).

[0154] The field of view associated with each of the image capture devices 122, 124, and 126 can depend on its respective focal length. For example, as the focal length increases, the corresponding field of view decreases.

[0155] Image capture devices 122, 124, and 126 can be configured to have any suitable field of view. In one particular example, image capture device 122 may have a horizontal FOV of 46 degrees, image capture device 124 may have a horizontal FOV of 23 degrees, and image capture device 126 may have a horizontal FOV between 23 degrees and 46 degrees. In another example, image capture device 122 may have a horizontal FOV of 52 degrees, image capture device 124 may have a horizontal FOV of 26 degrees, and image capture device 126 may have a horizontal FOV between 26 degrees and 52 degrees. In some embodiments, the ratio of the FOV of image capture device 122 to the FOV of image capture device 124 and / or image capture device 126 may vary from 1.5 to 2.0. In other embodiments, this ratio may vary between 1.25 and 2.25.

[0156] System 100 may be configured such that the field of view of image capture device 122 at least partially or completely overlaps with the field of view of image capture devices 124 and / or 126. In some embodiments, system 100 may be configured such that the field of view of image capture devices 124 and 126, for example, falls within (e.g., is narrower than) the field of view of image capture device 122 and shares a common center with the field of view of image capture device 122. In other embodiments, image capture devices 122, 124, and 126 may capture adjacent FOVs, or may have partial overlap in their FOVs. In some embodiments, the field of view of image capture devices 122, 124, and 126 may be aligned such that the center of the narrower FOV image capture device 124 and / or 126 may be located in the lower half of the field of view of the wider FOV device 122.

[0157] Figure 2F This is a schematic representation of an exemplary vehicle control system consistent with the disclosed embodiments. (As...) Figure 2F As indicated, vehicle 200 may include a throttle control system 220, a braking system 230, and a steering system 240. System 100 may provide inputs (e.g., control signals) to one or more of the throttle control system 220, braking system 230, and steering system 240 via one or more data links (e.g., any wired and / or wireless links for transmitting data). For example, based on analysis of images acquired by image capture devices 122, 124, and / or 126, system 100 may provide control signals to one or more of the throttle control system 220, braking system 230, and steering system 240 to navigate vehicle 200 (e.g., by inducing acceleration, turning, lane changes, etc.). Furthermore, system 100 may receive inputs from one or more of the throttle control system 220, braking system 230, and steering system 240 indicative of operating conditions of vehicle 200 (e.g., speed, whether vehicle 200 is braking and / or turning, etc.). The following is in conjunction with... Figures 4 to 7 Further details will be provided.

[0158] like Figure 3AAs shown, vehicle 200 may also include a user interface 170 for interacting with the driver or passengers of vehicle 200. For example, user interface 170 in a vehicle application may include a touchscreen 320, a knob 330, a button 340, and a microphone 350. The driver or passengers of vehicle 200 may also interact with system 100 using handles (e.g., located on or near the steering wheel of vehicle 200, including, for example, a turn signal handle), buttons (e.g., located on the steering wheel of vehicle 200), etc. In some embodiments, microphone 350 may be located adjacent to rearview mirror 310. Similarly, in some embodiments, image capture device 122 may be located near rearview mirror 310. In some embodiments, user interface 170 may also include one or more speakers 360 (e.g., speakers of a vehicle audio system). For example, system 100 may provide various notifications (e.g., alarms) via speaker 360.

[0159] Figures 3B to 3D This is an example of an example camera mount 370 configured consistent with the disclosed embodiments, located behind a rearview mirror (e.g., rearview mirror 310) and opposite the vehicle's windshield. Figure 3B As shown, camera mount 370 may include image capture devices 122, 124, and 126. Image capture devices 124 and 126 may be located behind a light shield 380, wherein the light shield 380 may be flush relative to a vehicle windshield and comprises a composition of a film and / or anti-reflective material. For example, the light shield 380 may be positioned such that it is aligned with a vehicle windshield having a matching bevel. In some embodiments, each of image capture devices 122, 124, and 126 may be located behind the light shield 380, for example in Figure 3D The embodiments described herein are not limited to any particular configuration of image capture devices 122, 124 and 126, camera mount 370 and light shield 380. Figure 3C yes Figure 3B The image shows an example of a camera mount 370 from a frontal view.

[0160] As those skilled in the art who benefit from this disclosure will understand, many variations and / or modifications can be made to the disclosed embodiments. For example, not all components are necessary for the operation of system 100. Furthermore, any component may be located in any suitable part of system 100 and components may be rearranged into various configurations while providing the functionality of the disclosed embodiments. Thus, the foregoing configurations are exemplary, and regardless of the configurations discussed above, system 100 can provide a wide range of functions to analyze the environment surrounding vehicle 200 and navigate vehicle 200 in response to that analysis.

[0161] As discussed in more detail below and consistent with the various disclosed embodiments, system 100 can provide various features related to autonomous driving and / or driver assistance technologies. For example, system 100 can analyze image data, location data (e.g., GPS location information), map data, speed data, and / or data from sensors included in vehicle 200. System 100 can collect data from, for example, image acquisition unit 120, position sensor 130, and other sensors for analysis. Furthermore, system 100 can analyze the collected data to determine whether vehicle 200 should take a certain action, and then automatically take the determined action without human intervention. For example, when vehicle 200 is navigating without human intervention, system 100 can automatically control the braking, acceleration, and / or steering of vehicle 200 (e.g., by sending control signals to one or more of throttle adjustment system 220, braking system 230, and steering system 240). Additionally, system 100 can analyze the collected data and issue warnings and / or alerts to vehicle occupants based on the analysis of the collected data. Additional details regarding various embodiments of system 100 are provided below.

[0162] Forward Multi-Imaging System

[0163] As discussed above, system 100 can provide driver assistance functions using a multi-camera system. The multi-camera system can use one or more cameras facing forward of the vehicle. In other embodiments, the multi-camera system may include one or more cameras facing the side or rear of the vehicle. In one embodiment, for example, system 100 may use a dual-camera imaging system, wherein a first camera and a second camera (e.g., image capture devices 122 and 124) may be located at the front and / or side of the vehicle (e.g., vehicle 200). Other camera configurations are consistent with the disclosed embodiments, and the configurations disclosed herein are examples. For example, system 100 may include a configuration of any number of cameras (e.g., one, two, three, four, five, six, seven, eight, etc.). Furthermore, system 100 may include a camera “cluster.” For example, a camera cluster (including any suitable number of cameras, such as one, four, eight, etc.) may be forward-facing relative to the vehicle, or may be oriented in any other direction (e.g., rearward, sideways, angled, etc.). Thus, system 100 may include multiple camera clusters, each oriented in a specific direction to capture images from a specific area of ​​the vehicle environment.

[0164] The first camera may have a field of view that is larger than, smaller than, or partially overlaps with the field of view of the second camera. Furthermore, the first camera may be connected to a first image processor to perform monocular image analysis on images provided by the first camera, and the second camera may be connected to a second image processor to perform monocular image analysis on images provided by the second camera. The outputs of the first and second image processors (e.g., processed information) may be combined. In some embodiments, the second image processor may receive images from both the first and second cameras to perform stereo analysis. In another embodiment, system 100 may use a three-camera imaging system, where each camera has a different field of view. Therefore, such a system can make decisions based on information obtained from objects located at varying distances in front of and to the sides of the vehicle. The reference to monocular image analysis may refer to an instance of performing image analysis based on images captured from a single viewpoint (e.g., from a single camera). Stereo image analysis may refer to an instance of performing image analysis based on two or more images captured using one or more variations of image capture parameters. For example, images captured suitable for performing stereo image analysis may include images captured from two or more different locations, from different fields of view, using different focal lengths, or with parallax information, etc.

[0165] For example, in one embodiment, system 100 may use image capture devices 122-126 to implement a three-camera configuration. In this configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees or other values ​​selected from the range of approximately 20 to 45 degrees), image capture device 124 may provide a wide field of view (e.g., 150 degrees or other values ​​selected from the range of approximately 100 to approximately 180 degrees), and image capture device 126 may provide an intermediate field of view (e.g., 46 degrees or other values ​​selected from the range of approximately 35 to approximately 60 degrees). In some embodiments, image capture device 126 may serve as a main camera or a base camera. Image capture devices 122-126 may be located behind rearview mirror 310 and substantially side-by-side (e.g., 6 cm apart). Furthermore, in some embodiments, as discussed above, one or more of image capture devices 122-126 may be mounted behind a sun visor 380 flush with the windshield of vehicle 200. This occlusion can be used to minimize the impact of any reflections from inside the vehicle on the image capture devices 122-126.

[0166] In another embodiment, as described above... Figure 3B and 3CThe wide field-of-view camera discussed (e.g., image capture device 124 in the example above) can be mounted below the narrow field-of-view camera and the main field-of-view camera (e.g., image capture devices 122 and 126 in the example above). This configuration provides a free line of sight from the wide field-of-view camera. To reduce reflections, the camera can be mounted close to the windshield of the vehicle 200, and a polarizer can be included on the camera to dampen reflected light.

[0167] A three-camera system can provide certain performance characteristics. For example, some embodiments may include the ability to verify object detection by one camera based on detection results from another camera. In the three-camera configuration discussed above, processing unit 110 may include, for example, three processing devices (e.g., three EyeQ series processor chips as discussed above), each dedicated to processing one or more images captured by image capture devices 122-126.

[0168] In a three-camera system, the first processing device can receive images from both the main camera and the narrow field-of-view camera, and perform visual processing on the narrow FOV camera to detect, for example, other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the first processing device can calculate the pixel parallax between the images from the main camera and the narrow camera, and create a 3D reconstruction of the vehicle 200's environment. The first processing device can then combine the 3D reconstruction with 3D map data, or combine the 3D reconstruction with 3D information calculated based on information from the other camera.

[0169] The second processing device can receive images from the main camera and perform visual processing to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the second processing device can calculate camera displacement and, based on this displacement, calculate the parallax of pixels between consecutive images and create a 3D reconstruction of the scene (e.g., structure from motion). The second processing device can then send the structure from motion based on the 3D reconstruction to the first processing device for combination with the stereoscopic 3D images.

[0170] The third processing device can receive images from a wide field of view (FOV) camera and process them to detect vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. The third processing device can also execute additional processing instructions to analyze the images to identify moving objects, such as vehicles changing lanes or pedestrians.

[0171] In some embodiments, allowing the image-based information stream to be captured and processed independently can provide opportunities for redundancy in the system. This redundancy may include, for example, using a first image capture device and images processed from that device to verify and / or supplement information obtained by capturing and processing image information from at least a second image capture device.

[0172] In some embodiments, system 100 uses two image capture devices (e.g., image capture devices 122 and 124) to provide navigation assistance to vehicle 200, and uses a third image capture device (e.g., image capture device 126) to provide redundancy and verify the analysis of data received from the other two image capture devices. For example, in this configuration, image capture devices 122 and 124 can provide images for stereo analysis by system 100 to navigate vehicle 200, while image capture device 126 can provide images for monocular analysis by system 100 to provide redundancy and verification of information obtained based on images captured from image capture devices 122 and / or 124. That is, image capture device 126 (and the corresponding processing device) can be considered as providing a redundant subsystem for providing checks on the analysis obtained from image capture devices 122 and 124 (e.g., to provide an automatic emergency braking (AEB) system). Furthermore, in some embodiments, the redundancy and verification of received data may be supplemented based on information received from one or more sensors (e.g., radar, lidar, acoustic sensors, information received from one or more transceivers outside the vehicle, etc.).

[0173] Those skilled in the art will recognize that the camera configurations, camera placements, number of cameras, camera positions, etc., described above are merely examples. These components, and other components described regarding the overall system, can be assembled and used in a variety of different configurations without departing from the scope of the disclosed embodiments. Further details regarding the use of a multi-camera system to provide driver assistance and / or autonomous vehicle functionality are as follows.

[0174] Figure 4 This is an exemplary functional block diagram of memory 140 and / or memory 150 that can store / program instructions for performing one or more operations consistent with the disclosed embodiments. Although memory 140 is referred to below, those skilled in the art will recognize that instructions can be stored in memory 140 and / or memory 150.

[0175] like Figure 4As shown, memory 140 may store monocular image analysis module 402, stereo image analysis module 404, velocity and acceleration module 406, and navigation response module 408. The disclosed embodiments are not limited to any particular configuration of memory 140. Furthermore, application processor 180 and / or image processor 190 may execute instructions stored in any of the modules 402 to 408 included in memory 140. Those skilled in the art will understand that in the following discussion, references to processing unit 110 may individually or uniformly refer to application processor 180 and image processor 190. Therefore, any steps of the following processes may be performed by one or more processing devices.

[0176] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis on a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensing information (e.g., information from radar) to perform monocular image analysis. As combined below... Figures 5A to 5D As described, the monocular image analysis module 402 may include instructions for detecting a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other features associated with the vehicle's environment. Based on this analysis, system 100 (e.g., via processing unit 110) may induce one or more navigation responses in vehicle 200, such as steering, lane changes, changes in acceleration, etc., as discussed below in conjunction with navigation response module 408.

[0177] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis on a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensing information (e.g., information from radar, lidar, etc.) to perform monocular image analysis. As combined below... Figures 5A to 5D As described, the monocular image analysis module 402 may include instructions for detecting a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other features associated with the vehicle's environment. Based on this analysis, system 100 (e.g., via processing unit 110) may induce one or more navigation responses in vehicle 200, such as steering, lane changes, changes in acceleration, etc., as discussed below in conjunction with determining navigation responses.

[0178] In one embodiment, the stereo image analysis module 404 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform stereo image analysis on a first set and a second set of images acquired by a combination of image capture devices selected from any one of image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the first and second sets of images with additional sensing information (e.g., information from radar) to perform stereo image analysis. For example, the stereo image analysis module 404 may include instructions for performing stereo image analysis based on the first set of images acquired by image capture device 124 and the second set of images acquired by image capture device 126. (See below for further details.) Figure 6 As described, the stereo image analysis module 404 may include instructions for detecting a set of features within a first and second set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, etc. Based on this analysis, the processing unit 110 may induce one or more navigation responses in the vehicle 200, such as steering, lane changes, changes in acceleration, etc., as discussed below in conjunction with the navigation response module 408. Furthermore, in some embodiments, the stereo image analysis module 404 may implement techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems.

[0179] In one embodiment, the speed and acceleration module 406 may store software configured to analyze data received from one or more computing and electromechanical devices in the vehicle 200, the computing and electromechanical devices being configured to cause changes in the speed and / or acceleration of the vehicle 200. For example, the processing unit 110 may execute instructions associated with the speed and acceleration module 406 to calculate a target rate of the vehicle 200 based on data obtained from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include, for example, target position, speed and / or acceleration, position and / or speed of the vehicle 200 relative to nearby vehicles, pedestrians, or road objects, position information of the vehicle 200 relative to lane markings on the road, etc. Furthermore, the processing unit 110 may calculate the target rate of the vehicle 200 based on sensor inputs (e.g., information from radar) and inputs from other systems of the vehicle 200 (such as the throttle control system 220, braking system 230, and / or steering system 240 of the vehicle 200). Based on the calculated target rate, the processing unit 110 can transmit electronic signals to the throttle adjustment system 220, braking system 230 and / or steering system 240 of the vehicle 200, for example by physically pressing the brake or releasing the accelerator of the vehicle 200 to trigger changes in speed and / or acceleration.

[0180] In one embodiment, the navigation response module 408 may store software that can be executed by the processing unit 110 to determine the desired navigation response based on data obtained from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. This data may include position and rate information associated with nearby vehicles, pedestrians, and road objects, target position information of vehicle 200, etc. Additionally, in some embodiments, the navigation response may be (partially or entirely) based on map data, the predetermined position of vehicle 200, and / or the relative velocity or relative acceleration between vehicle 200 and one or more objects detected from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. The navigation response module 408 may also determine the desired navigation response based on sensor inputs (e.g., information from radar) and inputs from other systems of vehicle 200 (such as the throttle control system 220, braking system 230, and steering system 240 of vehicle 200). Based on the desired navigation response, the processing unit 110 can transmit electronic signals to the throttle adjustment system 220, braking system 230, and steering system 240 of the vehicle 200 to trigger the desired navigation response, for example, by turning the steering wheel of the vehicle 200 to achieve a predetermined angle of rotation. In some embodiments, the processing unit 110 can use the output of the navigation response module 408 (e.g., the desired navigation response) as input to the execution of the speed and acceleration module 406 to calculate the change in the rate of the vehicle 200.

[0181] Furthermore, any of the modules disclosed herein (e.g., modules 402, 404, and 406) can implement techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems.

[0182] Figure 5A This is a flowchart illustrating an exemplary process 500A for inducing one or more navigation responses based on monocular image analysis, consistent with the disclosed embodiments. In step 510, processing unit 110 may receive multiple images via data interface 128 between processing unit 110 and image acquisition unit 120. For example, a camera included in image acquisition unit 120 (such as image capture device 122 having a field of view 202) may capture multiple images of a region in front of vehicle 200 (e.g., or to the side or rear of the vehicle) and transmit them to processing unit 110 via a data connection (e.g., digital, wired, USB, wireless, Bluetooth, etc.). In step 520, processing unit 110 may execute monocular image analysis module 402 to analyze the multiple images, as described below. Figures 5B to 5D In a further detailed description, by performing this analysis, the processing unit 110 can detect a set of features within the set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, etc.

[0183] In step 520, processing unit 110 may also execute monocular image analysis module 402 to detect various road hazards, such as components of truck tires, fallen road signs, loose cargo, small animals, etc. Road hazards may vary in structure, shape, size, and color, which may make the detection of such hazards more challenging. In some embodiments, processing unit 110 may execute monocular image analysis module 402 to perform multi-frame analysis on the multiple images to detect road hazards. For example, processing unit 110 may estimate camera motion between consecutive image frames and calculate disparity in pixels between frames to construct a 3D map of the road. Processing unit 110 can then use this 3D map to detect road surfaces and hazards present on the road surface.

[0184] In step 530, processing unit 110 may execute navigation response module 408 to perform the analysis performed in step 520 and as described above. Figure 4 The described techniques induce one or more navigation responses. Navigation responses may include, for example, steering, lane changes, acceleration changes, etc. In some embodiments, processing unit 110 may induce one or more navigation responses using data obtained from the execution of speed and acceleration module 406. Furthermore, multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof. For example, processing unit 110 may induce a lane change and then acceleration by sequentially transmitting control signals to the steering system 240 and throttle control system 220 of vehicle 200, for example. Alternatively, processing unit 110 may induce a lane change and then acceleration by simultaneously transmitting control signals to the braking system 230 and steering system 240 of vehicle 200, for example.

[0185] Figure 5B This is a flowchart illustrating an exemplary process 500B for detecting one or more vehicles and / or pedestrians in a set of images, consistent with the disclosed embodiments. Processing unit 110 may implement process 500B by executing monocular image analysis module 402. In step 540, processing unit 110 may determine a set of candidate objects representing possible vehicles and / or pedestrians. For example, processing unit 110 may scan one or more images, compare the images with one or more predetermined patterns, and identify possible locations within each image that may contain objects of interest (e.g., vehicles, pedestrians, or portions thereof). The predetermined patterns may be designed to achieve a high "false hit" rate and a low "miss" rate. For example, processing unit 110 may use a low similarity threshold in the predetermined patterns to identify candidate objects as possible vehicles or pedestrians. Doing so allows processing unit 110 to reduce the likelihood of misses (e.g., failure to identify) candidate objects representing vehicles or pedestrians.

[0186] In step 542, processing unit 110 may filter the group of candidate objects based on classification criteria to exclude certain candidates (e.g., irrelevant or less relevant objects). Such criteria can be derived from various attributes associated with object types stored in a database (e.g., a database stored in memory 140). Attributes may include object shape, size, texture, location (e.g., relative to vehicle 200), etc. Therefore, processing unit 110 may use one or more sets of criteria to reject false candidates from the group of candidate objects.

[0187] In step 544, processing unit 110 may analyze multiple frames of images to determine whether an object in the group of candidate objects represents a vehicle and / or a pedestrian. For example, processing unit 110 may track detected candidate objects across consecutive frames and accumulate frame-by-frame data associated with the detected objects (e.g., size, position relative to vehicle 200, etc.). Furthermore, processing unit 110 may estimate parameters of the detected objects and compare the object's frame-by-frame position data with its predicted position.

[0188] In step 546, processing unit 110 can construct a set of measurements for the detected objects. Such measurements may include, for example, position, velocity, and acceleration values ​​(relative to vehicle 200) associated with the detected objects. In some embodiments, processing unit 110 can construct these measurements based on a series of time-based observational estimation techniques, such as Kalman filters or linear quadratic estimation (LQE), and / or based on modeling data available for different object types (e.g., cars, trucks, pedestrians, bicycles, road signs, etc.). The Kalman filter may be based on a measurement of the object's scale, where the scale measurement is proportional to the time of collision (e.g., the amount of time it takes for vehicle 200 to arrive at the object). Therefore, by performing steps 540 through 546, processing unit 110 can identify vehicles and pedestrians appearing within the set of captured images and obtain information associated with the vehicles and pedestrians (e.g., position, velocity, size). Based on this identification and the obtained information, processing unit 110 can induce one or more navigation responses in vehicle 200, as described above. Figure 5A As described.

[0189] In step 548, processing unit 110 may perform optical flow analysis on one or more images to reduce the likelihood of detecting "false hits" and misses representing candidate objects representing vehicles or pedestrians. Optical flow analysis may refer to, for example, analyzing motion patterns relative to vehicle 200, associated with other vehicles and pedestrians, and distinct from road surface motion in one or more images. Processing unit 110 can calculate the motion of candidate objects by observing different positions of the objects across multiple image frames captured at different times. Processing unit 110 can use position and time values ​​as input to a mathematical model used to calculate the motion of candidate objects. Therefore, optical flow analysis can provide an alternative method for detecting vehicles and pedestrians near vehicle 200. Processing unit 110 may perform optical flow analysis in conjunction with steps 540 to 546 to provide redundancy in detecting vehicles and pedestrians and improve the reliability of system 100.

[0190] Figure 5C This is a flowchart illustrating an exemplary process 500C for detecting road markings and / or lane geometry information in a set of images, consistent with the disclosed embodiments. Processing unit 110 may implement process 500C by executing monocular image analysis module 402. In step 550, processing unit 110 may detect a set of objects by scanning one or more images. To detect segments of lane markings, lane geometry information, and other relevant road markings, processing unit 110 may filter the set of objects to exclude those determined to be irrelevant (e.g., potholes, pebbles, etc.). In step 552, processing unit 110 may group segments belonging to the same road marking or lane marking detected in step 550 together. Based on this grouping, processing unit 110 may develop a model, such as a mathematical model, representing the detected segments.

[0191] In step 554, processing unit 110 may construct a set of measurements associated with the detected segment. In some embodiments, processing unit 110 may create a projection of the detected segment from the image plane to a real-world plane. This projection may be characterized using a cubic polynomial with coefficients corresponding to physical properties such as the detected road's location, slope, curvature, and derivative of curvature. In generating this projection, processing unit 110 may consider variations in the road surface, as well as the pitch and roll rates associated with vehicle 200. Furthermore, processing unit 110 may model the road elevation by analyzing positional and motion cues present on the road surface. Additionally, processing unit 110 may estimate the pitch and roll rates associated with vehicle 200 by tracking a set of feature points in one or more images.

[0192] In step 556, processing unit 110 can perform multi-frame analysis, for example, by tracking detected segments across consecutive image frames and accumulating frame-by-frame data associated with the detected segments. Because processing unit 110 performs multi-frame analysis, the set of measurements constructed in step 554 can become more reliable and associated with increasingly higher confidence levels. Therefore, by performing steps 550 to 556, processing unit 110 can identify road markings appearing in the set of captured images and obtain lane geometry information. Based on this identification and the obtained information, processing unit 110 can induce one or more navigation responses in vehicle 200, as described above. Figure 5A As described.

[0193] In step 558, processing unit 110 may consider additional information sources to further develop a safety model of vehicle 200 in its surrounding environment. Processing unit 110 can use this safety model to define an environment in which system 100 can safely perform autonomous control of vehicle 200. To develop this safety model, in some embodiments, processing unit 110 may consider the positions and movements of other vehicles, detected curbs and guardrails, and / or general road shape descriptions extracted from map data (such as data from map database 160). By considering additional information sources, processing unit 110 can provide redundancy for detecting road markings and lane geometry, and increase the reliability of system 100.

[0194] Figure 5D This is a flowchart illustrating an exemplary process 500D for detecting traffic lights in a set of images, consistent with the disclosed embodiments. Processing unit 110 may implement process 500D by executing monocular image analysis module 402. In step 560, processing unit 110 may scan the set of images and identify objects appearing in the images at locations that may contain traffic lights. For example, processing unit 110 may filter the identified objects to construct a set of candidate objects, excluding those that could not possibly correspond to traffic lights. Filtering may be based on various attributes associated with traffic lights, such as shape, size, texture, location (e.g., relative to vehicle 200), etc. Such attributes may be based on multiple examples of traffic lights and traffic control signals and stored in a database. In some embodiments, processing unit 110 may perform multi-frame analysis on the set of candidate objects reflecting possible traffic lights. For example, processing unit 110 may track candidate objects across consecutive image frames, estimate the real-world location of the candidate objects, and filter out those that are moving (which could not be traffic lights). In some embodiments, processing unit 110 may perform color analysis on the candidate objects and identify the relative position of detected colors appearing within possible traffic lights.

[0195] In step 562, processing unit 110 can analyze the geometry of the intersection. This analysis can be based on any combination of: (i) the number of lanes detected on either side of vehicle 200, (ii) markings detected on the road (such as arrow markings), and (iii) a description of the intersection extracted from map data (e.g., data from map database 160). Processing unit 110 can use information obtained from the execution of monocular analysis module 402 for this analysis. Furthermore, processing unit 110 can determine the correspondence between traffic lights detected in step 560 and lanes appearing near vehicle 200.

[0196] In step 564, as vehicle 200 approaches the intersection, processing unit 110 can update the confidence level associated with the analyzed intersection geometry and detected traffic lights. For example, comparing the number of traffic lights estimated to be present at the intersection with the actual number present at the intersection may affect the confidence level. Therefore, based on this confidence level, processing unit 110 can delegate control to the driver of vehicle 200 to improve safety conditions. By performing steps 560 to 564, processing unit 110 can identify traffic lights appearing in the set of captured images and analyze intersection geometry information. Based on this identification and analysis, processing unit 110 can induce one or more navigation responses in vehicle 200, as described above. Figure 5A As described.

[0197] Figure 5E This is a flowchart illustrating an exemplary process 500E for inducing one or more navigation responses in vehicle 200 based on a vehicle path, consistent with the disclosed embodiments. In step 570, processing unit 110 may construct an initial vehicle path associated with vehicle 200. The vehicle path can be represented using a set of points expressed in coordinates (x, z), and the distance d between any two points in this set is... i The distance can fall within a range of 1 to 5 meters. In one embodiment, processing unit 110 can use two polynomials, such as a left-road polynomial and a right-road polynomial, to construct an initial vehicle path. Processing unit 110 can calculate the geometric midpoint between the two polynomials and offset each point included in the resulting vehicle path by a predetermined offset (e.g., intelligent lane offset), if any (zero offset could correspond to traveling in the middle of the lane). This offset can be in a direction perpendicular to the line segment between any two points in the vehicle path. In another embodiment, processing unit 110 can use a polynomial and an estimated lane width to offset each point in the vehicle path by half the estimated lane width plus a predetermined offset (e.g., intelligent lane offset).

[0198] In step 572, processing unit 110 may update the vehicle path constructed in step 570. Processing unit 110 may use a higher resolution to reconstruct the vehicle path constructed in step 570, such that the distance d between two points in the set representing the vehicle path is... k less than the above distance d i For example, the distance d k It can fall within the range of 0.1 to 0.3 meters. The processing unit 110 can reconstruct the vehicle path using the parabolic spline algorithm, which can generate a cumulative distance vector S corresponding to the total length of the vehicle path (i.e., based on the set of points representing the vehicle path).

[0199] In step 574, processing unit 110 can determine the look-ahead point (expressed in coordinates as (x, y, x) based on the updated vehicle path constructed in step 572. l ,z l Processing unit 110 can extract a forward-looking point from the accumulated distance vector S, and this forward-looking point can be associated with forward-looking distance and forward-looking time. The forward-looking distance, which may have a lower bound ranging from 10 meters to 20 meters, can be calculated as the product of the vehicle 200's speed and the forward-looking time. For example, as the vehicle 200's speed decreases, the forward-looking distance can also decrease (e.g., until it reaches the lower bound). The forward-looking time, ranging from 0.5 to 1.5 seconds, can be inversely proportional to the gain of one or more control loops associated with the navigation response in the vehicle 200, such as the heading error tracking control loop. For example, the gain of this heading error tracking control loop can depend on the bandwidth of the yaw rate loop, steering actuator loop, vehicle lateral dynamics, etc. Therefore, the higher the gain of the heading error tracking control loop, the shorter the forward-looking time.

[0200] In step 576, processing unit 110 can determine the heading error and yaw rate commands based on the foresight point determined in step 574. Processing unit 110 can do this by calculating the arctangent of the foresight point, for example, arctan(x... l / z l The yaw rate command is determined by the product of the heading error and the high-level control gain. If the forward look-ahead distance is not at the lower limit, the high-level control gain can be equal to: (2 / forward look-ahead time). Otherwise, the high-level control gain can be equal to: (2 × vehicle speed 200 / forward look-ahead distance).

[0201] Figure 5FThis is a flowchart illustrating an exemplary process 500F for determining whether a vehicle ahead is changing lanes, consistent with the disclosed embodiments. In step 580, processing unit 110 may determine navigation information associated with the vehicle ahead (e.g., a vehicle traveling in front of vehicle 200). For example, processing unit 110 may use the above combination... Figure 5A and Figure 5B The described technology determines the position, speed (e.g., direction and rate), and / or acceleration of a vehicle ahead. Processing unit 110 can also utilize a combination of the above. Figure 5E The described technique determines one or more road polynomials, forward viewpoints (associated with vehicle 200), and / or a snail trail (e.g., a set of points describing the path taken by the vehicle ahead).

[0202] In step 582, processing unit 110 may analyze the navigation information determined in step 580. In one embodiment, processing unit 110 may calculate the distance between the tracking trajectory and the road polynomial (e.g., along the trajectory). If the variation in this distance along the trajectory exceeds a predetermined threshold (e.g., 0.1 to 0.2 meters on a straight road, 0.3 to 0.4 meters on a moderately curved road, and 0.5 to 0.6 meters on a sharp turn), processing unit 110 may determine that the vehicle ahead is likely changing lanes. In the case of multiple vehicles detected traveling in front of vehicle 200, processing unit 110 may compare the tracking trajectory associated with each vehicle. Based on this comparison, processing unit 110 may determine that a vehicle whose tracking trajectory does not match the tracking trajectories of other vehicles is likely changing lanes. Processing unit 110 may additionally compare the curvature of the tracking trajectory (associated with the vehicle ahead) with the expected curvature of the road segment in which the vehicle ahead is traveling. The desired curvature can be extracted from map data (e.g., data from map database 160), from road polynomials, from tracking trajectories of other vehicles, from existing knowledge about the road, etc. If the difference between the curvature of the tracking trajectory and the desired curvature of the road segment exceeds a predetermined threshold, the processing unit 110 can determine that the vehicle ahead is likely changing lanes.

[0203] In another embodiment, processing unit 110 may compare the instantaneous position of the vehicle ahead with a forward viewpoint (associated with vehicle 200) over a specific time period (e.g., 0.5 to 1.5 seconds). If the distance between the instantaneous position of the vehicle ahead and the forward viewpoint changes during this specific time period, and the cumulative sum of the changes exceeds a predetermined threshold (e.g., 0.3 to 0.4 meters on a straight road, 0.7 to 0.8 meters on a moderately curved road, and 1.3 to 1.7 meters on a sharp turn), then processing unit 110 may determine that the vehicle ahead is likely changing lanes. In another embodiment, processing unit 110 may analyze the geometry of the tracking trajectory by comparing the lateral distance traveled along the trajectory with the desired curvature of the tracking trajectory. The desired radius of curvature may be determined by the following calculation: (δ... z 2 +δ x 2 ) / 2 / (δ x ), where δ x Indicates the lateral distance traveled and δ z The longitudinal distance traveled is represented. If the difference between the lateral distance traveled and the desired curvature exceeds a predetermined threshold (e.g., 500 to 700 meters), the processing unit 110 can determine that the vehicle ahead is likely changing lanes. In another embodiment, the processing unit 110 can analyze the position of the vehicle ahead. If the position of the vehicle ahead obscures the road polynomial (e.g., the vehicle ahead is overlaid on the road polynomial), the processing unit 110 can determine that the vehicle ahead is likely changing lanes. If the position of the vehicle ahead is such that another vehicle is detected in front of the vehicle ahead and the tracking trajectories of the two vehicles are not parallel, the processing unit 110 can determine that the (closer) vehicle ahead is likely changing lanes.

[0204] In step 584, processing unit 110 may determine whether the vehicle 200 ahead is changing lanes based on the analysis performed in step 582. For example, processing unit 110 may make this determination based on a weighted average of the various analyses performed in step 582. In this approach, for example, a decision by processing unit 110 based on a particular type of analysis that the vehicle ahead is likely to change lanes may be assigned a value "1" (and "0" to indicate that the vehicle ahead is unlikely to change lanes). Different analyses performed in step 582 may be assigned different weights, and the disclosed embodiments are not limited to any particular combination of analysis and weights. Furthermore, in some embodiments, the analysis may utilize a trained system (e.g., a machine learning or deep learning system) that may estimate the future path ahead of the vehicle's current position, for example, based on an image captured at the current location.

[0205] Figure 6This is a flowchart illustrating an exemplary process 600 for inducing one or more navigation responses based on stereoscopic image analysis, consistent with the disclosed embodiments. In step 610, processing unit 110 may receive first and second plurality of images via data interface 128. For example, a camera included in image acquisition unit 120 (such as image capture devices 122 and 124 having fields of view 202 and 204) may capture first and second plurality of images of an area in front of vehicle 200 and transmit them to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first and second plurality of images via two or more data interfaces. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0206] In step 620, the processing unit 110 can execute the stereo image analysis module 404 to perform stereo image analysis on the first and second plurality of images to create a 3D map of the road in front of the vehicle and detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, etc. Stereo image analysis can be performed in a manner similar to the above combination. Figures 5A-5D The steps described are performed in a manner consistent with the description. For example, processing unit 110 may execute stereo image analysis module 404 to detect candidate objects (e.g., vehicles, pedestrians, road markings, traffic lights, road hazards, etc.) in the first and second plurality of images, filter out subsets of candidate objects based on various criteria, and perform multi-frame analysis, construct measurements, and determine confidence levels for the remaining candidate objects. In performing the above steps, processing unit 110 may consider information from both the first and second plurality of images, rather than information from a single set of images. For example, processing unit 110 may analyze differences in pixel-level data (or other subsets of data from the two streams of captured images) of candidate objects appearing in both the first and second plurality of images. As another example, processing unit 110 may estimate the position and / or velocity of a candidate object (e.g., relative to vehicle 200) by observing whether an object appears in one of the plurality of images but not in another, or by other differences that may exist relative to objects appearing in the two image streams. For example, the position, velocity, and / or acceleration relative to vehicle 200 may be determined based on the trajectory, position, movement characteristics, etc., of features associated with an object appearing in one or both of the image streams.

[0207] In step 630, processing unit 110 may execute navigation response module 408 to perform the analysis performed in step 620 and as described above. Figure 4The described technology induces one or more navigation responses in vehicle 200. Navigation responses may include, for example, steering, lane changing, acceleration change, speed change, braking, etc. In some embodiments, processing unit 110 may use data obtained from the execution of speed and acceleration module 406 to induce the one or more navigation responses. Furthermore, multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof.

[0208] Figure 7 This is a flowchart illustrating an exemplary process 700 consistent with the disclosed embodiments for evoking one or more navigation responses based on the analysis of three sets of images. In step 710, processing unit 110 may receive first, second, and third plurality of images via data interface 128. For example, cameras included in image acquisition unit 120 (such as image capture devices 122, 124, and 126 having fields of view 202, 204, and 206) may capture first, second, and third plurality of images of areas in front of and / or to the sides of vehicle 200 and transmit them to processing unit 110 via digital connections (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive first, second, and third plurality of images via three or more data interfaces. For example, each of image capture devices 122, 124, and 126 may have an associated data interface for transmitting data to processing unit 110. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0209] In step 720, the processing unit 110 can analyze the first, second, and third images to detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and road hazards. This analysis can be performed in a manner similar to the combination described above. Figures 5A-5D and Figure 6 The steps described are performed in a manner that allows for execution. For example, processing unit 110 may perform monocular image analysis on each of the first, second, and third plurality of images (e.g., via execution by monocular image analysis module 402 and based on the above combination). Figures 5A-5D The steps described above). Alternatively, processing unit 110 may perform stereoscopic image analysis on the first and second plurality of images, the second and third plurality of images, and / or the first and third plurality of images (e.g., via execution by stereoscopic image analysis module 404 and based on the above combination). Figure 6(The steps described). Processed information corresponding to the analysis of the first, second, and / or third plurality of images can be combined. In some embodiments, processing unit 110 can perform a combination of monocular and stereoscopic image analysis. For example, processing unit 110 can perform monocular image analysis on the first plurality of images (e.g., via execution of monocular image analysis module 402) and perform stereoscopic image analysis on the second and third plurality of images (e.g., via execution of stereoscopic image analysis module 404). The configuration of image capture devices 122, 124, and 126—including their respective positions and fields of view 202, 204, and 206—can affect the type of analysis performed on the first, second, and third plurality of images. The disclosed embodiments are not limited to a specific configuration of image capture devices 122, 124, and 126 or the type of analysis performed on the first, second, and third plurality of images.

[0210] In some embodiments, processing unit 110 may perform tests on system 100 based on the images acquired and analyzed in steps 710 and 720. Such tests can provide an indicator of the overall performance of system 100 for certain configurations of image acquisition devices 122, 124, and 126. For example, processing unit 110 may determine the ratio of "false hits" (e.g., situations where system 100 incorrectly determines the presence of vehicles or pedestrians) to "misses".

[0211] In step 730, processing unit 110 may induce one or more navigation responses in vehicle 200 based on information obtained from two of the first, second, and third plurality of images. The selection of two of the first, second, and third plurality of images may depend on various factors, such as, for example, the number, type, and size of objects detected in each of the plurality of images. Processing unit 110 may also make the selection based on image quality and resolution, the effective field of view reflected in the image, the number of captured frames, and the degree to which one or more objects of interest actually appear in the frames (e.g., the percentage of frames in which objects appear, the proportion of objects appearing in each such frame), etc.

[0212] In some embodiments, processing unit 110 can select two pieces of information obtained from a first, second, and third plurality of images by determining the degree of consistency between information obtained from one image source and information obtained from other image sources. For example, processing unit 110 can combine processed information obtained from each of image capture devices 122, 124, and 126 (whether through monocular analysis, stereo analysis, or any combination of both) and determine consistent visual indicators (e.g., lane markings, detected vehicles and their positions and / or paths, detected traffic lights, etc.) across each captured image from image capture devices 122, 124, and 126. Processing unit 110 can also exclude inconsistent information between the captured images (e.g., vehicles changing lanes, lane models indicating that a vehicle is too close to vehicle 200, etc.). Therefore, processing unit 110 can select two pieces of information obtained from the first, second, and third plurality of images based on the determination of consistent and inconsistent information.

[0213] Navigation responses may include, for example, steering, lane changes, and changes in acceleration. Processing unit 110 may base its responses on the analysis performed in step 720 and the above-described combination. Figure 4 The described techniques induce one or more navigation responses. Processing unit 110 may also induce one or more navigation responses using data obtained from the execution of velocity and acceleration module 406. In some embodiments, processing unit 110 may induce one or more navigation responses based on the relative position, relative velocity, and / or relative acceleration between the vehicle 200 and objects detected in any of the first, second, and third plurality of images. The plurality of navigation responses may occur simultaneously, sequentially, or in any combination thereof.

[0214] Reinforcement learning and trained navigation systems

[0215] The following sections discuss autonomous driving and the systems and methods used to achieve autonomous control of a vehicle, whether that control is fully autonomous (self-driving vehicle) or partially autonomous (e.g., one or more driver assistance systems or functions). Figure 8As shown, the autonomous driving task can be divided into three main modules, including a sensing module 801, a driving strategy module 803, and a control module 805. In some embodiments, modules 801, 803, and 805 may be stored in memory units 140 and / or 150 of system 100, or modules 801, 803, and 805 (or portions thereof) may be stored remotely from system 100 (e.g., in a server accessible to system 100 via, for example, wireless transceiver 172). Furthermore, any module disclosed herein (e.g., modules 801, 803, and 805) may implement techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems.

[0216] The sensing module 801, implemented using processing unit 110, can handle various tasks related to sensing the navigation state in the environment of the host vehicle. Such tasks may rely on inputs from various sensors and sensing systems associated with the host vehicle. These inputs may include images or image streams from one or more onboard cameras, GPS location information, accelerometer output, user feedback or user input to one or more user interface devices, radar, lidar, etc. Sensing data, including data from cameras and / or any other available sensors, as well as map information, can be collected, analyzed, and formulated into a “sensed state” that describes information extracted from the scene in the host vehicle’s environment. In addition to any other potentially sensed information, the sensed state may include sensed information related to target vehicles, lane markings, pedestrians, traffic lights, road geometry, lane shape, obstacles, distance to other objects / vehicles, relative speed, and relative acceleration. Supervised machine learning can be implemented to generate a sensed state output based on the sensed data provided to the sensing module 801. The output of the sensing module can represent the host vehicle’s sensed navigation “state,” which can be passed to the driving strategy module 803.

[0217] While a sensed state can be developed based on image data received from one or more cameras or image sensors associated with the host vehicle, any suitable sensor or combination of sensors can also be used to develop a sensed state used in navigation. In some embodiments, a sensed state can be developed without relying on captured image data. In fact, any navigation principles described herein can be applied to sensed states developed based on captured image data as well as sensed states developed using other non-image-based sensors. The sensed state can also be determined via sources external to the host vehicle. For example, a sensed state can be developed, in whole or in part, based on information received from sources remote from the host vehicle (e.g., sensor information, processed state information, etc., shared from other vehicles, shared from a central server, or from any other information source relating to the navigation state of the host vehicle).

[0218] Driving strategy module 803 (discussed in more detail below and implemented using processing unit 110) can implement a desired driving strategy to determine one or more navigation actions to be taken by the primary vehicle in response to sensed navigation state. If no other agent (e.g., target vehicle or pedestrian) is present in the primary vehicle's environment, the sensed state input to driving strategy module 803 can be processed in a relatively straightforward manner. The task becomes more complex when the sensed state requires negotiation with one or more other agents. Techniques used to generate the output of driving strategy module 803 can include reinforcement learning (discussed in more detail below). Among other potential desired navigation actions, the output of driving strategy module 803 can include at least one navigation action of the primary vehicle and may include desired acceleration (which can be translated into updated velocity of the primary vehicle), desired yaw rate of the primary vehicle, and desired trajectory.

[0219] Based on the output from the driving strategy module 803, the control module 805, which can also be implemented using the processing unit 110, can develop control commands for one or more actuators or controlled devices associated with the host vehicle. Such actuators and devices may include accelerators, one or more steering controls, brakes, signal transmitters, displays, or any other actuators or devices that can be controlled as part of navigation operations associated with the host vehicle. Aspects of control theory can be used to generate the output of the control module 805. The control module 805 can be responsible for developing and outputting commands to the controllable components of the host vehicle to implement the desired navigation goals or requirements of the driving strategy module 803.

[0220] Returning to the driving strategy module 803, in some embodiments, the driving strategy module 803 can be implemented using a trained system trained through reinforcement learning. In other embodiments, the driving strategy module 803 can be implemented "manually" by using specified algorithms to resolve various scenarios that may arise during autonomous navigation, without machine learning methods. However, while this approach is feasible, it may lead to an oversimplified driving strategy and may lack the flexibility of a machine learning-based trained system. For example, a trained system may be better equipped to handle complex navigation states and can better determine whether a taxi is parked or stopped to pick up or drop off passengers; determine whether a pedestrian intends to cross the street in front of the main vehicle; defensively balance the unexpected behavior of other drivers; negotiate in dense traffic involving target vehicles and / or pedestrians; decide when to suspend certain navigation rules or augment others; anticipate unsensed but anticipated situations (e.g., whether a pedestrian will appear from behind a car or obstacle); and so on. A reinforcement learning-based trained system is also better equipped to handle continuous and high-dimensional state spaces and continuous action spaces.

[0221] Training a system using reinforcement learning can involve learning a driving policy that maps sensed states to navigation actions. The driving policy is a function π: S → A, where S is a set of states, and... This is the action space (e.g., desired speed, acceleration, yaw commands, etc.). The state space is S = S x xS p S s It is the sensing state, and S p This is additional information about the state saved through this strategy. Working at discrete time intervals, at time t, the current state s can be observed. t ∈S, and this strategy can be applied to obtain the desired action a. t =π(s) t ).

[0222] The system can be trained by exposing it to various navigation states, enabling it to apply the policy, and providing rewards (based on a reward function designed to reward desired navigational behavior). Based on this reward feedback, the system can "learn" the policy and become trained in generating desired navigational actions. For example, the learning system can observe the current state s. t ∈S, and based on policy To determine action a t ∈A. Based on the determined action (and the execution of that action), the environment moves to the next state s. t+1∈S, to be observed by the learning system. For each action developed in response to the observed state, the feedback to the learning system is a reward signal r1, r2, ...

[0223] The goal of reinforcement learning (RL) is to find a policy π. It is typically assumed that at time t, there exists a reward function r. t Its measurement is in state S i And take action a t The instantaneous mass. However, taking action a at time t. t The environment is affected, and therefore the values ​​of future states are influenced. Therefore, when deciding which action to take, not only current rewards but also future rewards should be considered. In some cases, when the system determines that taking a lower-reward option now may result in a higher reward in the future, the system should take that action even if it is associated with a lower reward than another available option. To formalize this, we observe that the policy π and the initial state s are inductively derived in... The distribution on, where if the agent starts at state s0 = s and follows policy π from there, then the vector (r1, ..., r2) is... T The probability of observing a reward is r1, ..., r. T The probability of . The value of the initial state s can be defined as:

[0224]

[0225] Instead of limiting the time range to T, we can define it by discounting future rewards for some fixed γ∈(0,1):

[0226]

[0227] In any case, the optimal strategy is the solution to the following equation:

[0228]

[0229] Here, expectation is based on the initial state s.

[0230] There are several possible methods for training driving strategy systems. For example, imitation methods (e.g., behavioral cloning) can be used, in which the system learns from state / action pairs, where actions are those actions a good agent (e.g., a human) would choose in response to a particular observed state. Suppose we observe a human driver. Through this observation, the form (s) t a t Many examples of ) (where s t It is a state, and a tThe actions of a human driver can be acquired, observed, and used as the basis for training driving strategy systems. For example, supervised learning can be used to learn policy π, such that π(s) t )≈a t This method has many potential advantages. First, it does not require defining a reward function. Second, the learning is supervised and occurs offline (no agent needs to be applied during the learning process). The disadvantage of this method is that different human drivers, and even the same human driver, are not deterministic in their policy choices. Therefore, learning for ||π(s) t )-a t Very small functions are generally not feasible. Moreover, even small errors can accumulate over time to produce large errors.

[0231] Another approach is policy-based learning. Here, the policy can be expressed parametrically, and a suitable optimization technique (e.g., stochastic gradient descent) can be used to directly optimize the policy. This method directly optimizes the policy... The given problem is solved. Of course, there are many other methods to solve this problem. One advantage of this method is that it directly solves the problem, thus often leading to good practical results. A potential disadvantage is that it often requires "on-policy" training, meaning that the learning of π is an iterative process, where the policy π has an imperfect form during iteration j. j And in order to construct the next strategy π j It must be based on π j It interacts with the environment while performing actions.

[0232] The system can also be trained through value-based learning (learning the Q or V function). Assume that it can learn the optimal value function V. * A good approximation of the value function. Optimal policies can be constructed (e.g., relying on the Bellman equation). Some versions of value-based learning can be implemented offline (called "offline policy" training). Some drawbacks of value-based methods may be due to their strong reliance on Markovian assumptions and the need to approximate complex functions (approximating value functions can be more difficult than directly approximating the policy).

[0233] Another technique may include model-based learning and planning (learning the probabilities of state transitions and solving an optimization problem to find the optimal V). Combinations of these techniques can also be used to train learning systems. In this approach, the dynamics of the process can be learned, i.e., the actions taken (s... t a t ) and in the next state s t+1This is a function that generates a distribution. Once this function is learned, an optimal problem can be solved to find the policy π whose value is optimal. This is called "planning". One advantage of this method may be that the learning part is supervised and can be learned by observing triples (s t a t s t+1 This approach is similar to the "imitation" method, but one drawback is that small errors during the learning process can accumulate and result in under-executed strategies.

[0234] Another approach for training the driving policy module 803 may include decomposing the driving policy function into semantically meaningful components. This allows certain parts of the policy to be implemented manually, ensuring the policy's safety; and reinforcement learning techniques can be used to implement other parts of the policy, enabling adaptation to many scenarios, a human-like balance between defensive / aggressive behavior, and human-like negotiation with other drivers. Technically, reinforcement learning methods can combine several approaches and provide easy-to-operate training procedures, where most training can be performed using recorded data or a self-built simulator.

[0235] In some embodiments, the training of the driving strategy module 803 may rely on an "options" mechanism. For illustration, consider a simple scenario of driving strategy on a two-lane highway. In the direct RL method, policy π maps the state to... In this context, the first component of π(s) is the desired acceleration command, and the second component of π(s) is the yaw rate. In the modified method, the following strategy can be constructed:

[0236] Automatic Cruise Control (ACC) strategy, o ACC S→A: This strategy always outputs a yaw rate of 0 and only changes the speed to achieve smooth and accident-free driving.

[0237] ACC + Left Strategy, o L S→A: The longitudinal command for this strategy is the same as the ACC command. Yaw rate is a direct implementation of centering the vehicle toward the center of the left lane while ensuring safe lateral movement (e.g., not moving to the left if there is a vehicle on the left).

[0238] ACC + right strategy, o R S→A: with o L The same applies, but vehicles can center themselves in the middle of the right lane.

[0239] These strategies can be called "options". Using these "options", we can learn the strategy π that has chosen those options. oS→O, where O is a set of available options. In one case, O={o ACC o L o R Option selector strategy π o By setting each s Let's define the actual strategy π: S→A.

[0240] In practice, the policy function can be decomposed into option graph 901, such as... Figure 9 As shown. Figure 10 Another example, option graph 1000, is shown. An option graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There exists a special node called the root node 903 of the graph. This node has no incoming nodes. The decision-making process traverses the entire graph from the root node until it reaches a "leaf" node, which is a node with no outgoing decision lines. Figure 9 As shown, leaf nodes may include, for example, nodes 905, 907, and 909. Upon encountering a leaf node, the driving strategy module 803 may output acceleration and steering commands associated with a desired navigation action, which is associated with the leaf node.

[0241] Internal nodes (such as nodes 911, 913, and 915) can, for example, lead to the implementation of a policy that selects children from its available options. The set of available children for an internal node includes all nodes associated with a particular internal node via a decision line. For example, Figure 9 The internal node 913 designated as "merge" includes three child nodes 909, 915 and 917, each connected to node 913 via a decision line ("hold", "overtake on the right" and "overtake on the left" respectively).

[0242] Flexibility in decision-making systems can be achieved by enabling nodes to adjust their position within the hierarchy of the option graph. For example, any node could be allowed to declare itself "critical." Each node could implement an "is critical" function, which outputs "true" if the node is in a critical part of its policy implementation. For instance, a node responsible for overtaking could declare itself critical midway through a maneuver. This imposes a constraint on the set of available offspring of node u, which can include all nodes v that are children of node u, and that there exists a path from v to a leaf node passing through all nodes designated as critical. On the one hand, this approach allows desired paths to be declared on the graph at each time step; on the other hand, it preserves policy stability, particularly when critical parts of the policy are being implemented.

[0243] By defining an option graph, the problem of learning the driving policy π: S→A can be decomposed into defining a policy for each node in the graph, where the policy at an internal node should be selected from its available child nodes. For some nodes, the corresponding policy can be implemented manually (e.g., by specifying a set of actions in response to an observed state using an if-then type algorithm), while for others it can be implemented using a trained system built through reinforcement learning. The choice between manual or trained / learned methods can depend on the safety aspects associated with the task and on its relative simplicity. The option graph can be constructed in a way that allows some nodes to implement directly, while others can rely on a trained model. This approach ensures safe operation of the system.

[0244] The following discussion provides information regarding the driving strategy module 803, such as... Figure 9 Further details regarding the role of the option graph in the above description. As mentioned above, the input to the driving strategy module is the "sensed state," which summarizes the environment map, for example, obtained from available sensors. The output of the driving strategy module 803 is a set of expectations (optionally along with a set of hard constraints) that define the trajectory as a solution to the optimization problem.

[0245] As described above, the option graph represents a hierarchical set of decisions organized as a Directed Acyclic Graph (DAG). There exists a special node called the "root" of the graph. The root node is the only node without any incoming edges (e.g., decision lines). The decision-making process traverses the graph from the root node until it reaches a "leaf" node, which is a node without any outgoing edges. Each internal node should implement a strategy of selecting offspring from its available offspring. Each leaf node should implement a strategy based on a set of expectations (e.g., a set of navigation goals for the main vehicle) defined by the entire path from the root to the leaf. This set of expectations, directly defined based on the sensed state, along with a set of hard constraints, establishes an optimization problem whose solution is the vehicle's trajectory. Hard constraints can be employed to further enhance system safety, and the expectations can be used to provide driving comfort and human-like driving behavior. The trajectory, provided as a solution to the optimization problem, further defines the commands that should be provided to the steering, braking, and / or engine actuators to complete that trajectory.

[0246] return Figure 9Option diagram 901 represents the option diagram for a two-lane highway, including merging lanes (meaning that at some point, a third lane merges into the right or left lane of the highway). Root node 903 first determines whether the primary vehicle is in a normal road scenario or approaching a merging scenario. This is an example of a decision that can be made based on the sensed state. Normal road node 911 includes three child nodes: hold node 909, left overtaking node 917, and right overtaking node 915. Hold refers to the situation where the primary vehicle wants to continue traveling in the same lane. The hold node is a leaf node (without an outgoing edge / line). Therefore, the hold node defines a set of expectations. The first expectation it defines may include the desired lateral position—e.g., as close as possible to the center of the current lane. Smooth navigation may also be desired (e.g., within a predetermined or allowed maximum acceleration). The hold node may also define how the primary vehicle reacts to other vehicles. For example, the hold node may look at sensed target vehicles and assign a semantic meaning to each vehicle, which can be translated into components of the trajectory.

[0247] Various semantic meanings can be assigned to target vehicles in the environment of the master vehicle. For example, in some embodiments, semantic meanings may include any of the following designations: 1) Irrelevant: Indicates that the sensed vehicle in the scene is currently irrelevant; 2) Next Lane: Indicates that the sensed vehicle is in the adjacent lane and that an appropriate offset should be maintained relative to that vehicle (the precise offset can be computed in an optimization problem that constructs a trajectory given expectations and hard constraints, and the precise offset may potentially depend on the semantic type of the target vehicle set by the hold-alive leaf of the vehicle's option graph, which defines the expectations relative to the target vehicle); 3) Give Way: The master vehicle will attempt to give way to the sensed target vehicle by, for example, reducing speed (especially if the master vehicle determines that the target vehicle may cut into the master vehicle's lane); 4) Takeway: The master vehicle will attempt to take over the right-of-way by, for example, increasing speed; 5) Follow: The master vehicle expects to follow the target vehicle and maintain smooth driving; 6) Left / Right Overtake: This means that the master vehicle wants to initiate a lane change to the left or right lane. Left Overtake node 917 and Left Overtake node 915 are internal nodes whose expectations are not yet defined.

[0248] The next node in option diagram 901 is the gap selection node 919. This node is responsible for selecting the gap between two target vehicles in a specific target lane that the master vehicle wishes to enter. By selecting a node of the form IDj, for some value of j, the master vehicle arrives at a leaf that specifies the desired outcome for the trajectory optimization problem—for example, the master vehicle wishes to maneuver to reach the selected gap. This maneuver might involve first accelerating / braking in the current lane and then moving to the target lane at an appropriate time to enter the selected gap. If the gap selection node 919 cannot find a suitable gap, it moves to the abort node 921, which defines the expectation of returning to the center of the current lane and canceling the overtaking maneuver.

[0249] Returning to lane merging node 913, when the primary vehicle approaches to merge, it has several options that may vary depending on the specific circumstances. For example, such as... Figure 11A As shown, the main vehicle 1105 is traveling along a two-lane road where no other target vehicle is detected in the main lane or merging lane 1111 of the two-lane road. In this case, the driving strategy module 803 can choose to remain in the lane 909 when it reaches the merging node 913. That is, it is expected to remain in its current lane if no target vehicle is sensed merging into the roadway.

[0250] exist Figure 11B In the latter case, the situation is slightly different. Here, the main vehicle 1105 senses one or more target vehicles 1107 entering the main lane 1112 from the merging lane 1111. In this case, once the driving strategy module 803 encounters the merging node 913, it can choose to initiate a left overtaking maneuver to avoid the merging situation.

[0251] exist Figure 11C In this scenario, the primary vehicle 1105 encounters one or more target vehicles 1107 entering the main lane 1112 from the merging lane 1111. The primary vehicle 1105 also detects a target vehicle 1109 traveling in a lane adjacent to its own. The primary vehicle also detects one or more target vehicles 1110 traveling in the same lane as the primary vehicle 1105. In this situation, the driving strategy module 803 can decide to adjust the speed of the primary vehicle 1105 to give way to target vehicle 1107 and proceed ahead of target vehicle 1115. This can be achieved, for example, by proceeding to a gap selection node 919, which then selects the gap between ID0 (vehicle 1107) and ID1 (vehicle 1115) as the appropriate merging gap. In this case, the appropriate gap for the merging situation defines the objective of the trajectory planner optimization problem.

[0252] As described above, nodes in the option graph can declare themselves as "critical," ensuring that the selected option passes through critical nodes. Formally, each node can implement the IsCritical function. After performing a forward pass from root to leaf on the option graph and solving the optimization problem for the trajectory planner, a backward pass can be performed from leaf back to root. Along this backward pass, the IsCritical function of all nodes in the pass can be called, and a list of all critical nodes can be maintained. In the forward path corresponding to the next time frame, the driving strategy module 803 may need to select a path from the root node to the leaf that passes through all critical nodes.

[0253] Figures 11A to 11C This can be used to demonstrate the potential benefits of this method. For example, when initiating an overtaking maneuver and the driving strategy module 803 reaches the leaf corresponding to IDk, such as when the master vehicle is in the middle of an overtaking maneuver, selecting the hold node 909 would be undesirable. To avoid such a jump, the IDj node can designate itself as critical. During the maneuver, the success of the trajectory planner can be monitored, and if the overtaking maneuver proceeds as expected, the IsCritical function will return a "true" value. This method ensures that the overtaking maneuver will continue in the next time frame (instead of jumping to another potentially inconsistent maneuver before completing the initially selected maneuver). On the other hand, if the monitoring of the maneuver indicates that the selected maneuver does not proceed as expected, or if the maneuver becomes unnecessary or impossible, the IsCritical function can return a "false" value. This allows the gap selection node to select a different gap in the next time frame, or to abort the overtaking maneuver entirely. This method, on the one hand, allows the desired path to be declared on the option graph at each time step, while on the other hand, it can help improve the stability of the strategy during the execution of critical parts.

[0254] Hard constraints can differ from navigation expectations, as will be discussed in more detail below. For example, hard constraints can ensure safe driving by applying an additional filtering layer to planned navigation actions. The hard constraints involved can be determined based on sensed states and can be manually programmed and defined rather than using a trained system built on reinforcement learning. However, in some embodiments, the trained system can learn applicable hard constraints to apply and follow. This approach can prompt the driving strategy module 803 to reach a selected action that already conforms to the applicable hard constraints, which can reduce or eliminate the need for subsequent modifications to conform to the applicable hard constraints. Nevertheless, as a redundancy safety measure, hard constraints can be applied to the output of the driving strategy module 803 even if it has been trained to account for predetermined hard constraints.

[0255] There are many examples of potential hard constraints. For instance, a hard constraint can be defined in conjunction with a guardrail at the edge of a road. Under no circumstances should a primary vehicle be allowed to pass over the guardrail. This rule creates a hard lateral constraint on the trajectory of the primary vehicle. Another example of a hard constraint can include road bumps (e.g., rate-controlled bumps), which can impose hard constraints on driving speed both before and while passing over the bump. Hard constraints can be considered safety-critical and therefore can be defined manually rather than solely relying on a trained system that learns the constraints during training.

[0256] Compared to hard constraints, the desired outcome can be to achieve or attain comfortable driving. As discussed above, examples of desired outcomes could include the objective of positioning the primary vehicle in a lateral position within the lane corresponding to the center of the primary vehicle's lane. Another desired outcome might include the ID of a suitable entry gap. Note that the primary vehicle does not need to be precisely centered in the lane; rather, the desired outcome of being as close to it as possible ensures that even if it deviates from the lane center, the primary vehicle tends to migrate to the center of the lane. Desired outcomes may not be safety-critical. In some embodiments, desired outcomes may need to be negotiated with other drivers and pedestrians. One approach to constructing desired outcomes could rely on an option graph, and the policies implemented at at least some nodes of the graph could be based on reinforcement learning.

[0257] For nodes in option graph 901 or 1000 implemented as learning-based training nodes, the training process may include decomposing the problem into a supervised learning phase and a reinforcement learning phase. In the supervised learning phase, the node can learn from... to (s t a t Differentiable mappings of ) make This can be similar to "model-based" reinforcement learning. However, in the forward loop of the network, s can be used t+1 The actual value is used to replace This eliminates the problem of error accumulation. The role of prediction is to propagate messages from the future back to past actions. In this sense, the algorithm can be considered a combination of "model-based" reinforcement learning and "policy-based learning".

[0258] In certain scenarios, an important element that can be provided is a differentiable path from future loss / reward back to the decision about the action. Using an option graph structure, the implementation of options involving safety constraints is often non-differentiable. To overcome this problem, the selection of offspring in the learned policy nodes can be randomized. That is, a node can output a probability vector p, which assigns a probability to each offspring selected for a particular node. Assume a node has k offspring, and let a... (1) , ..., a (k)For the action of the path from each child to the leaf, the resulting predicted action is therefore... This could lead to a differentiable path from the action to p. In practice, action a can be chosen as a for i ~ p. (i) And a and The difference between them can be called additive noise.

[0259] For a given s t a t Down Training can combine supervised learning with real-world data. For training node policies, a simulator can be used. Afterward, fine-tuning of the policy can be done using real data. Two concepts can make the simulation more realistic. First, using imitation, the "behavioral cloning" paradigm can be used to build an initial policy using a large real-world dataset. In some cases, the resulting agent may be suitable. In other cases, the resulting agent at least forms a very good initial policy for other agents on the road. Second, using self-play, our own policy can be used to reinforce training. For example, given an initial implementation by another agent (car / pedestrian) that may be experienced, the policy can be trained based on a simulator. Some other agents can be replaced by the new policy, and the process can be repeated. Therefore, the policy can continue to improve because it should respond to a wider variety of other agents with different levels of complexity.

[0260] Furthermore, in some embodiments, the system may implement a multi-agent approach. For example, the system may consider data from various sources and / or images captured from multiple angles. Additionally, some disclosed embodiments may provide energy economy because the anticipation of events that do not directly involve the primary vehicle but may affect it, or even the anticipation of events that may lead to unpredictable situations involving other vehicles, can be taken into consideration (e.g., the radar may "see through" vehicles ahead and the unavoidable anticipation, or even the high probability of events that will affect the primary vehicle).

[0261] Trained system with imposed navigation constraints

[0262] In autonomous driving scenarios, a key concern is ensuring the safety of the learned strategies of the trained navigation network. In some embodiments, constraints can be used to train the driving strategy system so that the actions selected by the trained system take into account applicable safety constraints. Additionally, in some embodiments, an extra layer of safety can be provided by passing the selected actions of the trained system through one or more hard constraints related to a specific sensed scene in the host vehicle's environment. This approach ensures that actions taken by the host vehicle are limited to those determined to satisfy applicable safety constraints.

[0263] At its core, the navigation system may include a learning algorithm based on a policy function that maps observed states to one or more desired actions. In some implementations, the learning algorithm is a deep learning algorithm. Desired actions may include at least one action expected to maximize the vehicle's anticipated reward. While in some cases the actual action taken by the vehicle may correspond to one of the desired actions, in others the actual action may be determined based on the observed state, one or more desired actions, and non-learning hard constraints (e.g., safety constraints) imposed on the learning navigation engine. These constraints may include a driverless zone around various types of detected objects (e.g., target vehicles, pedestrians, stationary objects on the roadside or in the carriageway, moving objects on the roadside or in the carriageway, guardrails, etc.). In some cases, the size of this zone may vary based on the detected motion (e.g., speed and / or direction) of the detected objects. Other constraints may include maximum speed when crossing within a pedestrian's influence zone, maximum deceleration (to account for the distance between the target vehicle and the main vehicle), mandatory stopping at sensed crosswalks or railway crossings, etc.

[0264] Hard constraints, when used in conjunction with systems trained through machine learning, can provide a level of safety in autonomous driving that exceeds what can be achieved solely based on the output of a trained system. For example, a desired set of constraints can be used as training guidance to train a machine learning system, allowing it to select actions in response to sensed navigation states that take into account and comply with applicable navigation constraints. However, trained systems retain some flexibility in selecting navigation actions, and therefore, there are at least some cases where actions selected by the trained system may not strictly adhere to the relevant navigation constraints. Therefore, to require selected actions to strictly comply with the relevant navigation constraints, non-machine learning components, outside the learning / training framework, that guarantee strict application of the relevant navigation constraints, can be used to combine, compare, filter, adjust, and modify the output of the trained system.

[0265] The following discussion provides additional details about trained systems and the potential benefits (particularly from a security perspective) of combining trained systems with algorithmic components outside the trained / learning framework. As previously mentioned, reinforcement learning objectives for policies can be optimized via stochastic gradient ascent. The objective (e.g., expected reward) can be defined as...

[0266] Objectives involving expectations can be used in machine learning scenarios. However, such objectives, when not constrained by navigational constraints, may not return actions strictly bound by those constraints. For example, consider a reward function, where... Trajectories used to represent rare "corner" events (e.g., accidents) to be avoided, while For the remaining trajectories, one goal of the learning system could be to learn to perform overtaking maneuvers. Typically, on accident-free trajectories, Successful, smooth overtaking will be rewarded, while failing to overtake while remaining in the lane will be penalized—therefore, the range is [-1, 1]. If the sequence... If the incident represents an accident, then the reward -r should provide a sufficiently high penalty to prevent it from happening. The question is, what should the value of r be to ensure accident-free driving?

[0267] It was observed that the accident had the following impact: The effect is an additional term -pr, where p is the probabilistic quality of the trajectory with the accident event. If this term is negligible, i.e., p << 1 / r, then the learning system can more often preferentially execute the accident-prone strategy (or typically adopt a reckless driving strategy) to successfully perform overtaking maneuvers, compared to a more defensive strategy that comes at the cost of some overtaking maneuvers not being successfully completed. In other words, if the probability of an accident is at most p, then r must be set such that r >> 1 / p. It can be expected that p is minimized (e.g., at p = 10). -9 (on the order of magnitude of). Therefore, r should be large. In the policy gradient, it can be estimated that... The gradient of the random variable. The following lemma shows that the gradient of the random variable. The variance of pr 2 As the variance increases, it becomes greater than r for the condition r >> 1 / p. Therefore, estimating the objective may be difficult, and estimating its gradient may be even more difficult.

[0268] Lemma: Let π o Let p and r be scalars, such that we obtain with probability p. And obtain with probability 1-p So,

[0269]

[0270] The final approximation applies to the case where r ≥ 1 / p.

[0271] This discussion indicates that the form is The goal may not ensure functional safety without introducing variance problems. Baseline subtraction methods used for variance reduction may not provide sufficient remedy for this problem, as it will stem from... The high variance of the baseline constant is transferred to the same high variance, and the estimate of the baseline constant will also be affected by numerical instability. Furthermore, if the probability of an accident is p, then at least 1 / p sequences should be sampled on average before obtaining the accident event. This means that the methods used to achieve this... The lower bound of the learning algorithm's sequence of 1 / p samples is minimized. The solution to this problem can be found in the architectural design described in this paper, rather than through numerical tuning techniques. The approach here is based on the idea that hard constraints should be injected outside the learning framework. In other words, the policy function can be decomposed into learnable and non-learnable parts. Formally, the policy function can be constructed as follows: in Mapping the (unknown) state space to a set of expectations (e.g., expected navigation goals, etc.), while π (T) This expectation is mapped to a trajectory (which determines how the car should move over short distances). Function It is responsible for driving comfort and making strategic decisions, such as which other vehicles should be overtaken or yielded to, and what the desired position of the main vehicle is within its lane. The mapping from the sensed navigation state to the desired outcome is the strategy. It can learn from experience by maximizing expected rewards. The resulting expectation can be transformed into a cost function for the driving trajectory. The function π (T) It is not a learned function, but rather a path that minimizes the cost of being subject to hard constraints in terms of functional safety. This decomposition ensures functional safety while providing a comfortable driving experience.

[0272] like Figure 11D As shown, a dual lane merging navigation scenario provides a further example illustrating these concepts. In dual lane merging, vehicles approach merging area 1130 from both the left and right sides. Vehicles from each side (such as vehicle 1133 or vehicle 1135) can then decide whether to merge into the lane on the other side of merging area 1130. Successfully executing dual lane merging in busy traffic can require significant negotiation skills and experience, and may be difficult to perform heuristically or brute-force by enumerating all possible trajectories that all agents might take in the scenario. In this dual lane merging example, an expectation set suitable for dual lane merging maneuvering can be defined. It can be the Cartesian product of the following sets:

[0273] Where, [0, v max Let L be the desired target speed of the primary vehicle, L = {1, 1.5, 2, 2.5, 3, 3.5, 4} be the desired lateral position in lane units, where integers represent lane centers and fractions represent lane boundaries, and {g, t, o} are classification labels assigned to each of the n other vehicles. If the primary vehicle is to yield to another vehicle, the other vehicle can be assigned "g"; if the primary vehicle is to occupy the lane relative to another vehicle, the other vehicle can be assigned "t"; or if the primary vehicle is to maintain an offset distance relative to another vehicle, the other vehicle can be assigned "o".

[0274] The following describes a set of expectations. How can this be transformed into a cost function for the driving trajectory? The driving trajectory can be represented by (x1, y1), ..., (x... k y k ) indicates that (x i y i Let be the (lateral, longitudinal) position of the main vehicle at time τ·i (in egocentric units), where in some experiments τ = 0.1 sec and k = 10. Of course, other values ​​can be chosen. The cost assigned to the trajectory can be a weighted sum of the individual costs assigned to the desired speed, lateral position, and the label assigned to each of the other n vehicles.

[0275] Given the desired rate v∈[0, v max The cost of the trajectory associated with the rate is

[0276]

[0277] Given a desired lateral position l∈L, the cost associated with that desired lateral position is:

[0278]

[0279] Where dist(x, y, l) is the distance from point (x, y) to lane position l. Regarding the cost due to other vehicles, for any other vehicle, (x′1, y′1), ..., (x′1, y′1)... k y′ k Let be the other vehicles in the egocentric unit of the main vehicle, and i can be the earliest point, for which there exists j such that (x i y i ) and (x′ j y′ jThe distance between them is very small. If there is no such point, then i can be set to i = ∞. If the other vehicle is classified as "giving way", then it can be expected that τi > τj + 0.5, which means that the main vehicle will arrive at the trajectory intersection point at least 0.5 seconds after the other vehicle arrives at the same point. A possible formula for converting the above constraints into costs is [τ(ji) + 0.5]. + .

[0280] Similarly, if another car is classified as "obstructing traffic," we can expect τj > τi + 0.5, which can be converted into a cost [τ(ij) + 0.5]. + If another car is classified as "offset," then we can expect i = ∞, meaning the trajectory of the main vehicle and the trajectory of the offset car do not intersect. This situation can be converted into a cost by penalizing the distance between the trajectories.

[0281] Assigning weights to each of these costs can provide a single objective function π for the trajectory planner. (T) The costs of encouraging smooth driving can be added to this objective. Furthermore, to ensure the functional safety of the trajectory, hard constraints can be added to this objective. For example, (x) can be prohibited. i y i ) leaves the roadway, and for any trajectory point (x′) of any other vehicle j y′ j ), can prohibit (x) i y i ) close to (x′ j y′ j If |ij| is small.

[0282] In summary, strategy π θ It can be decomposed into a mapping from the unknown state to the expected set and a mapping from the expected to the actual trajectory. The latter mapping is not based on learning and can be achieved by solving an optimization problem whose cost depends on the expectation, and whose hard constraints guarantee the functional safety of the policy.

[0283] The following discussion describes the mapping from the unknown state to the expected set. As mentioned above, to meet functional safety requirements, systems relying solely on reinforcement learning may suffer from inconsistencies regarding rewards. The high and inconvenient variance of the system can be avoided by decomposing the problem into a mapping from the (unknown) state space to the desired set using policy gradient iteration, followed by mapping to the actual trajectory of the system that is not involved in machine learning-based training.

[0284] For various reasons, decision-making can be further broken down into semantically meaningful components. For example, The size can be large or even continuous. Regarding Figure 11D In the dual lane merging scenario described above, Additionally, gradient estimators may involve terms In such an expression, the variance can increase with the time interval T. In some cases, the value of T can be approximately 250, which may be high enough to produce significant variance. Assuming a sampling rate in the range of 10 Hz and a lane merging zone of 1130 meters, preparation for merging can begin approximately 300 meters before the merging zone. If the main vehicle is traveling at a speed of 16 meters per second (approximately 60 km / h), the value of T for an episode can be approximately 250.

[0285] Returning to the concept of option diagrams, Figure 11E The text shows what can be represented. Figure 11D The option graph depicts a dual-lane scenario. As previously mentioned, an option graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There may be a special node in this graph called the "root" node 1140, which can be the only node without any incoming edges (e.g., decision lines). The decision-making process can traverse the graph starting from the root node until it reaches a "leaf" node, i.e., a node without any outgoing edges. Each internal node can implement a policy function that selects offspring from its available offspring. There can exist a set of traversals from the option graph leading to the desired set. The predefined mapping. In other words, traversal on the option graph can be automatically converted to... The expectation in the graph. Given a node v in the graph, and a parameter vector θ v You can specify a strategy for selecting the offspring of v. If θ is all θ v If a concatenation exists, then it can be defined by traversing from the root of the graph to the leaves. At the same time, at each node v, use θ v Define a strategy to select child nodes.

[0286] exist Figure 11E In the dual lane merging option diagram 1139, the root node 1140 can first determine whether the main vehicle is within the lane merging area (e.g., Figure 11DIn area 1130, the primary vehicle may be approaching a lane-merging area and needs to prepare for a possible lane change. In both cases, the primary vehicle may need to decide whether to change lanes (e.g., to the left or right) or to remain in its current lane. If the primary vehicle has decided to change lanes, it may need to determine whether conditions are suitable for continuing and perform a lane-change maneuver (e.g., at “Continue” node 1142). If changing lanes is not possible, the primary vehicle may attempt to “advance” towards the desired lane by aiming at a lane marking (e.g., at node 1144, as part of negotiation with vehicles in the desired lane). Alternatively, the primary vehicle may choose to “remain” in the same lane (e.g., at node 1146). This process can naturally determine the lateral position of the primary vehicle. For example,

[0287] This allows for the natural determination of the desired lateral position. For example, if the master vehicle changes lane from lane 2 to lane 3, the "Continue" node can set the desired lateral position to 3, the "Maintain" node to 2, and the "Advance" node to 2.5. Next, the master vehicle can decide whether to maintain the "same" rate (node ​​1148), "accelerate" (node ​​1150), or "decelerate" (node ​​1152). Then, the master vehicle can enter a "chain" structure 1154 that overtakes other vehicles and sets their semantic meaning to values ​​in the set {g,t,o}. This process sets the expectation relative to other vehicles. Parameters can be shared across all nodes in this chain (similar to a recurrent neural network).

[0288] A potential benefit of the option is the interpretability of the results. Another potential benefit is the ability to rely on sets. The structure is decomposable, and therefore, the policy at each node can be selected from a small number of possibilities. Furthermore, this structure allows for a reduction in the variance of the policy gradient estimator.

[0289] As mentioned above, the length of a segment in a dual lane-changing scenario can be approximately T = 250 steps. Such a value (or any other suitable value depending on the specific navigation scenario) provides sufficient time to see the consequences of the lead vehicle's actions (e.g., if the lead vehicle decides to change lanes in preparation for a lane change, it will only see the benefit after successfully completing the lane change). On the other hand, due to the dynamic nature of driving, the lead vehicle must make decisions at a sufficiently high frequency (e.g., 10 Hz in the above case).

[0290] Option graphs can reduce the effective value of T in at least two ways. First, given higher-level decisions, rewards can be defined for lower-level decisions while considering shorter segments. For example, when the master vehicle has already selected the "lane change" and "continue" nodes, by watching a 2-3 second segment (meaning T becomes 20-30 instead of 250), a policy for assigning semantic meaning to the vehicle can be learned. Second, for higher-level decisions (such as whether to change lanes or stay in the same lane), the master vehicle may not need to make a decision every 0.1 seconds. Instead, the master vehicle can make decisions at a lower frequency (e.g., per second), or implement an "option termination" function, then the gradient can be computed only after each option termination. In both cases, the effective value of T may be an order of magnitude smaller than its original value. In summary, the estimator for each node can depend on the value of T, which is an order of magnitude smaller than the original 250 steps, which may immediately translate to a smaller variance.

[0291] As mentioned above, hard constraints can promote safer driving, and several different types of constraints can exist. For example, static hard constraints can be defined directly based on the sensing state. These hard constraints can include speed bumps, speed limits, road curves, intersections, etc., within the host vehicle's environment, and may involve one or more constraints on vehicle speed, heading, acceleration, braking (deceleration), etc. Static hard constraints may also include semantic free space, where, for example, the host vehicle is prohibited from driving out of free space and from driving too close to physical obstacles. Static hard constraints can also restrict (e.g., prohibit) maneuvers that do not conform to various aspects of the vehicle's kinematics; for example, static hard constraints can be used to prohibit maneuvers that may cause the host vehicle to roll over, slide, or otherwise lose control.

[0292] Hard constraints can also be associated with vehicles. For example, a constraint could be required that a vehicle maintain a longitudinal distance of at least one meter and a lateral distance of at least 0.5 meters from other vehicles. Constraints can also be applied such that the primary vehicle will avoid maintaining a collision path with one or more other vehicles. For example, time τ can be a time metric based on a specific scenario. Predicted trajectories of the primary vehicle and one or more other vehicles can be considered from the current time to time τ. In the case where two trajectories intersect, This can represent the arrival and departure times of vehicle i at the intersection point. That is, each car will arrive at the intersection point when the first part of the car passes through it, and a certain amount of time is required before the last part of the car passes through it. This amount of time separates the arrival time from the departure time. Assume... (That is, if vehicle 1 arrives at a later time than vehicle 2), then we want to ensure that vehicle 1 has left the intersection point before vehicle 2 arrives. Otherwise, a collision will occur. Therefore, a hard constraint can be implemented such that... Furthermore, to ensure that vehicles 1 and 2 do not stagger each other by a minimum amount, an additional safety margin can be obtained by including a buffer time (e.g., 0.5 seconds or another appropriate value) in the constraints. The hard constraints related to the predicted intersection trajectories of the two vehicles can be expressed as follows:

[0293] The amount of time τ during which the trajectories of the primary vehicle and one or more other vehicles are tracked can vary. However, in the intersection scenario, where the speed may be lower, τ may be longer, and τ can be defined as such that the primary vehicle will enter and leave the intersection in less than τ seconds.

[0294] Of course, applying hard constraints to vehicle trajectories requires predicting the trajectories of those vehicles. For the primary vehicle, trajectory prediction may be relatively straightforward, as it typically already understands and is, in fact, planning the expected trajectory at any given time. Predicting the trajectories of other vehicles may be less straightforward. For other vehicles, the baseline calculations used to determine the predicted trajectories may depend on their current speed and heading, for example, as determined by analysis of image streams captured by one or more cameras and / or other sensors (radar, lidar, acoustic, etc.) on the primary vehicle.

[0295] However, there may be exceptions that can simplify the problem or at least provide increased confidence in the predicted trajectory for another vehicle. For example, in structured roads with lane markings and the possibility of yield rules, the trajectory of other vehicles can be based at least in part on the position of the other vehicles relative to the lanes and on applicable yield rules. Therefore, in some cases, when lane structure is observed, it can be assumed that vehicles in the next lane will adhere to lane boundaries. That is, the primary vehicle can assume that vehicles in the next lane will remain in their lane unless evidence indicating that a vehicle in the next lane will cut into the primary vehicle's lane is observed (e.g., traffic lights, strong lateral movement, movement across lane boundaries).

[0296] Other situations can also provide clues about the expected trajectories of other vehicles. For example, where the primary vehicle may have the right-of-way, at stop signs, traffic lights, roundabouts, etc., it can be assumed that other vehicles will respect that right-of-way. Therefore, unless evidence of a violation is observed, it can be assumed that other vehicles will continue along the trajectory of those who respect the primary vehicle's right-of-way.

[0297] Hard constraints can also be applied relative to pedestrians in the host vehicle's environment. For example, a buffer distance can be established relative to pedestrians, prohibiting the host vehicle from driving closer than a predetermined buffer distance relative to any observed pedestrian. The pedestrian buffer distance can be any suitable distance. In some embodiments, the buffer distance can be at least one meter relative to an observed pedestrian.

[0298] Similar to the case with vehicles, hard constraints can also be applied relative to the relative motion between the pedestrian and the main vehicle. For example, the pedestrian's trajectory (based on heading and speed) can be monitored relative to the expected trajectory of the main vehicle. Given a specific pedestrian trajectory, where for each point p on the trajectory, t(p) can represent the time required for the pedestrian to reach point p. To maintain the required buffer distance of at least 1 meter from the pedestrian, t(p) must be greater than the time it takes for the main vehicle to reach point p (with a sufficient time difference so that the main vehicle passes at least 1 meter ahead of the pedestrian), or t(p) must be less than the time it takes for the main vehicle to reach point p (e.g., if the main vehicle brakes to give way to the pedestrian). However, in the latter example, the hard constraint might require the main vehicle to arrive at point p sufficiently later than the pedestrian so that the main vehicle can pass behind the pedestrian and maintain the required buffer distance of at least 1 meter. Of course, there may be exceptions to the hard constraints for pedestrians. For example, if the primary vehicle has the right-of-way or is traveling at a very slow speed, and there is no evidence that the pedestrian will refuse to give way to the primary vehicle or will otherwise navigate toward the primary vehicle, the pedestrian hard constraint can be relaxed (e.g., to a smaller buffer of at least 0.75 meters or 0.50 meters).

[0299] In some examples, constraints may be relaxed when it is determined that not all constraints can be satisfied. For example, if a road is too narrow to allow for two curbs or the required clearance (e.g., 0.5 meters) between a curb and a parked vehicle, one or more constraints may be relaxed if mitigation is possible. For example, if there are no pedestrians (or other objects) on the sidewalk, one may proceed slowly at a distance of 0.1 meters from the curb. In some embodiments, constraints may be relaxed if relaxing them improves the user experience. For example, to avoid potholes, constraints may be relaxed to allow vehicles to approach lane edges, curbs, or pedestrians more closely than would normally be permissible. Furthermore, in determining which constraints to relax, in some embodiments, one or more constraints to relax are those considered to have the least available negative impact on safety. For example, constraints regarding how close a vehicle can get to a curb or concrete barrier may be relaxed before relaxing constraints on proximity to other vehicles. In some embodiments, pedestrian constraints may be the last to be relaxed, or in some cases may never be relaxed.

[0300] Figure 12Examples of scenes that can be captured and analyzed during navigation of the host vehicle are shown. For example, the host vehicle may include a navigation system (e.g., system 100) as described above, which can receive multiple images representing the host vehicle's environment from cameras (e.g., at least one of image capture device 122, image capture device 124, and image capture device 126) associated with the host vehicle. Figure 12 The scene shown is an example of one of the images that can be captured at time t from the environment of a master vehicle traveling along a predicted trajectory 1212 in lane 1210. The navigation system may include at least one processing device (e.g., any EyeQ processor or other device described above) specifically programmed to receive multiple images and analyze them to determine actions in response to the scene. Specifically, as... Figure 8 As shown, at least one processing device can implement a sensing module 801, a driving strategy module 803, and a control module 805. The sensing module 801 can collect and output image information collected from a camera, and provide this information to the driving strategy module 803 in the form of a recognized navigation state, which can constitute a trained navigation system trained using machine learning techniques (such as supervised learning, reinforcement learning, etc.). Based on the navigation state information provided to the driving strategy module 803 by the sensing module 801, the driving strategy module 803 (e.g., by implementing the option map method described above) can generate the desired navigation actions to be performed by the main vehicle in response to the recognized navigation state.

[0301] In some embodiments, at least one processing device may use, for example, control module 805, to directly translate the desired navigation action into a navigation command. However, in other embodiments, hard constraints may be applied to test the desired navigation action provided by driving strategy module 803 against various predetermined navigation constraints that may be involved in the scene and the desired navigation action. For example, if driving strategy module 803 outputs a desired navigation action that would cause the master vehicle to follow trajectory 1212, the navigation action may be tested against one or more hard constraints associated with various aspects of the master vehicle's environment. For example, captured image 1201 may show curb 1213, pedestrian 1215, target vehicle 1217, and stationary objects present in the scene (e.g., an overturned box). Each of these may be associated with one or more hard constraints. For example, curb 1213 may be associated with a static constraint that prevents the master vehicle from navigating into or past the curb and onto pedestrian crossing 1214. The curb 1213 can also be associated with a barrier envelope that defines a distance (e.g., a buffer zone) extending away from the curb (e.g., 0.1 m, 0.25 m, 0.5 m, 1 m, etc.) and along the curb, defining a navigation-free zone for the main vehicle. Of course, static constraints can also be associated with other types of roadside boundaries (e.g., guardrails, concrete posts, traffic cones, bridge towers, or any other type of roadside barrier).

[0302] It should be noted that distance and range can be determined by any suitable method. For example, in some embodiments, distance information can be provided by an onboard radar and / or lidar system. Alternatively or additionally, distance information can be derived from the analysis of one or more images captured from the environment of the host vehicle. For example, multiple pixels representing an identified object in an image can be determined and compared with the known field of view and focal length geometry of the image capturing device to determine scale and distance. For example, velocity and acceleration can be determined by observing the scale change between objects in one image and another over a known time interval. This analysis can indicate the direction of movement toward or away from the host vehicle, and at what speed the object is moving away from or toward the host vehicle. Crossing speed can be determined by analyzing the change in the X-coordinate position of the object from one image to another over a known time interval.

[0303] Pedestrian 1215 can be associated with a pedestrian envelope defining buffer zone 1216. In some cases, the imposed hard constraints can prohibit the host vehicle from navigating within one meter of pedestrian 1215 (in any direction relative to pedestrian 1215). Pedestrian 1215 can also define the location of pedestrian influence zone 1220. This influence zone can be associated with constraints limiting the speed of the host vehicle within the influence zone. The influence zone can extend from pedestrian 1215 to 5 meters, 10 meters, 20 meters, etc. Each gradient of the influence zone can be associated with a different speed limit. For example, within the area one to five meters from pedestrian 1215, the host vehicle can be limited to a first speed (e.g., 10 mph, 20 mph, etc.), which can be less than the speed limit in a pedestrian influence zone extending from 5 meters to 10 meters. Any gradient can be used for each gradient of the influence zone. In some embodiments, the first gradient can be narrower than from one meter to five meters and can extend only from one meter to two meters. In other embodiments, the first level of the influence zone may extend from one meter (the boundary of the unnavigated area surrounding the pedestrian) to a distance of at least 10 meters. The second level may then extend from ten meters to at least approximately twenty meters. The second level may be associated with the maximum speed of the main vehicle, which is greater than the maximum speed associated with the first level of the pedestrian influence zone.

[0304] One or more stationary object constraints may also be associated with a scene detected in the environment of the host vehicle. For example, in image 1201, at least one processing device may detect a stationary object, such as box 1219 present in a roadway. The detected stationary object may include at least one of various objects, such as a tree, pole, road sign, or other objects in the roadway. One or more predetermined navigation constraints may be associated with the detected stationary object. For example, such constraints may include a stationary object envelope that defines a buffer zone about the object, within which navigation of the host vehicle may be prohibited. At least a portion of the buffer zone may extend a predetermined distance from the edge of the detected stationary object. For example, in the scene represented by image 1201, a buffer zone of at least 0.1 meters, 0.25 meters, 0.5 meters, or more may be associated with box 1219, such that the host vehicle will pass at least a distance (e.g., buffer zone distance) to the right or left of the box to avoid a collision with the detected stationary object.

[0305] Predefined hard constraints can also include one or more target vehicle constraints. For example, a target vehicle 1217 can be detected in image 1201. To ensure that the primary vehicle does not collide with the target vehicle 1217, one or more hard constraints can be employed. In some cases, the target vehicle envelope can be associated with a single buffer distance. For example, a buffer can be defined by a distance of 1 meter around the target vehicle in all directions. The buffer can define an area extending at least one meter from the target vehicle, prohibiting the primary vehicle from navigating into that area.

[0306] However, the envelope surrounding the target vehicle 1217 does not need to be defined by a fixed buffer distance. In some cases, the predefined hard constraints associated with the target vehicle (or any other movable object detected in the environment of the master vehicle) may depend on the orientation of the master vehicle relative to the detected target vehicle. For example, in some cases, the longitudinal buffer distance (e.g., the distance extending from the target vehicle toward the front or rear of the master vehicle—such as when the master vehicle is traveling toward the target vehicle) may be at least one meter. The lateral buffer distance (e.g., the distance extending from the target vehicle toward either side of the master vehicle—such as when the master vehicle is traveling in the same or opposite direction as the target vehicle, such that one side of the master vehicle passes the side adjacent to the target vehicle) may be at least 0.5 meters.

[0307] As mentioned above, other constraints may also be involved in detecting target vehicles or pedestrians in the environment of the main vehicle. For example, the predicted trajectories of the main vehicle and target vehicle 1217 can be considered, and hard constraints may be required if the two trajectories intersect (e.g., at intersection point 1230). or The primary vehicle is vehicle 1, and the target vehicle 1217 is vehicle 2. Similarly, the trajectory of pedestrian 1215 can be monitored relative to the expected trajectory of the primary vehicle (based on heading direction and speed). Given a specific pedestrian trajectory, for each point p on the trajectory, t(p) will represent the pedestrian's arrival point p (i.e., Figure 12 The time required for the main vehicle to reach point p (points 1, 2, 3, and 1) is as follows. To maintain the required buffer distance of at least 1 meter from the pedestrian, t(p) must be greater than the time it takes for the main vehicle to reach point p (with a sufficient time difference so that the main vehicle passes in front of the pedestrian at least 1 meter ahead), or t(p) must be less than the time it takes for the main vehicle to reach point p (e.g., if the main vehicle brakes to give way to the pedestrian). Nevertheless, in the latter example, the hard constraint would require the main vehicle to arrive at point p a sufficient time later than the pedestrian so that the main vehicle can pass behind the pedestrian and maintain the required buffer distance of at least 1 meter.

[0308] Other hard constraints may also be employed. For example, in at least some cases, a maximum deceleration rate of the master vehicle may be used. This maximum deceleration rate can be determined based on the detected distance to the target vehicle following the master vehicle (e.g., using images collected from a rear-view camera). Hard constraints may include mandatory stops at sensed crosswalks or railway crossings, or other applicable constraints.

[0309] In cases where the analysis of a scenario within the host vehicle's environment indicates that one or more predefined navigation constraints may be involved, these constraints can be imposed relative to one or more planned navigation actions of the host vehicle. For example, if the analysis of a scenario causes the driving strategy module 803 to return a desired navigation action, that desired navigation action can be tested against one or more of the involved constraints. If the desired navigation action is determined to violate any aspect of the involved constraints (e.g., if the desired navigation action would drive the host vehicle within 0.7 meters of pedestrian 1215, where a predefined hard constraint requires the host vehicle to maintain a distance of at least 1.0 meter from pedestrian 1215), at least one modification to the desired navigation action can be made based on one or more predefined navigation constraints. Adjusting the desired navigation action in this way can provide the host vehicle with actual navigation actions by adhering to the constraints involved in the specific scenario detected in the host vehicle's environment.

[0310] After determining the actual navigation action of the main vehicle, the navigation action can be achieved by adjusting at least one of the navigation actuators of the main vehicle in response to the determined actual navigation action. Such a navigation actuator may include at least one of the main vehicle's steering mechanism, brake, or accelerator.

[0311] Priority constraints

[0312] As mentioned above, navigation systems can employ various hard constraints to ensure the safe operation of the primary vehicle. Constraints may include minimum safe driving distances relative to pedestrians, target vehicles, roadblocks, or detected objects; maximum driving speed when passing within the influence zone of a detected pedestrian; or the maximum deceleration rate of the primary vehicle, etc. These constraints can be imposed on trained systems based on machine learning (supervised, reinforcement, or combined) training, but they can also be useful for untrained systems (e.g., those that employ algorithms to directly handle expected situations occurring in a scenario derived from the primary vehicle's environment).

[0313] In any situation, a hierarchy of constraints may exist. In other words, some navigation constraints have priority over others. Therefore, if a situation arises where a navigation action that would satisfy all involved constraints is unavailable, the navigation system can first determine the available navigation action that fulfills the highest priority constraint. For example, the system could cause the vehicle to avoid the pedestrian, even if avoiding the pedestrian would result in a collision with another vehicle or object detected in the road. In another example, the system could cause the vehicle to ride on the curb to avoid the pedestrian.

[0314] Figure 13 A flowchart illustrating an algorithm for implementing a hierarchy of relevant constraints determined based on analysis of a scene in the environment of the host vehicle is provided. For example, in step 1301, at least one processing device associated with the navigation system (e.g., an EyeQ processor, etc.) can receive multiple images representing the environment of the host vehicle from a camera mounted on the host vehicle. Through analysis of one or more images representing the scene of the host vehicle's environment in step 1303, a navigation state associated with the host vehicle can be identified. In addition to various other attributes of the scene, for example, the navigation state may indicate that the host vehicle is traveling along a two-lane road 1210, such as... Figure 12 As shown, target vehicle 1217 is moving through an intersection in front of the main vehicle, pedestrian 1215 is waiting to cross the road where the main vehicle is traveling, and object 1219 is in front of the main vehicle's lane.

[0315] In step 1305, one or more navigation constraints related to the navigation state of the primary vehicle can be determined. For example, after analyzing a scene in the environment of the primary vehicle represented by one or more captured images, at least one processing device can determine one or more navigation constraints related to objects, vehicles, pedestrians, etc., identified through image analysis of the captured images. In some embodiments, at least one processing device can determine at least a first predefined navigation constraint and a second predefined navigation constraint related to the navigation state, and the first predefined navigation constraint may be different from the second predefined navigation constraint. For example, the first navigation constraint may relate to one or more target vehicles detected in the environment of the primary vehicle, and the second navigation constraint may relate to pedestrians detected in the environment of the primary vehicle.

[0316] In step 1307, at least one processing device may determine the priority associated with the constraints identified in step 1305. In the described example, a second predefined navigation constraint involving pedestrians may have a higher priority than a first predefined navigation constraint involving a target vehicle. While priorities associated with navigation constraints can be determined or assigned based on various factors, in some embodiments, the priority of a navigation constraint may be related to its relative importance from a safety perspective. For example, while it may be important to comply with or satisfy all implemented navigation constraints as many times as possible, some constraints may be associated with a higher safety risk compared to others, and therefore may be assigned a higher priority. For example, a navigation constraint requiring the primary vehicle to maintain a distance of at least 1 meter from a pedestrian may have a higher priority than a constraint requiring the primary vehicle to maintain a distance of at least 1 meter from a target vehicle. This may be because a collision with a pedestrian may have more serious consequences than a collision with another vehicle. Similarly, maintaining space between the primary vehicle and the target vehicle may have a higher priority than constraints requiring the primary vehicle to avoid boxes on the road, drive below a certain speed on speed bumps, or expose the primary vehicle's passengers to no more than the maximum acceleration level.

[0317] Although the driving strategy module 803 is designed to maximize safety by satisfying navigation constraints involved in a specific scenario or navigation state, in some cases it is practically impossible to satisfy every single constraint involved. In such cases, as shown in step 1309, the priority of each involved constraint can be used to determine which involved constraint should be satisfied first. Continuing the example above, in the case where it is impossible to satisfy both the pedestrian clearance constraint and the target vehicle clearance constraint, and only one of these constraints can be satisfied, the higher-priority pedestrian clearance constraint may cause that constraint to be satisfied before attempting to maintain clearance to the target vehicle. Therefore, under normal circumstances, as shown in step 1311, where both the first predefined navigation constraint and the second predefined navigation constraint can be satisfied, at least one processing device can determine the first navigation action of the main vehicle that satisfies both the first and second predefined navigation constraints based on the identified navigation state of the main vehicle. However, in other cases, where not all involved constraints can be satisfied, such as in step 1313, where the first predefined navigation constraint and the second predefined navigation constraint cannot be satisfied simultaneously, at least one processing device can determine a second navigation action of the master vehicle that satisfies the second predefined navigation constraint (i.e., the higher priority constraint) but does not satisfy the first predefined navigation constraint (which has a lower priority than the second navigation constraint) based on the identified navigation state.

[0318] Next, in step 1315, in order to implement the determined navigation action of the main vehicle, at least one processing device may, in response to the determined first navigation action or the determined second navigation action of the main vehicle, cause at least one adjustment to the navigation actuator of the main vehicle. As described in the preceding examples, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0319] Constraint relaxation

[0320] As mentioned above, navigation constraints can be imposed for safety purposes. Constraints may include a minimum safe driving distance relative to a pedestrian, target vehicle, roadblock, or detected object; a maximum driving speed when passing within the influence zone of a detected pedestrian; or a maximum deceleration rate for the primary vehicle, etc. These constraints can be imposed on learning or non-learning navigation systems. In some cases, these constraints can be relaxed. For example, if the primary vehicle slows down or stops near a pedestrian and then slowly moves forward to convey its intention to pass the pedestrian, the pedestrian's response can be detected from the acquired images. If the pedestrian's response is to remain still or stop moving (and / or if eye contact with the pedestrian is sensed), it can be understood that the pedestrian recognizes the navigation system's intention to pass the pedestrian. In this case, the system can relax one or more predefined constraints and enforce less stringent constraints (e.g., allowing the vehicle to navigate within 0.5 meters of the pedestrian, rather than within a more stringent 1-meter boundary).

[0321] Figure 14 A flowchart is provided for implementing control of a primary vehicle based on the relaxation of one or more navigation constraints. In step 1401, at least one processing device may receive multiple images representing the environment of the primary vehicle from a camera associated with the primary vehicle. Analysis of the images in step 1403 enables the identification of a navigation state associated with the primary vehicle. In step 1405, at least one processor may determine navigation constraints associated with the navigation state of the primary vehicle. Navigation constraints may include a first predefined navigation constraint relating to at least one aspect of the navigation state. In step 1407, analysis of the multiple images may reveal the presence of at least one navigation constraint relaxation factor.

[0322] Navigation constraint slack factors may include any suitable indicator that one or more navigation constraints can be suspended, altered, or otherwise relaxed in at least one respect. In some embodiments, at least one navigation constraint slack factor may include (based on image analysis) the determination that a pedestrian's eyes are looking in the direction of the primary vehicle. In this case, it can be more safely assumed that the pedestrian is aware of the primary vehicle. Therefore, the confidence that the pedestrian will not engage in an unintended action that would cause the pedestrian to move into the path of the primary vehicle may be higher. Other constraint slack factors may also be used. For example, at least one navigation constraint slack factor may include: a pedestrian determined not to be moving (e.g., a pedestrian assumed to be unlikely to enter the path of the primary vehicle); or a pedestrian whose movement is determined to be slowing down. Navigation constraint slack factors may also include more complex actions, such as a pedestrian determined not to be moving after the primary vehicle has come to a stop and then resumed movement. In this case, it can be assumed that the pedestrian understands that the primary vehicle has the right-of-way, and that the pedestrian's coming to a stop may indicate the pedestrian's intention to give way to the primary vehicle. Other circumstances that may lead to the slack of one or more constraints include the type of curb (e.g., a low curb or a curb with a gradual slope may allow for a slack distance constraint), the absence of pedestrians or other objects on the sidewalk, vehicles whose engines are not running may have a slack distance, or situations where pedestrians are moving away from and / or away from the area that the main vehicle is traveling towards.

[0323] Upon identification of a navigation constraint slack factor (e.g., in step 1407), a second navigation constraint can be determined or developed in response to the detection of the constraint slack factor. This second navigation constraint may differ from the first navigation constraint and may include at least one characteristic that is relaxed relative to the first navigation constraint. The second navigation constraint may include a newly generated constraint based on the first constraint, wherein the newly generated constraint includes at least one modification that relaxes the first constraint in at least one aspect. Alternatively, the second constraint may constitute a predetermined constraint that is less stringent than the first navigation constraint in at least one aspect. In some embodiments, such a second constraint may be retained only for cases where constraint slack factors are identified in the environment of the primary vehicle. Whether the second constraint is newly generated or selected from a set of fully or partially available predetermined constraints, applying the second navigation constraint instead of the more stringent first navigation constraint (which can be applied where no relevant navigation constraint slack factor is detected) can be referred to as constraint slack and can be completed in step 1409.

[0324] If at least one constraint relaxation factor is detected in step 1407, and at least one constraint has been relaxed in step 1409, the navigation action of the master vehicle can be determined in step 1411. The navigation action of the master vehicle can be based on the identified navigation state and can satisfy a second navigation constraint. The navigation action can be implemented in step 1413 by causing at least one adjustment to the navigation actuator of the master vehicle in response to the determined navigation action.

[0325] As described above, the use of navigation constraints and relaxed navigation constraints can be employed by navigation systems that are trained (e.g., through machine learning) or untrained (e.g., a navigation system programmed to respond with a predetermined action in response to a specific navigation state). In the case of a trained navigation system, the availability of relaxed navigation constraints for certain navigation situations can represent a mode switch from trained system response to untrained system response. For example, a trained navigation network can determine the initial navigation action of the primary vehicle based on a first navigation constraint. However, the action taken by the vehicle can be different from the navigation action that satisfies the first navigation constraint. Instead, the action taken can satisfy a more relaxed second navigation constraint and can be an action developed by an untrained system (e.g., as a response to detecting a specific condition in the primary vehicle's environment, such as the presence of a navigation constraint relaxation factor).

[0326] There are numerous examples of navigation constraints that can be relaxed in response to the detection of constraint relaxation factors in the environment of the master vehicle. For example, where predefined navigation constraints include a buffer associated with a detected pedestrian, and at least a portion of the buffer extends a certain distance from the detected pedestrian, the relaxed navigation constraints (newly generated, retrieved from memory from a predetermined set, or generated as a relaxed version of a previously existing constraint) can include different or modified buffers. For example, the different or modified buffers may have a smaller distance relative to the pedestrian than the original or unmodified buffer relative to the detected pedestrian. Thus, given the relaxed constraints, the master vehicle can be allowed to navigate closer to the detected pedestrian, where an appropriate constraint relaxation factor is detected in the environment of the master vehicle.

[0327] As described above, the slack characteristics of navigation constraints may include a reduced width of the buffer associated with at least one pedestrian. However, the slack characteristics may also include a reduced width of the buffer associated with a target vehicle, a detected object, a roadside obstacle, or any other object detected in the environment of the primary vehicle.

[0328] At least one relaxation characteristic may also include other types of modifications to the navigation constraint characteristics. For example, a relaxation characteristic may include an increase in the rate associated with at least one predefined navigation constraint. A relaxation characteristic may also include an increase in the maximum permissible deceleration / acceleration associated with at least one predefined navigation constraint.

[0329] While constraints can be relaxed in certain situations, as described above, navigation constraints can be enhanced in others. For example, in some cases, the navigation system can determine a condition warrant to enhance a set of normal navigation constraints. This enhancement may include adding new constraints to a predefined set of constraints or adjusting one or more aspects of predefined constraints. Additions or adjustments may result in more conservative navigation relative to the predefined set of constraints applicable under normal driving conditions. Conditions that may warrant constraint enhancement may include sensor malfunctions, adverse environmental conditions (rain, snow, fog, or other conditions associated with reduced visibility or reduced vehicle traction), etc.

[0330] Figure 15 A flowchart is provided for implementing control of a primary vehicle based on enhancements to one or more navigation constraints. In step 1501, at least one processing device may receive multiple images representing the environment of the primary vehicle from a camera associated with the primary vehicle. In step 1503, analysis of the images enables the identification of a navigation state associated with the primary vehicle. In step 1505, at least one processor may determine navigation constraints associated with the navigation state of the primary vehicle. The navigation constraints may include a first predefined navigation constraint relating to at least one aspect of the navigation state. In step 1507, analysis of the multiple images may reveal the presence of at least one navigation constraint enhancement factor.

[0331] The navigation constraints involved may include those mentioned above (e.g., regarding...). Figure 12Any navigation constraint or any other suitable navigation constraint. Navigation constraint enhancement factors may include any indicator that one or more navigation constraints can be supplemented / enhanced in at least one aspect. Supplementation or enhancement of navigation constraints may be performed on a per-set basis (e.g., by adding a new navigation constraint to a predetermined set of constraints) or on a per-constraint basis (e.g., modifying a particular constraint so that the modified constraint is more restrictive than the original constraint, or adding a new constraint corresponding to a predetermined constraint, wherein the new constraint is more restrictive than the corresponding constraint in at least one aspect). Alternatively or additionally, supplementation or enhancement of navigation constraints may refer to selection from a predetermined set of constraints based on hierarchy. For example, a set of enhanced constraints may be used for selection based on whether a navigation enhancement factor is detected in or relative to the host vehicle's environment. In the normal case where no enhancement factor is detected, the relevant navigation constraints can be extracted from constraints applicable to the normal case. On the other hand, in the case where one or more constraint enhancement factors are detected, the relevant constraints can be extracted from enhanced constraints that are generated or predefined relative to one or more enhancement factors. The enhanced constraints may be more restrictive in at least one aspect compared to the corresponding constraints applicable under normal conditions.

[0332] In some embodiments, at least one navigation constraint enhancement factor may include detecting (e.g., based on image analysis) the presence of ice, snow, or water on a road surface in the environment of the primary vehicle. For example, such determination may be based on the detection of: areas with higher reflectivity than expected on a dry roadway (e.g., indicating ice or water on the roadway); white areas on the roadway indicating the presence of snow; shadows on the roadway consistent with the presence of longitudinal grooves on the roadway (e.g., tire tracks in snow); water droplets or ice / snow particles on the primary vehicle's windshield; or any other suitable indicator of the presence of water or ice / snow on the road surface.

[0333] At least one navigation constraint enhancement factor may also include the detection of particles on the outer surface of the windshield of the host vehicle. Such particles may impair the image quality of one or more image capture devices associated with the host vehicle. While the description pertains to the windshield of the host vehicle in relation to a camera mounted behind the windshield, the detection of particles on other surfaces (e.g., camera lenses or lens caps, headlight lenses, rear windshields, taillight lenses, or any other surface of the host vehicle visible to (or detected by) image capture devices associated with the host vehicle) may also indicate the presence of a navigation constraint enhancement factor.

[0334] Navigation constraint enhancement factors can also be detected as attributes of one or more image acquisition devices. For example, a detected reduction in image quality of one or more images captured by an image capture device (e.g., a camera) associated with the primary vehicle can also constitute a navigation constraint enhancement factor. The degradation in image quality may be associated with a hardware malfunction or partial hardware malfunction related to the image capture device or components associated with it. This degradation in image quality can also be caused by environmental conditions. For example, the presence of smoke, fog, rain, snow, etc., in the air surrounding the primary vehicle can also lead to reduced image quality relative to roads, pedestrians, target vehicles, etc., that may be present in the environment surrounding the primary vehicle.

[0335] Navigation constraint enhancement factors can also involve other aspects of the primary vehicle. For example, in some cases, navigation constraint enhancement factors may include detected faults or partial faults in systems or sensors associated with the primary vehicle. Such enhancement factors may include, for example, the detection of faults or partial faults in speed sensors, GPS receivers, accelerometers, cameras, radar, lidar, brakes, tires, or any other system associated with the primary vehicle that could affect the primary vehicle's ability to navigate relative to navigation constraints associated with the primary vehicle's navigation state.

[0336] Upon identifying the presence of a navigation constraint enhancement factor (e.g., in step 1507), a second navigation constraint can be determined or developed in response to the detection of the constraint enhancement factor. This second navigation constraint may differ from the first navigation constraint, and may include at least one characteristic that is enhanced relative to the first navigation constraint. The second navigation constraint may be more restrictive than the first navigation constraint because detecting constraint enhancement factors in or associated with the primary vehicle's environment may indicate that the primary vehicle may have at least one reduced navigation capability relative to normal operating conditions. Such reduced capability may include lower road traction (e.g., ice, snow, or water on the roadway; reduced tire pressure, etc.); impaired vision (e.g., rain, snow, dust, smoke, fog, etc., reducing the quality of captured images); impaired detection capabilities (e.g., sensor malfunction or partial malfunction, reduced sensor performance, etc.), or any other reduction in the primary vehicle's ability to navigate in response to a detected navigation state.

[0337] If at least one constraint enhancement factor is detected in step 1507, and at least one constraint has been enhanced in step 1509, a navigation action for the primary vehicle can be determined in step 1511. The navigation action for the primary vehicle can be based on the identified navigation state and can satisfy a second navigation (i.e., enhanced) constraint. The navigation action can be implemented in step 1513 by inducing at least one adjustment to the navigation actuator of the primary vehicle in response to the determined navigation action.

[0338] As discussed, the use of navigation constraints and enhanced navigation constraints can be employed by a trained (e.g., through machine learning) or untrained (e.g., a system programmed to respond with predetermined actions in response to a specific navigation state) navigation system. In the case of a trained navigation system, the availability of enhanced navigation constraints for certain navigation situations can represent a mode switch from trained to untrained system responses. For example, a trained navigation network can determine the initial navigation action of the primary vehicle based on a first navigation constraint. However, the action taken by the vehicle can be different from the navigation action that satisfies the first navigation constraint. Instead, the action taken can satisfy an enhanced second navigation constraint and can be an action developed by an untrained system (e.g., as a response to the detection of a specific condition in the primary vehicle's environment, such as the presence of navigation constraint enhancement factors).

[0339] There are numerous examples of navigation constraints that can be generated, supplemented, or enhanced in response to the detection of constraint enhancement factors in the environment of the master vehicle. For example, where predefined navigation constraints include buffers associated with detected pedestrians, objects, vehicles, etc., and at least a portion of the buffers extends a certain distance from the detected pedestrian / object / vehicle, the enhanced navigation constraints (newly developed, retrieved from memory from a predetermined set, or generated as an enhanced version of a previously existing constraint) may include different or modified buffers. For example, the different or modified buffers may have a greater distance relative to the pedestrian / object / vehicle than the original or unmodified buffer relative to the detected pedestrian / object / vehicle. Therefore, given this enhanced constraint, the master vehicle can be forced to navigate further away from the detected pedestrian / object / vehicle, where appropriate constraint enhancement factors are detected in or relative to the master vehicle's environment.

[0340] At least one enhanced characteristic may also include other types of modifications to the navigation constraint characteristics. For example, an enhanced characteristic may include a reduction in velocity associated with at least one predefined navigation constraint. An enhanced characteristic may also include a reduction in the maximum permissible deceleration / acceleration associated with at least one predefined navigation constraint.

[0341] Navigation based on long-term planning

[0342] In some embodiments, the disclosed navigation system can respond not only to navigation states detected in the environment of the host vehicle, but also to determining one or more navigation actions based on long-term planning. For example, the system can consider the potential impact of one or more navigation actions, which are options for navigating relative to the detected navigation state, on future navigation states. Considering the impact of available actions on future states allows the navigation system to determine navigation actions not only based on the currently detected navigation state, but also based on long-term planning. Navigation using long-term planning techniques may be particularly suitable where the navigation system employs one or more reward functions as a technique for selecting navigation actions from available options. Potential rewards can be analyzed relative to available navigation actions that can be taken in response to the detected current navigation state of the host vehicle. Furthermore, potential rewards can also be analyzed relative to actions that can be taken in response to future navigation states, which are expected to result from available actions in response to the current navigation state. Therefore, the disclosed navigation system can, in certain circumstances, select a navigation action from available actions that can be taken in response to the current navigation state in response to the detected navigation state, even if the selected navigation action may not produce the highest reward. This is especially true when the system determines that the chosen action may lead to a future navigation state that gives rise to one or more potential navigation actions, where the potential actions offer a higher reward than the chosen action, or in some cases, than any action available relative to the current navigation state. This principle can be expressed more simply as: taking a less advantageous action now in order to generate a higher reward option in the future. Therefore, the disclosed navigation system capable of long-term planning can choose suboptimal short-term actions, where short-term losses in rewards indicated by long-term predictions lead to increases in long-term rewards.

[0343] Typically, autonomous driving applications involve a series of planning problems where the navigation system decides immediate actions to optimize a longer-term goal. For example, when a vehicle encounters a lane-changing situation at a roundabout, the navigation system can decide on an immediate acceleration or braking command to initiate navigation into the roundabout. While the immediate action at the roundabout in response to a detected navigation state might involve an acceleration or braking command in response to the detected state, the long-term goal is a successful lane change, and the long-term effect of the chosen command is the success or failure of the lane change. The planning problem can be addressed by decomposing it into two phases. First, supervised learning can be applied to predict the near future based on the present (assuming the predictor will be differentiable with respect to the current representation). Second, a recurrent neural network can be used to model the agent's complete trajectory, where unexplained factors are modeled as (additive) input nodes. This allows the use of supervised learning techniques and direct optimization of the recurrent neural network to determine solutions to the long-term planning problem. This approach also enables the learning of robust policies by incorporating adversarial elements into the environment.

[0344] The two most fundamental elements of an autonomous driving system are sensing and planning. Sensing processes find a compact representation of the current state of the environment, while planning processes determine what actions to take to optimize future goals. Supervised machine learning techniques are useful for solving the sensing problem. Machine learning algorithm frameworks can also be used for the planning part, especially reinforcement learning (RL) frameworks, such as those mentioned above.

[0345] RL can be executed sequentially in consecutive rounds. In round t, the planner (also known as the agent or driving strategy module 803) can observe the state s. t ∈S, which represents the agent and the environment. Then it should determine the action a. t ∈A. After performing this action, the agent receives an immediate reward. And move to the new state s t+1 As an example, the primary vehicle may include an adaptive cruise control (ACC) system, where the vehicle should autonomously apply acceleration / braking to maintain an appropriate distance from the vehicle in front while maintaining smooth driving. This state can be modeled as a pair Where x t It is the distance to the car in front, and v t It is the speed of the main vehicle relative to the vehicle in front. The action... It will be the acceleration command (where if a) t (If <0, the main vehicle decelerates). The reward may depend on |a t |(reflects driving smoothness) and s t(A function reflecting the safe distance maintained between the lead vehicle and the vehicle in front). The planner's goal is to maximize the cumulative reward (which can be up to the time horizon or the discounted sum of future rewards). To do this, the planner can rely on policy π: S→A, which maps states to actions.

[0346] Supervised learning (SL) can be viewed as a special case of RL, where s t It is sampled from some distribution on S, and the reward function can have In the form of, It is the loss function, and the learner observes y. t The value of y t To view status s t The optimal action to take (which may be noisy). There may be several differences between the typical RL model and the special case SL, and these differences make the typical RL problem more challenging.

[0347] In some cases of SolidWorks (SL), the actions (or predictions) taken by the learner may have no impact on the environment. In other words, s t+1 and a t They are independent. This has two important implications. First, in SL, samples (s1, y1), ..., (s m y m The states (s) can be collected in advance before the search for a strategy (or predictor) with good accuracy relative to the samples can begin. Conversely, in RL, the states (s) are collected before the search for a strategy (or predictor) with good accuracy relative to the samples can begin. t+1 Typically, the choice depends on the action taken (and the previous state), which in turn depends on the policy used to generate that action. This binds the data generation process to the policy learning process. Secondly, because actions in SL do not affect the environment, the choice of a... t The contribution to the performance of π is local. Specifically, a t It only affects the value of the immediate reward. In contrast, in RL, an action taken in round t may have a long-term impact on the reward value in future rounds.

[0348] In SL, the knowledge of the "correct" answer is y t Along with rewards The shape together can provide for a t Having complete knowledge of the rewards of all possible choices makes it possible to calculate the reward relative to a. tThe derivative of the reward. Conversely, in RL, the "one-off" value of the reward may be all that is observable for the specific choice of action taken. This can be called "bandit" feedback. This is one of the most important reasons for the need for "exploration" as part of long-term navigation planning, because in RL-based systems, if only "bandit" feedback is available, the system may not always know whether the action taken is the best action to take.

[0349] Many RL algorithms rely at least in part on the mathematical optimization model of Markov Decision Processes (MDPs). The Markov assumption is: given s... t and a t s t+1 The distribution of the policy is completely determined. Based on the stable distribution over the MDP states, a closed-form expression for the cumulative reward of a given policy is given. The stable distribution of the policy can be expressed as a solution to a linear programming problem. This leads to two classes of algorithms: 1) optimization relative to the primal problem, known as policy search; and 2) optimization relative to the dual problem, whose variable is called the value function V. π If the MDP starts from an initial state s and selects actions from there based on π, then the value function determines the expected cumulative reward. The relevant quantity is the state-action value function Q. π (s, a), assuming starting from state s, immediately choosing action a, and from there choosing actions according to π, the state-action value function determines the cumulative reward. The Q-function may lead to a characterization of the optimal policy (using the Bellman equation). In particular, the Q-function may indicate that the optimal policy is a deterministic function from S to A (in fact, it may be characterized as a "greedy" policy relative to the optimal Q-function).

[0350] One potential advantage of the MDP model is that it allows coupling the future to the present using a Q-function. For example, suppose the master vehicle is currently in state s, Q... π The value of (s, a) can indicate the impact of performing action a on the future. Therefore, the Q function can provide a local measure of the quality of action a, making the RL problem more similar to the SL scenario.

[0351] Many RL algorithms approximate the V or Q function in one way or another. Value iterative algorithms (such as Q-learning algorithms) may rely on the fact that the V and Q functions of the optimal policy can be fixed points of some operators derived from the Bellman equations. Actor-critic policy iterative algorithms aim to learn the policy in an iterative manner, where at iteration t, the "criterionist" estimates... And based on this estimate, the "actors" improved their strategies.

[0352] Despite the mathematical advantages of MDPs and the convenience of switching to Q-function representations, this approach may have several limitations. For example, the approximate concept of Markov behavioral states may be all that can be found in some cases. Furthermore, state transitions depend not only on the agent's actions but also on the actions of other actors in the environment. For instance, in the ACC example mentioned above, while the dynamics of the autonomous vehicle may be Markovian, the next state may depend on the actions of the driver of another vehicle, which are not necessarily Markovian. One possible solution to this problem is to use a partially observed MDP, where it is assumed that Markovian states exist, but what is observed is a distribution based on the hidden states.

[0353] A more direct approach could be to consider game theory generalizations of MDPs (e.g., stochastic game frameworks). In fact, algorithms used for MDPs can be generalized to multi-agent games (e.g., min-max-Q learning or Nash-Q learning). Other approaches could include explicit modeling of other participants and vanishing regret learning algorithms. Learning in multi-agent scenarios can be more complex than in single-agent scenarios.

[0354] A second constraint on the Q-function representation can arise by deviating from the tabular setting. The tabular setting occurs when the number of states and actions is small, so Q can be represented as a table with |S| rows and |A| columns. However, if the natural representations of S and A involve Euclidean space, and the state and action spaces are discrete, the number of states / actions can be exponential in dimensionality. In this case, adopting the tabular setting may not be practical. Instead, the Q-function can be approximated by a function derived from the category of parameter assumptions (e.g., a neural network of a certain architecture). For example, a deep Q-network (DQN) learning algorithm can be used. In DQN, the state space can be continuous, while the action space can still be a small, discrete set. Methods for handling continuous action spaces may exist, but they may rely on approximations of the Q-function. In any case, the Q-function can be complex and sensitive to noise, thus potentially posing a challenge to learning.

[0355] A different approach could be to use recurrent neural networks (RNNs) to solve RL problems. In some cases, RNNs can be combined with the concept of multi-agent games and robustness to adversarial environments derived from game theory. Furthermore, this approach can be one that does not explicitly rely on any Markov assumptions.

[0356] The following describes in more detail the method of navigation through prediction-based planning. In this method, the state space S can be assumed to be... A subset of, and the action space A is A subset of. This may be a natural representation in many applications. As mentioned above, there may be two key differences between RL and SL: (1) because past actions affect future rewards, information from the future may need to be propagated back to the past; (2) the “robbery” nature of rewards may blur the dependency between state, action and reward, which can complicate the learning process.

[0357] As a first step in this method, an interesting problem can be observed where the robbery nature of the reward is not the issue. For example, the reward value of the ACC application (discussed in more detail below) may be differentiable with respect to the current state and action. In fact, even if the reward is given in a "robbery" manner, learning a differentiable function... Make The problem can be a relatively straightforward SL problem (e.g., a one-dimensional regression problem). Therefore, the first step of this method could be to define the reward as a function. The function is differentiable with respect to s and a, or the first step of this method could be to use a regression learning algorithm to learn the differentiable function. This function minimizes at least some regression loss on the sample, where the instance vector is... And the target scalar is r t In some cases, exploratory elements can be used to create training sets.

[0358] A similar idea can be used to address the connection between the past and the future. For example, suppose we can learn differentiable functions. Make Learning such a function can be characterized as a learning problem (SL). It can be considered a near-future predictor. Next, the parametric function π can be used. θ Let S→A be used to describe the strategy of mapping from S to A. Let π... θ Representing it as a neural network allows the use of recurrent neural networks (RNNs) to represent segments of an agent running for T rounds, where the next state is defined as... Here, It can be defined by the environment and can express unpredictable aspects of the near future. t+1 Dependent on s in a differentiable manner t and a t The fact that this enables a link between future rewards and past actions. The policy function π θ The parameter vector can be learned by backpropagation on the obtained RNN. Note that it is not necessary to learn the parameter vector of v. tExplicit probability assumptions are imposed on it. In particular, no Markov relation requirement is needed. Instead, it can rely on the recurrent network to propagate “sufficient” information between the past and future. Intuitively, It can describe the predictable part of the near future, while v t It can express unpredictable aspects that may arise due to the behavior of other participants in the environment. The learning system should learn strategies robust to the behavior of other participants. If ||v t Larger relationships between past actions and future rewards can be too noisy for learning meaningful strategies. Explicitly expressing the system's dynamics in a transparent manner makes it easier to incorporate prior knowledge. For example, prior knowledge can be simplified in its definition. The problem.

[0359] As mentioned above, the learning system can benefit from robustness relative to adversarial environments, such as the environment of the primary vehicle, which may include multiple other drivers who might act in unexpected ways. In the absence of v t In models that impose probabilistic assumptions, it can be considered that the choice of v is adversarial. t The environment. In some cases, it is possible to target μ. t Imposing constraints is necessary, otherwise the opposing side might make the planning problem difficult or even impossible. One natural constraint might be requiring a constant to limit ||μ. t ||.

[0360] Robustness to adversarial environments could be useful in autonomous driving applications. Selecting μ in an adversarial manner... t It can even accelerate the learning process because it focuses the learning system on robust, optimal strategies. This concept can be illustrated using a simple game theory approach. The state is... Action as And the instantaneous loss function is 0.1|a t |+[|s t |-2] + , where [x] + =max{x, 0} is the ReLU (corrected linear unit) function. The next state is s. t+1 =s t +a t +v t , where v t ∈[-0.5, 0.5] represents the adversarial choice made against the environment. Here, the optimal policy can be written as a two-layer network with ReLU: a t =-[s t -1.5] + +[-s t -1.5] + It was observed that when |st When |∈(1.5, 2], the optimal action may have a larger immediate loss than the action a=0. Therefore, the system can plan for the future and can rely on more than just immediate loss. It is observed that the loss is related to a t The derivative is 0.1sign(a) t ), and about s t The derivative is 1[|s t |>2]sign(s t ). In s t In the case of ∈(1.5, 2], v t The adversarial choice will set v t =0.5, therefore, whenever a t >1.5-s t In round t+1, there can be a non-zero loss. In this case, the derivative of the loss can be directly propagated back to a. t Therefore, in a t In the case that the choice is not optimal, v t Adversarial choices can help navigation systems obtain non-zero backpropagation messages. This relationship can help navigation systems select current actions based on the expectation that the current action (even if it leads to a suboptimal reward or even a loss) will provide opportunities for more optimal actions that lead to higher rewards in the future.

[0361] This approach can be applied to virtually any navigation situation that might arise. The following description applies to an example approach: Adaptive Cruise Control (ACC). In the ACC problem, the primary vehicle might attempt to maintain an appropriate distance from a target vehicle ahead (e.g., 1.5 seconds from the target vehicle). Another objective might be to drive as smoothly as possible while maintaining the desired clearance. A model representing this situation can be defined as follows. The state space is... And the motion space is The first coordinate of the state is the speed of the target car, the second coordinate is the speed of the master vehicle, and the last coordinate is the distance between the master vehicle and the target vehicle (e.g., the position of the master vehicle along the road curve minus the position of the target vehicle). The action taken by the master vehicle is to accelerate, and this can be expressed as 'a'. t Let τ represent the time difference between consecutive rounds. While τ can be set to any suitable quantity, in one example, τ could be 0.1 seconds. Position s t It can be represented as And the (unknown) acceleration of the target vehicle can be expressed as

[0362] The complete dynamics of the system can be described as follows:

[0363]

[0364] This can be described as the sum of two vectors:

[0365]

[0366] The first vector represents the predictable portion, while the second vector represents the unpredictable portion. The reward for round t is defined as follows:

[0367] in

[0368] The first term results in a penalty for non-zero acceleration, thus encouraging smooth driving. The second term depends on the distance x to the target car. t Distance from expectations The ratio between the desired distance and the braking distance is defined as the maximum value between a distance of 1 meter and a braking distance of 1.5 seconds. In some cases, this ratio may be exactly 1, but as long as the ratio is in the range of [0.7, 1.3], the strategy may waive any penalty, which may allow the primary vehicle to relax its navigation—a feature that may be important in achieving smooth driving.

[0369] By implementing the methods outlined above, the navigation system of the main vehicle (e.g., through the operation of the driving strategy module 803 within the processing unit 110 of the navigation system) can select an action in response to the observed state. The selected action can be based not only on the analysis of rewards associated with responsive actions available relative to the sensed navigation state, but also on consideration and analysis of future states, potential actions in response to future states, and rewards associated with potential actions.

[0370] Figure 16 An algorithmic approach to navigation based on detection and long-term planning is illustrated. For example, in step 1601, at least one processing device 110 of the main vehicle's navigation system may receive multiple images. These images may capture scenes representing the environment of the main vehicle and may be provided by any of the aforementioned image capture devices (e.g., cameras, sensors, etc.). Analyzing one or more of these images in step 1603 enables at least one processing device 110 to identify the current navigation state associated with the main vehicle (as described above).

[0371] In steps 1605, 1607, and 1609, various potential navigation actions in response to the sensed navigation state can be determined. These potential navigation actions (e.g., first navigation action to Nth available navigation action) can be determined based on the sensed state and the long-term goal of the navigation system (e.g., completing a lane change, smoothly following the vehicle in front, overtaking a target vehicle, avoiding an object in the roadway, slowing down in response to a detected stop sign, avoiding an intruding target vehicle, or any other navigation action that can advance the system's navigation goal).

[0372] For each of the identified potential navigation actions, the system can determine an expected reward. The expected reward can be determined according to any of the techniques described above and may include analysis of one or more reward functions for a particular potential action. For each potential navigation action (e.g., first, second, and Nth) identified in steps 1605, 1607, and 1609, expected rewards 1606, 1608, and 1610 can be determined, respectively.

[0373] In some cases, the primary vehicle's navigation system may select from available potential actions based on values ​​associated with expected rewards 1606, 1608, and 1610 (or any other type of indicator of expected reward). For example, in some situations, the action that produces the highest expected reward may be selected.

[0374] In other cases, particularly when the navigation system is performing long-term planning to determine the primary vehicle's navigation actions, the system may not select the potential action that yields the highest expected reward. Instead, the system can look to the future to analyze whether there is an opportunity to achieve a higher reward later if a lower-reward action is selected in response to the current navigation state. For example, for any or all of the potential actions determined in steps 1605, 1607, and 1609, future states can be determined. Each future state determined in steps 1613, 1615, and 1617 can represent a future navigation state expected to be obtained based on the current navigation state, modified by the corresponding potential action (e.g., the potential actions determined in steps 1605, 1607, and 1609).

[0375] For each of the future states predicted in steps 1613, 1615, and 1617, one or more future actions (as navigation options available in response to the determined future state) can be identified and evaluated. In steps 1619, 1621, and 1623, for example, values ​​of expected rewards or any other type of indicator associated with the one or more future actions can be developed (e.g., based on one or more reward functions). The expected reward associated with the one or more future actions can be evaluated by comparing the values ​​of the reward functions associated with each future action or by comparing any other indicator associated with the expected reward.

[0376] In step 1625, the primary vehicle's navigation system may select a navigation action based on a comparison of anticipated rewards, not only based on potential actions identified relative to the current navigation state (e.g., in steps 1605, 1607, and 1609), but also based on anticipated rewards determined as a result of potential future actions available in response to a predicted future state (e.g., determined in steps 1613, 1615, and 1617). The selection at step 1625 may be based on the option and reward analysis performed in steps 1619, 1621, and 1623.

[0377] The selection of a navigation action at step 1625 can be based solely on a comparison of the expected rewards associated with future action options. In this case, the navigation system can select an action for the current state based solely on a comparison of the expected rewards derived from actions to potential future navigation states. For example, the system could select a potential action identified at steps 1650, 1610, and 1609 that is associated with the highest future reward value determined by the analysis at steps 1619, 1621, and 1623.

[0378] The selection of a navigation action in step 1625 can also be based solely on a comparison of current action options (as described above). In this case, the navigation system can select a potential action identified in steps 1605, 1607, or 1609 that is associated with the highest expected reward 1606, 1608, or 1610. This selection can be performed with little or no consideration of future navigation states or the future expected reward of the navigation action available in response to an expected future navigation state.

[0379] On the other hand, in some cases, the selection of a navigation action in step 1625 can be based on a comparison of the expected rewards associated with both future action options and current action options. In fact, this could be based on one of the navigation principles of long-term planning. For example, the expected reward for a future action can be analyzed to determine if there are any expected rewards that authorize a selection of a lower-reward action in response to the current navigation state in order to achieve a potentially higher reward in response to a subsequent navigation action that is expected to be available in response to a future navigation state. As an example, the value of expected reward 1606 or other indicators could indicate the highest expected reward among rewards 1606, 1608, and 1610. On the other hand, expected reward 1608 could indicate the lowest expected reward among rewards 1606, 1608, and 1610. Instead of simply selecting the potential action determined in step 1605 (i.e., the action that results in the highest expected reward 1606), the analysis of the future state, potential future actions, and future rewards can be used to select a navigation action in step 1625. In one example, it can be determined that the reward identified in step 1621 (in response to at least one future action to a future state determined in step 1615, which is based on a second potential action determined in step 1607) may be higher than the expected reward 1606. Based on this comparison, the second potential action determined in step 1607 can be selected instead of the first potential action determined in step 1605, even though the expected reward 1606 is higher than the expected reward 1608. In one example, the potential navigation action determined in step 1605 may include merging lanes in front of the detected target vehicle, while the potential navigation action determined in step 1607 may include merging lanes behind the target vehicle. While the expected reward 1606 for merging lanes in front of the target vehicle may be higher than the expected reward 1608 associated with merging lanes behind the target vehicle, it can be determined that merging lanes behind the target vehicle may result in a future state for which there may be action options that produce potential rewards higher than the expected rewards 1606, 1608, or other available actions based on the current, sensed navigation state.

[0380] The selection from potential actions at step 1625 can be based on any suitable comparison of anticipated rewards (or any other measure or indicator of the benefit of one potential action relative to another). In some cases, as described above, if the second potential action is expected to provide at least one future action associated with an anticipated reward higher than that associated with the first potential action, the second potential action can be selected over the first. In other cases, more complex comparisons can be employed. For example, the reward associated with an action option in response to a anticipated future state can be compared with more than one anticipated reward associated with the identified potential action.

[0381] In some scenarios, if at least one future action is expected to produce a reward higher than any reward expected as a result of a potential action to the current state (e.g., expected rewards 1606, 1608, 1610, etc.), then the action based on the expected future state and the expected reward can influence the selection of a potential action to the current state. In some cases, the future action option that produces the highest expected reward (e.g., from the expected reward associated with a potential action to the sensed current state, and from the expected reward associated with a potential future action option relative to a potential future navigation state) can be used as guidance for selecting a potential action to the current navigation state. That is, after identifying the future action option that produces the highest expected reward (or a reward above a predetermined threshold, etc.), step 1625 can select a potential action that will lead to a future state associated with the identified future action that produces the highest expected reward.

[0382] In other cases, the selection of available actions can be based on the difference determined between expected rewards. For example, if the difference between the expected reward associated with the future action determined in step 1621 and expected reward 1606 is greater than the difference between expected reward 1608 and expected reward 1606 (assuming a difference in positive signs), then the second potential action determined in step 1607 can be selected. In another example, if the difference between the expected reward associated with the future action determined in step 1621 and the expected reward associated with the future action determined in step 1619 is greater than the difference between expected reward 1608 and expected reward 1606, then the second potential action determined in step 1607 can be selected.

[0383] Several examples of selecting from potential actions for the current navigation state have been described. However, any other suitable comparison techniques or criteria can be used to select available actions through long-term planning based on action and reward analysis extending to anticipated future states. Furthermore, although Figure 16 This represents two layers in long-term planning analysis (e.g., the first layer considers the rewards generated by potential actions in response to the current state, while the second layer considers the rewards generated by future action options in response to the anticipated future state), but analysis based on more layers is possible. For example, instead of basing long-term planning analysis on one or two layers, three, four, or more layers of analysis can be used to select from available potential actions in response to the current navigation state.

[0384] After selecting from potential actions in response to the sensed navigation state, in step 1627, at least one processor may cause at least one adjustment to the navigation actuator of the host vehicle in response to the selected potential navigation action. The navigation actuator may include any suitable device for controlling at least one aspect of the host vehicle. For example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0385] Navigation based on inference of other parties' aggression

[0386] Target vehicles can be monitored by analyzing acquired image streams to identify indicators of driving aggression. This paper describes aggression as a qualitative or quantitative parameter, but other characteristics can be used: perceived levels of attention (potential driver injury, distraction from a mobile phone, drowsiness, etc.). In some cases, a target vehicle can be perceived as having a defensive posture, and in others, a more aggressive posture can be determined. Navigation actions can be selected or developed based on aggression indicators. For example, in some cases, relative speed, relative acceleration, increase in relative acceleration, following distance, etc., relative to the host vehicle can be tracked to determine whether the target vehicle is aggressive or defensive. For instance, if a target vehicle is determined to have an aggression level exceeding a threshold, the host vehicle may be inclined to yield to the target vehicle. The level of aggression of a target vehicle can also be identified based on its behavior relative to one or more obstacles in or near its path (e.g., vehicles ahead, obstacles in the road, traffic lights, etc.).

[0387] As an introduction to this concept, an exemplary experiment will be described relative to a primary vehicle merging into a roundabout, where the navigation objective is to cross and exit the roundabout. This scenario can begin with the primary vehicle approaching the entrance to the roundabout and end with the primary vehicle reaching the exit (e.g., a second exit). Success can be measured based on whether the primary vehicle consistently maintains a safe distance from all other vehicles, whether the primary vehicle completes the route as quickly as possible, and whether the primary vehicle follows a smooth acceleration strategy. In this example, N T A target vehicle can be randomly placed on a roundabout. To model a mixture of adversarial and typical behaviors, the target vehicle can be modeled with probability p using an "aggressive" driving strategy, whereby the aggressive target vehicle accelerates when the master vehicle attempts to merge into its lane. With probability 1-p, the target vehicle can be modeled with a "defensive" driving strategy, whereby the target vehicle decelerates and allows the master vehicle to merge into its lane. In this experiment, p = 0.5, and the master vehicle's navigation system may not provide information about the types of other drivers. The types of other drivers can be randomly selected at the start of the segment.

[0388] Navigation state can be represented as the speed and position of the primary vehicle (agent), and the position, speed, and acceleration of the target vehicles. Maintaining target acceleration observation may be important to differentiate between aggressive and defensive drivers based on the current state. All target vehicles may move along a one-dimensional curve with a circular path as its contour. The primary vehicle may move along its own one-dimensional curve, which intersects with the target vehicle's curve at a merge point, and this point is the starting point of both curves. To model rational driving, the absolute values ​​of the accelerations of all vehicles can be bounded to a constant. Since reverse driving is not allowed, speed can also be conveyed via ReLU. Note that by disallowing reverse driving, long-term planning may become necessary, as the agent will not regret its past actions.

[0389] As mentioned above, the next state s t+1 It can be decomposed into predictable parts and unpredictable parts v t The sum. Expression It can represent the dynamics of vehicle position and velocity (which can be well defined in a differentiable way), while v t It can represent the acceleration of the target vehicle. This can be verified. It can be represented as a combination of ReLU functions over affine transformations, therefore its relation to s t and a t It is differentiable. The vector v t It can be defined by the simulator in a non-differentiable manner, and can exhibit aggressive behavior towards some targets and defensive behavior towards others. Figure 17A and 17B The image shows two frames from this simulator. In this exemplary experiment, the master vehicle 1701 learns to slow down as it approaches the entrance to the roundabout. It also learns to give way to aggressive vehicles (e.g., vehicles 1703 and 1705) and to safely continue driving when merging ahead of defensive vehicles (e.g., vehicles 1706, 1708, and 1710). Figure 17A and 17B In the example shown, the navigation system of the primary vehicle 1701 does not provide the type of the target vehicle. Instead, it determines whether a particular vehicle is identified as aggressive or defensive by inference based on observed (e.g., the target vehicle's) position and acceleration. Figure 17A Based on position, speed, and / or relative acceleration, the primary vehicle 1701 can determine that the target vehicle 1703 has an aggressive tendency, and therefore the primary vehicle 1701 can stop and wait for the target vehicle 1703 to pass rather than attempting to merge in front of the target vehicle 1703. However, in Figure 17BIn the process, target vehicle 1701 identifies that target vehicle 1710 traveling behind vehicle 1703 exhibits a defensive tendency (also based on the observed position, speed and / or relative acceleration of vehicle 1710), and thus successfully merges into the lane in front of target vehicle 1710 and behind target vehicle 1703.

[0390] Figure 18 A flowchart is provided representing an exemplary algorithm for navigating a master vehicle based on predicted aggression from other vehicles. Figure 18 In the example, the level of aggression associated with at least one target vehicle can be inferred based on the behavior of the target vehicle as observed by objects in the environment relative to the target vehicle. For example, in step 1801, at least one processing device of the main vehicle navigation system (e.g., processing device 110) can receive multiple images representing the environment of the main vehicle from a camera associated with the main vehicle. In step 1803, analysis of one or more of the received images enables at least one processor to identify a target vehicle (e.g., vehicle 1703) in the environment of the main vehicle 1701. In step 1805, analysis of one or more of the received images enables at least one processing device to identify at least one obstacle to the target vehicle in the environment of the main vehicle. The object may include debris in the roadway, parking lights / traffic lights, pedestrians, another vehicle (e.g., a vehicle traveling in front of the target vehicle, a parked vehicle, etc.), boxes in the road, roadblocks, curbs, or any other type of object that may be encountered in the environment of the main vehicle. In step 1807, analysis of one or more of the received images enables at least one processing device to determine at least one navigation characteristic of the target vehicle relative to at least one identified obstacle.

[0391] Various navigation characteristics can be used to infer the aggression level of a detected target vehicle in order to develop an appropriate navigation response to the target vehicle. For example, such navigation characteristics may include the relative acceleration between the target vehicle and at least one identified obstacle, the distance of the target vehicle from the obstacle (e.g., the following distance of the target vehicle behind another vehicle), and / or the relative speed between the target vehicle and the obstacle, etc.

[0392] In some embodiments, the navigation characteristics of a target vehicle can be determined based on the outputs from sensors associated with the host vehicle (e.g., radar, speed sensors, GPS, etc.). However, in some cases, the navigation characteristics of a target vehicle can be determined partly or entirely based on the analysis of images of the host vehicle's environment. For example, image analysis techniques described above and, for example, in U.S. Patent No. 9,168,868, incorporated herein by reference, can be used to identify a target vehicle within the host vehicle's environment. Furthermore, monitoring the position of the target vehicle in the captured images over time and / or monitoring the position of one or more features associated with the target vehicle (e.g., taillights, headlights, bumpers, wheels, etc.) in the captured images can make it possible to determine the relative distance, velocity, and / or acceleration between the target vehicle and the host vehicle or between the target vehicle and one or more other objects in the host vehicle's environment.

[0393] The level of aggression of an identified target vehicle can be inferred from any suitable observed navigation characteristics of the target vehicle, or any combination of observed navigation characteristics. For example, a determination of aggression can be made based on any observed characteristic and one or more predetermined threshold levels or any other suitable qualitative or quantitative analysis. In some embodiments, a target vehicle may be considered aggressive if it is observed following a host vehicle or another vehicle at a distance less than a predetermined aggressive distance threshold. Conversely, a target vehicle observed following a host vehicle or another vehicle at a distance greater than a predetermined defensive distance threshold may be considered defensive. The predetermined aggressive distance threshold does not need to be the same as the predetermined defensive distance threshold. Furthermore, either or both of the predetermined aggressive distance threshold and the predetermined defensive distance threshold may include a range of values, rather than a bright line value. Moreover, neither the predetermined aggressive distance threshold nor the predetermined defensive distance threshold needs to be fixed. Instead, these values ​​or ranges of values ​​can change over time, and different thresholds / ranges of thresholds can be applied based on the observed characteristics of the target vehicle. For example, the applied threshold may depend on one or more other characteristics of the target vehicle. Higher observed relative speeds and / or accelerations may authorize the application of larger thresholds / ranges. Conversely, lower relative velocity and / or acceleration (including zero relative velocity and / or acceleration) can authorize the application of smaller distance thresholds / ranges when making aggressive / defensive inferences.

[0394] Aggression / defensivity inferences can also be based on relative speed and / or relative acceleration thresholds. A target vehicle is considered aggressive if its relative speed and / or relative acceleration relative to another vehicle exceeds a predetermined level or range. Conversely, a target vehicle is considered defensive if its relative speed and / or relative acceleration relative to another vehicle is below a predetermined level or range.

[0395] While aggression / defensiveness can be determined solely based on any observed navigational characteristics, the determination can also depend on any combination of observed characteristics. For example, as mentioned above, in some cases, a target vehicle may be considered aggressive simply because it is observed following another vehicle at a distance below a certain threshold or range. However, in other cases, a target vehicle may be considered aggressive if it follows another vehicle at a relative speed and / or relative acceleration greater than the predetermined amount or range (which may be the same as or different from the threshold applied when determining based solely on distance). Similarly, a target vehicle may be considered defensive simply because it is observed following another vehicle at a distance greater than a certain threshold or range. However, in other cases, a target vehicle may be considered defensive if it follows another vehicle at a relative speed and / or relative acceleration less than the predetermined amount or range (which may be the same as or different from the threshold applied when determining based solely on distance). If, for example, a vehicle accelerates or decelerates at a speed exceeding 0.5G (e.g., a rate of change of acceleration (jerk) of 5 m / s³), a vehicle has a lateral acceleration of 0.5G when changing lanes or on a curve, a vehicle causes another vehicle to do any of the above, a vehicle changes lanes and causes another vehicle to give way with a deceleration of 0.3G or a rate of change of acceleration of 3 m / s³ or more, and / or a vehicle changes two lanes without stopping, then system 100 can determine aggressiveness / defensiveness.

[0396] It should be understood that a reference to a quantity exceeding a range can indicate that the quantity exceeds all values ​​associated with that range or falls within that range. Similarly, a reference to a quantity falling below a range can indicate that the quantity is below all values ​​associated with that range or falls within that range. Furthermore, while examples of using distance, relative acceleration, and relative velocity for making aggressive / defensive inferences are described, any other suitable quantity can be used. For example, calculations of the time of collision, or any indirect indicator of the target vehicle's distance, acceleration, and / or velocity, can be used. It should also be noted that while the examples above focus on a target vehicle relative to other vehicles, aggressive / defensive inferences can be made by observing the target vehicle's navigational characteristics relative to any other type of obstacle (e.g., pedestrians, roadblocks, traffic lights, debris, etc.).

[0397] Back Figure 17A and 17B In the example shown, when the primary vehicle 1701 approaches a roundabout, the navigation system (including at least one of its processing devices) can receive an image stream from a camera associated with the primary vehicle. Based on the analysis of one or more received images, any one of the target vehicles 1703, 1705, 1706, 1708, and 1710 can be identified. Furthermore, the navigation system can analyze the navigation characteristics of one or more identified target vehicles. The navigation system can recognize that the gap between target vehicles 1703 and 1705 represents a potential first opportunity to merge into the roundabout. The navigation system can analyze target vehicle 1703 to determine an aggression indicator associated with target vehicle 1703. If target vehicle 1703 is deemed aggressive, the primary vehicle navigation system may choose to yield to vehicle 1703 instead of merging in front of vehicle 1703. On the other hand, if target vehicle 1703 is deemed defensive, the primary vehicle navigation system may attempt to complete the lane merge in front of vehicle 1703.

[0398] When the primary vehicle 1701 approaches the roundabout, at least one processing device of the navigation system can analyze the captured images to determine navigation characteristics associated with the target vehicle 1703. For example, based on the images, it can be determined that vehicle 1703 is following vehicle 1705 at a distance that provides sufficient clearance for the primary vehicle 1701 to enter safely. In fact, it can be determined that vehicle 1703 is following vehicle 1705 at a distance exceeding an aggressive distance threshold, and therefore, based on this information, the primary vehicle navigation system may tend to identify the target vehicle 1703 as defensive. However, in some cases, as described above, more than one navigation characteristic of the target vehicle can be analyzed when making an aggressive / defensive determination. Further analysis may allow the primary vehicle navigation system to determine that when the target vehicle 1703 is following behind the target vehicle 1705 at a non-aggressive distance, vehicle 1703 has a relative velocity and / or relative acceleration relative to vehicle 1705 that exceeds one or more thresholds associated with aggressive behavior. In fact, the primary vehicle 1701 can determine that the target vehicle 1703 is accelerating relative to vehicle 1705 and approaching the gap existing between vehicles 1703 and 1705. Based on further analysis of relative speed, acceleration, and distance (and even the rate at which the gap between vehicles 1703 and 1705 is closing), the primary vehicle 1701 can determine that the target vehicle 1703 is behaving aggressively. Therefore, although there may be a sufficient gap that the primary vehicle can safely navigate into, the primary vehicle 1701 can anticipate that a lane change in front of the target vehicle 1703 will result in an aggressively navigating vehicle immediately behind the primary vehicle. Furthermore, based on behavior observed through image analysis or other sensor outputs, the target vehicle 1703 can be expected to either continue accelerating toward the primary vehicle 1701 or continue moving toward the primary vehicle 1701 with a non-zero relative speed if the primary vehicle 1701 were to change lanes in front of vehicle 1703. From a safety perspective, this situation may be undesirable and could also cause discomfort to the passengers of the primary vehicle. For this reason, as Figure 17B As shown, the main vehicle 1701 may choose to give way to vehicle 1703 and merge into the roundabout behind vehicle 1703 and in front of vehicle 1710, which is considered defensive based on the analysis of one or more of its navigation characteristics.

[0399] Back Figure 18In step 1809, at least one processing device of the main vehicle's navigation system can determine a navigation action of the main vehicle (e.g., merging ahead of vehicle 1710 and behind vehicle 1703) based on at least one navigation characteristic of the target vehicle relative to the identified obstacle. To implement this navigation action (in step 1811), at least one processing device can cause at least one adjustment to the main vehicle's navigation actuator in response to the determined navigation action. For example, braking can be applied to give way to... Figure 17A Vehicle 1703, and the accelerator can be applied in conjunction with the steering of the master vehicle's wheels to allow the master vehicle to enter the roundabout behind vehicle 1703, such as Figure 17B As shown.

[0400] As illustrated in the examples above, the navigation of the primary vehicle can be based on the navigation characteristics of the target vehicle relative to another vehicle or object. Alternatively, the navigation of the primary vehicle can be based solely on the navigation characteristics of the target vehicle without specifically referencing another vehicle or object. For example, in... Figure 18 At step 1807, analysis of multiple images captured from the environment of the primary vehicle enables the determination of at least one navigation characteristic of the identified target vehicle, indicating the level of aggression associated with the target vehicle. Navigation characteristics may include speed, acceleration, etc., which do not require reference to another object or target vehicle for determining aggression / defensiveness. For example, observed acceleration and / or velocity associated with the target vehicle exceeding a predetermined threshold or falling within or exceeding a numerical range may indicate aggressive behavior. Conversely, observed acceleration and / or velocity associated with the target vehicle falling below a predetermined threshold or falling within or exceeding a numerical range may indicate defensive behavior.

[0401] Of course, in some cases, to determine aggression / defense, observed navigation characteristics (e.g., position, distance, acceleration, etc.) can be referenced relative to the host vehicle. For example, observed navigation characteristics associated with the target vehicle's indication of aggression level may include an increase in relative acceleration between the target vehicle and the host vehicle, the following distance of the target vehicle behind the host vehicle, and the relative speed between the target vehicle and the host vehicle.

[0402] Navigation based on liability constraints

[0403] As described in the preceding sections, planned navigation actions can be tested against predetermined constraints to ensure compliance with certain rules. In some embodiments, this concept can be extended to consideration of potential accident liability. As explained below, the primary objective of autonomous navigation is safety. Since absolute safety may be impossible (e.g., at least because a particular master vehicle under autonomous control cannot control other vehicles around it—it can only control its own actions), using potential accident liability as a consideration in autonomous navigation, and effectively as a constraint on planned actions, can help ensure that a particular autonomous vehicle does not take any actions deemed unsafe—e.g., those actions for which potential accident liability might be attached to the master vehicle. If the master vehicle only takes safe actions that are determined not to cause an accident due to the master vehicle's own fault or liability, then a desired level of accident avoidance (e.g., less than 10 per hour) can be achieved. -9 ).

[0404] Challenges to most current approaches to autonomous driving include a lack of safety guarantees (or at least an inability to provide the desired level of safety) and a lack of scalability. Consider the problem of ensuring safe driving in multi-agent scenarios. Since society is unlikely to tolerate deaths in road accidents caused by machines, an acceptable level of safety is crucial for the acceptance of autonomous vehicles. While the goal might be to provide zero accidents, this may be impossible because accidents typically involve multiple agents, and situations can be conceived where the accident is entirely due to the fault of other agents. For example, as... Figure 19 As shown, the master vehicle 1901 is driving on a multi-lane highway. While the master vehicle 1901 can control its own actions relative to target vehicles 1903, 1905, 1907, and 1909, it cannot control the actions of the target vehicles around it. As a result, if vehicle 1905 suddenly cuts into the master vehicle's lane, for example, along the collision path, the master vehicle 1901 may be unable to avoid an accident with at least one target vehicle. To address this challenge, a typical response from autonomous vehicle practitioners is to employ a statistically driven approach, in which safety verification becomes more rigorous as more mileage data is collected.

[0405] However, to understand the nature of the problem with data-driven approaches to safety, the first thing to consider is that the probability of death from an accident involving (human) driving per hour is known to be 10. -6 A reasonable assumption is that, in order for society to accept machines replacing humans in driving tasks, the mortality rate should be reduced by three orders of magnitude, i.e., to 10 per hour. -9 The probability. This estimate is similar to the assumed airbag mortality rate and comes from aviation standards. For example, 10 -9It's the probability of a wing spontaneously detaching from the aircraft in mid-air. However, attempting to guarantee safety using data-driven statistical methods that provide additional confidence through accumulated flight mileage is impractical. Guaranteeing 10 [units of something] per hour... -9 The amount of data required to determine the probability of death is its reciprocal (i.e., 10). 9 The data is proportional to hours of data, amounting to approximately 30 billion miles. Furthermore, multi-agent systems interact with their environment and may not be verifiable offline (unless a realistic simulator is available, simulating real human driving (such as reckless driving) with its full richness and complexity—but validating such a simulator is even more difficult than creating a safe autonomous vehicle agent). Any changes to the planning and control software would require the same amount of new data collection, which is clearly impractical and unrealistic. Moreover, developing systems through data always suffers from a lack of interpretability and explainability of the actions taken—if an autonomous vehicle (AV) accident results in death, we need to know the cause. Therefore, a model-based approach to safety is needed, but existing "functional safety" and ASIL requirements in the automotive industry are not designed for multi-agent environments.

[0406] The second major challenge in developing safe driving models for autonomous vehicles is the need for scalability. The premise of AVs is not merely "building a better world," but rather the assumption that driverless mobility can be sustained at a lower cost than drivered mobility. This premise is always intertwined with the concept of scalability—that is, supporting the large-scale production of AVs (in the millions), and more importantly, supporting negligible incremental costs to enable them to drive in new cities. Therefore, the costs of computing and sensing are indeed significant, and the costs of validation and the ability to drive "everywhere," rather than in a select few cities, are also essential requirements for sustaining the business if AVs are to be manufactured at scale.

[0407] The problem with most current approaches lies in a brute-force mentality along three axes: (i) the required computational density, (ii) how high-definition maps are defined and created, and (iii) the required sensor specifications. This brute-force approach runs counter to scalability and shifts the focus to a future where unrestricted in-vehicle computing is ubiquitous, where the cost of building and maintaining HD maps becomes negligible and scalable, and where exotic, ultra-advanced sensors are developed, manufactured to automotive standards at negligible cost. While a future where any one of these scenarios is indeed possible, making all of them a reality is likely a low-probability event. Therefore, it is necessary to provide a formal model that combines safety and scalability into a socially acceptable and scalable AV program that supports millions of vehicles driving anywhere in developed countries.

[0408] The disclosed embodiments represent a solution that can provide a target safety level (or even exceed the safety target) and can be scaled up to systems including millions (or more) of autonomous vehicles. In terms of safety, a model called "Responsibility Sensitive Safety (RSS)" is introduced, which formalizes the concept of "accident attribution" in a way that is interpretable and explainable, and incorporates the concept of "responsibility" into the actions of the robotic agent. The definition of RSS is agnostic to how it is implemented—a key feature that facilitates the creation of a compelling global safety model. RSS is observed as follows (e.g.) Figure 19 (As shown) Incentives: Agents play an asymmetric role in an accident, where typically only one agent is responsible for and therefore liable for the accident. RSS models also include formal treatment of "cautious driving" under limited sensing conditions, where not all agents are always visible (e.g., due to occlusion). A primary goal of RSS models is to guarantee that an agent will never be "attributed" to or responsible for an accident. A model is only useful if it comes with an effective strategy that conforms to RSS (e.g., a function that maps "sensing states" to actions). For example, an action that seems innocent at the present moment could lead to a catastrophic event in the distant future ("butterfly effect"). RSS can help build a set of local constraints in the near future that guarantee (or at least practically guarantee) that future accidents will not occur due to the actions of the primary vehicle.

[0409] Another contribution revolves around the introduction of a “semantic” language, which includes units, measurements, and action spaces, and how to incorporate them into the specifications for planning, sensing, and actuation of AV. To gain a semantic understanding, in this context, consider how humans taking a driving course are instructed to think about “driving strategies.” These instructions are not geometric—they don’t take things like, “drive 13.7 meters at the current speed, then at 0.8 m / s…” 2 Instead of the form of "acceleration at a certain rate," these instructions are semantic in nature—"follow the car in front of you" or "overtake that car on your left." The typical language of human driving strategies is about longitudinal and lateral targets, not geometric units of acceleration vectors. Formal semantic language can be useful in several ways, including the computational complexity of planning that does not increase exponentially with time and the number of agents, the way safe and comfortable interactions are handled, how computations of sensing are defined, the specification of sensor modalities, and how they interact in fusion methods. (Semantic language-based) fusion methods can ensure that the RSS model can achieve the desired 10 driving hours per hour. -9 The probability of death, all of which are only for orders of magnitude 10 5 Offline verification was performed on a dataset of hours of driving data.

[0410] For example, in a reinforcement learning setting, a Q-function can be defined in the semantic space (e.g., a function that evaluates the long-term quality of an action a∈A when the agent is in state s∈S; given such a Q-function, the natural selection of actions could be to pick the action with the highest quality π(s) = argamx). a Q(s, a)), in this semantic space, the number of trajectories to be examined at any given time is in the order of 10. 4The space is defined as a boundary, regardless of the time frame used for planning. The signal-to-noise ratio in this space can be high, allowing efficient machine learning methods to successfully model the Q-function. In the case of computation on sensing, semantics can allow for the differentiation between errors affecting safety and errors affecting driving comfort. We define a PAC model (Probably Approximate Correct, borrowing Valiant's PAC learning terminology) for sensing, fit this model to the Q-function, and demonstrate how measurement errors can be incorporated into planning in a way that conforms to RSS while allowing for optimization of driving comfort. Semantic language may be important for the success of some aspects of this model, as other standard measurements of errors (such as errors relative to the global coordinate system) may not be compatible with the PAC sensing model. Furthermore, semantic language may be a significant contributing factor to the definition of HD maps, which can be constructed using low-bandwidth sensing data, thus enabling crowdsourcing and supporting scalability.

[0411] In summary, the disclosed embodiments may include a formal model covering the following key components of AV: sensing, planning, and motion. This model can help ensure that, from a planning perspective, incidents of AV's own fault do not occur. Furthermore, with the PAC sensing model, even with sensing errors, the described fusion method may require only a very reasonable amount of offline data collection to conform to the described safety model. Moreover, this model can combine security and scalability through semantic language, thus providing a complete approach to secure and scalable AV. Finally, it is worth noting that developing an accepted safety model that will be adopted by industry and regulatory bodies may be a prerequisite for the success of AV.

[0412] RSS models typically follow the classic sense-plan-action robot control approach. The sensing system is responsible for understanding the current state of the host vehicle's environment. The planning part is responsible for determining the optimal next movement based on available options for achieving a driving objective (e.g., how to move from the left lane to the right lane to exit a highway). This planning part may be referred to as the "driving strategy" and can be implemented by a set of hard-coded instructions, through a trained system (e.g., a neural network), or a combination thereof. The action part is responsible for implementing the plan (e.g., a system of actuators and one or more controllers for steering, accelerating, and / or braking the vehicle to implement the selected navigation actions). The embodiments described below focus primarily on the sensing and planning parts.

[0413] Accidents can stem from sensing or planning errors. Planning is a multi-agent effort, as other road users (humans and machines) react to the actions of the AV. Among other things, the described RSS model is designed to address the safety issues of the planning component. This can be termed multi-agent safety. In statistical methods, the probability of planning errors can be estimated “online.” That is, after each software update, billions of miles must be driven with the new version to provide an estimate of an acceptable level of frequency of planning errors. This is clearly impractical. As an alternative, the RSS model can provide a 100% (or nearly 100%) guarantee that the planning module will not commit errors attributable to the AV (formally defining the concept of “attribution”). The RSS model also provides an efficient method for its validation that does not rely on online testing.

[0414] Errors in sensing systems are often easier to verify because sensing can be independent of vehicle movement, and therefore we can use "offline" data to verify the probability of severe sensing errors. However, even collecting more than 10 9 Offline data from hours of driving is also challenging. As part of the description of the disclosed sensing system, a fusion method is described that can be validated using significantly less data.

[0415] The described RSS system can also be scaled to millions of vehicles. For example, the described semantic driving strategy and the applied safety constraints can be aligned with sensing and mapping requirements that can be scaled to millions of vehicles even with today's technology.

[0416] The fundamental building block of such a system is a thorough safety definition, i.e., the minimum standards that the AV system may need to comply with. In the following technical lemma, statistical methods used to verify AV systems are proven infeasible, even for verifying simple statements such as "the system experiences N incidents per hour." This means that model-based safety definitions are the only feasible tool for verifying AV systems.

[0417] Lemma 1: Let X be a probability space, and A be the event Pr(A) = p1 < 0.1. Suppose we sample from X... 1 sample, and let So

[0418] Pr(Z=0)≥e -2 .

[0419] We will use the inequality 1 - x ≥ e to prove that... -2x (Completeness is demonstrated in Appendix A.1), thus obtaining

[0420]

[0421] Corollary 1: Assume that AV system AV1 experiences an accident with a small but insufficient probability p1. Any deterministic verification procedure given 1 / p1 samples will, with a constant probability, be unable to distinguish AV1 from the different AV system AV0 that never experiences an accident.

[0422] To gain insight into typical values ​​for these probabilities, let's assume we want the probability of an accident to be 10 per hour. -9 And a certain AV system provides only 10 -8 The probability. Even if the system obtains 10 8 Even after hours of driving, the verification process will still be unable to indicate a constant probability that the system is dangerous.

[0423] Finally, note that the difficulty lies in invalidating a single, specific, and potentially dangerous AV system. A complete solution cannot be viewed as a single system because new versions, bug fixes, and updates are required. From the validator's perspective, every change, even a single line of code, generates a new system. Therefore, a statistically validated solution must be validated online with a new sample after each small correction or change to account for the shift in the state distribution observed and derived from the new system. Repeatedly and systematically obtaining such a large sample size (and even then, there is a constant probability that the system cannot be validated) is infeasible.

[0424] Furthermore, any statistical claim must be formalized for measurement. A statistical statement about the number of accidents occurring with a system is far weaker than a statement that "it drives safely." To illustrate this, safety must be formally defined.

[0425] Absolute safety is impossible

[0426] If no accident-following action a occurs at some future time, then the action a taken by car c can be considered absolutely safe. This can be demonstrated by observing, for example,... Figure 19 The simple driving scenario depicted demonstrates that absolute safety is impossible. From vehicle 1901's perspective, no action can guarantee that surrounding vehicles won't collide with it. Solving this problem by prohibiting autonomous vehicles from being in this situation is also impossible. Since every highway with two or more lanes will lead to this scenario at some point, prohibiting it is tantamount to asking drivers to stay in their garages. At first glance, this suggestion may seem disappointing. Nothing is absolutely safe. However, as mentioned above, such a requirement for absolute safety may be overly demanding, and human drivers have proven not to adhere to absolute safety requirements. Instead, humans act based on a safety concept that relies on responsibility.

[0427] Responsibility-Sensitive Safety (RSS)

[0428] A crucial aspect missing from the concept of absolute safety is the asymmetry in most accidents—usually one driver is responsible for the crash and is held liable. Figure 19 In the example, if, for instance, car 1909 on the left suddenly veers into car 1901 in the center, then car 1901 should not be held liable. To formalize the fact that it lacks liability, the action of car 1901 staying in its own lane can be considered safe. For this purpose, a formal concept of "accident blame," or accident liability, is described, which can serve as a premise for safe driving practices.

[0429] As an example, consider two cars c f c r A simple case where two cars are traveling at the same speed, one behind the other, along a straight road. Assume car c is in front. f Because of an obstacle on the road, the brakes were suddenly applied and the driver managed to avoid it. Unfortunately, c r No with c f Maintaining sufficient distance, failing to respond in time, and colliding with c f The rear side. Clearly, the blame lies with c. r The following vehicle has a responsibility to maintain a safe distance from the vehicle in front and to be prepared for unexpected but reasonable braking.

[0430] Next, consider a broader range of scenarios: driving on multi-lane roads where cars are free to change lanes, cut into other cars' paths, travel at different speeds, and so on. To simplify the following discussion, assume a straight road on a plane, where the lateral and longitudinal axes are the x and y axes, respectively. Under mild conditions, this can be achieved by defining a homomorphism between actual curves and straight sections. Additionally, consider discrete time and space. The definition may help distinguish between two intuitively different sets of cases: a simple case where no significant lateral maneuver is performed, and a more complex case involving lateral motion.

[0431] Definition 1 (Corridor) The corridor of car c is the range [c x，left c x，right ]×[±∞], where c x，left c x，right It refers to the leftmost and rightmost corners of c.

[0432] Definition 2 (cut-in) If car c1 (e.g., Figure 20A and 20BCar 2003 in the car is not in the same car c0 at time t-1 (e.g., Figure 20A and 20B If the corridors of car 2001 intersect with the corridors of car c0 at time t, then car c1 will cut into the corridor of car c0 at time t.

[0433] Further distinctions can be made between the front and rear sections of the corridor. The term "cut-in direction" can describe movement in the direction of the relevant corridor boundary. These definitions can define cases where lateral movement occurs. For simple cases where this does not occur, such as a car following another car, a safe longitudinal distance is defined:

[0434] Definition 3 (Safe longitudinal distance) If for c f Any braking command a, |a|<a executed by (Automobile 2105) max，brake If c r (Car 2103) applies its maximum braking from time p until it comes to a complete stop, then it will not engage with c. f In a collision, then car C r and located in c r Another car in the corridor ahead f The longitudinal distance between them is 2101 ( Figure 21 The response time p is safe.

[0435] The following Lemma 2 calculates d as c r c f Velocity, response time ρ, and maximum acceleration a max，brake The function of ρ and a. max，brake Both are constants and should be determined to some reasonable values ​​through regulations.

[0436] Lemma 2 Let c r On the vertical axis at c f The vehicles behind. Let a max，brake a max，accel The maximum braking and maximum acceleration commands are given, and ρ is set to c. r The response time. Let v r v f Let l be the longitudinal speed of the car, and let l f , l r Let v be their length. p，max =v r +ρ·a max，accel and define as well as Let L = (l r +l f ) / 2. Then, c r The minimum safe longitudinal distance is:

[0437]

[0438] Proof. Let d t be the distance at time t. To prevent an accident, for every t, we must have d t > L. To construct d min , we need to find the tightest required lower bound for d0. Clearly, d0 must be at least L. As long as the two cars have not stopped after T ≥ ρ seconds, the speed of the leading car will be v f - Ta max，brake , and the upper bound of the speed of c r will be v p，max =(T - ρ)a max，accel . Thus, after T seconds, the lower bound of the distance between the cars will be:

[0439]

[0440] Note that T r is the time when c r reaches a complete stop (speed is 0), and T f is the time when the other vehicle reaches a complete stop. Note that a max，brake (T r - T f ) = v ρ，max - v f + ρa max，brake , so if T r ≤ T f , it is sufficient to require d0 > L. If T r > T f , then

[0441]

[0442] requiring d Tr > L and rearranging the terms leads to the conclusion of the proof.

[0443] Finally, a comparison operator is defined that allows for comparisons with the concept of some "margin": when comparing lengths, speeds, etc., it is necessary to accept very similar quantities as "equal".

[0444] Definition 4 (μ - c...

Claims

1. A system for navigating a host vehicle, the system comprising: at least one processing device programmed to: receiving at least one image representative of an environment of a host vehicle from an image capture device; determining a planned navigation action for achieving a navigation goal of the host vehicle based on at least one driving strategy; analyzing at least one of the images to identify a target vehicle in the environment of the host vehicle, wherein a direction of travel of the target vehicle is toward the host vehicle; determining a next state distance between the host vehicle and the target vehicle that would result if the planned navigation action were taken; determining a host vehicle braking rate, a host vehicle maximum acceleration capability, and a current speed of the host vehicle; determining a stopping distance for the host vehicle based on a host vehicle braking rate, a maximum acceleration capability of the host vehicle, and a current speed of the host vehicle; determining the current speed of the target vehicle, the maximum acceleration capability of the target vehicle, and the braking rate of the target vehicle; determining a stopping distance of the target vehicle based on a braking rate of the target vehicle, a maximum acceleration capability of the target vehicle, and a current speed of the target vehicle; as well as If the determined next state distance is greater than the sum of the host vehicle's stopping distance and the target vehicle's stopping distance, the planned navigation action is implemented.

2. The system according to claim 1, wherein: The target vehicle and the host vehicle are traveling in a parking lot.

3. The system according to claim 1, wherein: The target vehicle is traveling toward the host vehicle in a lane occupied by the host vehicle. The system of claim 1, wherein the host vehicle braking rate corresponds to a maximum braking capacity of the host vehicle.

4. The system according to claim 1, wherein: The host vehicle braking rate corresponds to a predetermined submaximal braking capacity of the host vehicle.

5. The system according to claim 1, wherein: The target vehicle braking rate corresponds to a maximum braking capacity of the target vehicle determined based on at least one characteristic of the target vehicle identified by analyzing the at least one image.

6. The system according to claim 1, wherein: The target vehicle braking rate corresponds to a predetermined submaximum braking capacity of the target vehicle.

7. The system according to claim 1, wherein: The host vehicle braking rate is less than the target vehicle braking rate.

8. The system according to claim 1, wherein: A current speed of the target vehicle is determined based on an analysis of two or more received images representative of the host vehicle's environment.

9. The system according to claim 1, wherein: The current speed of the target vehicle is determined based on the output of a RADAR unit or a LIDAR unit associated with the host vehicle.

10. The system according to claim 1, wherein: The host vehicle's stopping distance includes a host vehicle acceleration distance corresponding to a distance the host vehicle is capable of traveling at a host vehicle maximum acceleration capability during a host vehicle reaction time starting from a determined current speed of the host vehicle.

11. The system according to claim 1, wherein: The target vehicle stopping distance includes a target vehicle acceleration distance corresponding to a distance the target vehicle can travel at a maximum acceleration capability of the target vehicle during a target vehicle reaction time starting from a determined current speed of the target vehicle.

12. The system of claim 1, wherein: At least one of the host vehicle braking rate or the target vehicle braking rate is determined based on a sensed condition of a road surface.

13. The system of claim 1, wherein: At least one of the host vehicle braking rate or the target vehicle braking rate is determined based on sensed weather conditions.

14. The system of claim 1, wherein: The target vehicle maximum acceleration capability is determined based on at least one characteristic of the target vehicle identified by analyzing the at least one image.

15. The system of claim 14, wherein: The at least one characteristic of the target vehicle includes one or more of vehicle type, vehicle size, or vehicle model.

16. The system of claim 1, wherein: The planned navigation maneuver includes at least one of a lane change maneuver, a merging maneuver, a passing maneuver, a maintain forward heading maneuver, a parking lot navigation maneuver, or a maintain throttle maneuver.

17. The system of claim 1, wherein the at least one processing device is configured to implement the planned navigation action if the determined next-state distance is greater than the sum of the host vehicle's stopping distance and the target vehicle's stopping distance by at least a predetermined minimum distance.

18. The system of claim 17, wherein: The predetermined minimum distance corresponds to a predetermined separation distance to be maintained between the host vehicle and other vehicles.

19. The system of claim 17, wherein: The predetermined separation distance is at least one meter.

20. A method for navigating a host vehicle, the method comprising: receiving at least one image representative of an environment of a host vehicle from an image capture device; determining a planned navigation action for achieving a navigation goal of the host vehicle based on at least one driving strategy; analyzing at least one of the images to identify a target vehicle in the environment of the host vehicle, wherein a direction of travel of the target vehicle is toward the host vehicle; determining a next state distance between the host vehicle and the target vehicle that would result if the planned navigation action were taken; determining a host vehicle braking rate, a host vehicle maximum acceleration capability, and a current speed of the host vehicle; determining a stopping distance for the host vehicle based on a host vehicle braking rate, a maximum acceleration capability of the host vehicle, and a current speed of the host vehicle; determining the current speed of the target vehicle, the maximum acceleration capability of the target vehicle, and the braking rate of the target vehicle; determining a stopping distance of the target vehicle based on a braking rate of the target vehicle, a maximum acceleration capability of the target vehicle, and a current speed of the target vehicle; as well as If the determined next state distance is greater than the sum of the host vehicle's stopping distance and the target vehicle's stopping distance, the planned navigation action is implemented.

21. The method according to claim 20, wherein: The host vehicle braking rate corresponds to a maximum braking capacity of the host vehicle.

22. The method according to claim 20, wherein: The host vehicle braking rate corresponds to a predetermined submaximal braking capacity of the host vehicle.

23. The method according to claim 20, wherein: The target vehicle braking rate corresponds to a maximum braking capacity of the target vehicle determined based on at least one characteristic of the target vehicle identified by analyzing the at least one image.

24. The method according to claim 20, wherein: The target vehicle braking rate corresponds to a predetermined submaximum braking capacity of the target vehicle.

25. The method according to claim 20, wherein: The host vehicle braking rate is less than the target vehicle braking rate.

26. The method of claim 20, wherein: A current speed of the target vehicle is determined based on an analysis of two or more received images representative of the host vehicle's environment.

27. The method according to claim 20, wherein: The current speed of the target vehicle is determined based on the output of a RADAR unit or a LIDAR unit associated with the host vehicle.

28. The method of claim 20, wherein: The target vehicle maximum acceleration capability is determined based on at least one characteristic of the target vehicle identified by analyzing the at least one image.

29. The method according to claim 28, wherein: The at least one characteristic of the target vehicle includes one or more of vehicle type, vehicle size, or vehicle model.

30. The method of claim 20, wherein: The planned navigation maneuver includes at least one of a lane change maneuver, a merging maneuver, a passing maneuver, a maintain forward heading maneuver, a parking lot navigation maneuver, or a maintain throttle maneuver.

31. A system for navigating a host vehicle, the system comprising: at least one processing device programmed to: receiving at least one image representative of an environment of a host vehicle from an image capture device; determining a planned navigation action for achieving a navigation goal of the host vehicle based on at least one driving strategy; analyzing at least one image to identify a target vehicle in the host vehicle's environment; determining a next-state lateral distance between the host vehicle and the target vehicle that would result if the planned navigation action were taken; determining a maximum yaw rate capability of the host vehicle, a maximum change turning radius capability of the host vehicle, and a current lateral speed of the host vehicle; determining a lateral braking distance of the host vehicle based on a maximum yaw rate capability of the host vehicle, a maximum change turning radius capability of the host vehicle, and a current lateral speed of the host vehicle; determining a current lateral speed of the target vehicle, a maximum yaw rate capability of the target vehicle, and a maximum change turning radius capability of the target vehicle; determining a lateral braking distance of the target vehicle based on a current lateral speed of the target vehicle, a maximum yaw rate capability of the target vehicle, and a maximum change turning radius capability of the target vehicle; as well as If the determined next-state lateral distance is greater than the sum of the lateral braking distance of the host vehicle and the lateral braking distance of the target vehicle, the planned navigation action is implemented.

32. The system of claim 31, wherein: The lateral braking distance of the host vehicle corresponds to a distance required to reach a lateral velocity of the host vehicle of zero at the maximum yaw rate capability of the host vehicle and at the maximum changed turning radius capability of the host vehicle starting from the current lateral velocity of the host vehicle and following lateral acceleration of the host vehicle as a result of a turning maneuver made toward the target vehicle at the maximum yaw rate capability of the host vehicle and at the maximum changed turning radius capability of the host vehicle during a reaction time associated with the host vehicle.

33. The system of claim 31, wherein: The lateral braking distance of the target vehicle corresponds to a distance required to reach a target vehicle lateral velocity of zero starting from a current lateral velocity of the target vehicle and following lateral acceleration of the target vehicle as a result of a turning maneuver made by the target vehicle toward the host vehicle at the target vehicle maximum yaw rate capability and the target vehicle maximum changed turning radius capability during a reaction time associated with the target vehicle.

34. The system of claim 31, wherein: The at least one processing device is programmed to implement a planned navigation maneuver regardless of a next-state lateral distance if the planned navigation maneuver will result in a safe longitudinal distance between the host vehicle and the target vehicle.

35. The system of claim 34, wherein: The safety longitudinal distance is greater than or equal to the sum of the longitudinal stopping distance of the host vehicle and the longitudinal stopping distance of the target vehicle; wherein the host vehicle stopping distance is based on a maximum longitudinal braking capability of the host vehicle and a current longitudinal speed of the host vehicle, and wherein the host vehicle stopping distance includes a host vehicle acceleration distance corresponding to a distance that the host vehicle can travel at a maximum acceleration capability of the host vehicle during a reaction time associated with the host vehicle starting from the current longitudinal speed of the host vehicle; and wherein the target vehicle stopping distance is based on a maximum longitudinal braking capability of the target vehicle and a current longitudinal speed of the target vehicle, and wherein the target vehicle stopping distance includes a target vehicle acceleration distance corresponding to a distance the target vehicle is capable of traveling at a maximum acceleration capability of the target vehicle during a reaction time associated with the target vehicle starting from the current longitudinal speed of the target vehicle.

36. The system of claim 31, wherein: At least one of the target vehicle maximum yaw rate capability and the target vehicle maximum changed turning radius capability is determined based on at least one characteristic of the target vehicle identified by analyzing the at least one image.

37. The system of claim 6, wherein: The at least one characteristic of the target vehicle includes one or more of vehicle type, vehicle size, or vehicle model.

38. The system of claim 31, wherein: The maximum yaw rate capability of the host vehicle and the maximum changed turning radius capability of the host vehicle are based on predetermined constraints of the host vehicle.

39. The system of claim 31, wherein: A current lateral velocity of the target vehicle is determined based on an output of a RADAR unit or a LIDAR unit associated with the host vehicle.

40. The system of claim 31, wherein: At least one of a maximum yaw rate capability of the host vehicle or a maximum changed turning radius capability of the host vehicle is determined based on a sensed condition of a road surface.

41. The system of claim 31, wherein: At least one of a target vehicle maximum yaw rate capability or a target vehicle maximum changed turning radius capability is determined based on the sensed condition of the road surface.

42. The system of claim 31, wherein: At least one of the host vehicle's maximum yaw rate capability, the host vehicle's maximum variable turning radius capability, a target vehicle's maximum yaw rate capability, or a target vehicle's maximum variable turning radius capability is determined based on sensed weather conditions.

43. The system of claim 31, wherein: The planned navigation maneuver includes at least one of a lane change maneuver, a merging maneuver, a passing maneuver, a parking lot navigation maneuver, or a turning maneuver.

44. The system of claim 31 , wherein the at least one processing device is configured to implement a planned navigation action if the determined next-state lateral distance is greater than the sum of the lateral braking distance of the host vehicle and the lateral braking distance of the target vehicle by at least a predetermined minimum distance.

45. The system of claim 44, wherein: The predetermined minimum distance corresponds to a predetermined lateral separation distance to be maintained between the host vehicle and other vehicles.

46. ​​The system of claim 15, wherein: The predetermined lateral separation distance is at least one meter.

47. A method for navigating a host vehicle, the method comprising: receiving at least one image representative of an environment of a host vehicle from an image capture device; determining a planned navigation action for achieving a navigation goal of the host vehicle based on at least one driving strategy; analyzing at least one image to identify a target vehicle in the host vehicle's environment; determining a next-state lateral distance between the host vehicle and the target vehicle that would result if the planned navigation action were taken; determining a maximum yaw rate capability of the host vehicle, a maximum change turning radius capability of the host vehicle, and a current lateral speed of the host vehicle; determining a lateral braking distance of the host vehicle based on a maximum yaw rate capability of the host vehicle, a maximum change turning radius capability of the host vehicle, and a current lateral speed of the host vehicle; determining a current lateral speed of the target vehicle, a maximum yaw rate capability of the target vehicle, and a maximum change turning radius capability of the target vehicle; determining a lateral braking distance of the target vehicle based on a current lateral speed of the target vehicle, a maximum yaw rate capability of the target vehicle, and a maximum change turning radius capability of the target vehicle; as well as If the determined next-state lateral distance is greater than the sum of the lateral braking distance of the host vehicle and the lateral braking distance of the target vehicle, the planned navigation action is implemented.

48. The method of claim 47, wherein: The lateral braking distance of the host vehicle corresponds to a distance required to reach a lateral velocity of the host vehicle of zero at the maximum yaw rate capability of the host vehicle and at the maximum changed turning radius capability of the host vehicle starting from the current lateral velocity of the host vehicle and following lateral acceleration of the host vehicle as a result of a turning maneuver made toward the target vehicle at the maximum yaw rate capability of the host vehicle and at the maximum changed turning radius capability of the host vehicle during a reaction time associated with the host vehicle.

49. The method of claim 47, wherein: The lateral braking distance of the target vehicle corresponds to a distance required to reach a target vehicle lateral velocity of zero starting from a current lateral velocity of the target vehicle and following lateral acceleration of the target vehicle as a result of a turning maneuver made by the target vehicle toward the host vehicle at the target vehicle maximum yaw rate capability and the target vehicle maximum changed turning radius capability during a reaction time associated with the target vehicle.

50. The method of claim 47, wherein: The at least one processing device is programmed to implement a planned navigation maneuver regardless of a next-state lateral distance if the planned navigation maneuver will result in a safe longitudinal distance between the host vehicle and the target vehicle.

51. The method of claim 50, wherein: The safety longitudinal distance is greater than or equal to the sum of the longitudinal stopping distance of the host vehicle and the longitudinal stopping distance of the target vehicle; wherein the host vehicle stopping distance is based on a maximum longitudinal braking capability of the host vehicle and a current longitudinal speed of the host vehicle, and wherein the host vehicle stopping distance includes a host vehicle acceleration distance corresponding to a distance that the host vehicle can travel at a maximum acceleration capability of the host vehicle during a reaction time associated with the host vehicle starting from the current longitudinal speed of the host vehicle; and wherein the target vehicle stopping distance is based on a maximum longitudinal braking capability of the target vehicle and a current longitudinal speed of the target vehicle, and wherein the target vehicle stopping distance includes a target vehicle acceleration distance corresponding to a distance the target vehicle is capable of traveling at a maximum acceleration capability of the target vehicle during a reaction time associated with the target vehicle starting from the current longitudinal speed of the target vehicle.

52. The method of claim 47, wherein: At least one of the target vehicle maximum yaw rate capability and the target vehicle maximum changed turning radius capability is determined based on at least one characteristic of the target vehicle identified by analyzing the at least one image.

53. The system of claim 52, wherein: The at least one characteristic of the target vehicle includes one or more of vehicle type, vehicle size, or vehicle model.

54. The method of claim 47, wherein: The maximum yaw rate capability of the host vehicle and the maximum changed turning radius capability of the host vehicle are based on predetermined constraints of the host vehicle.

55. The method of claim 47, wherein: A current lateral velocity of the target vehicle is determined based on an output of a RADAR unit or a LIDAR unit associated with the host vehicle.

56. The method of claim 47, wherein: At least one of the host vehicle's maximum yaw rate capability, the host vehicle's maximum variable turning radius capability, a target vehicle's maximum yaw rate capability, or a target vehicle's maximum variable turning radius capability is determined based on a sensed weather condition of a road surface.

57. The method of claim 47, wherein: At least one of the host vehicle's maximum yaw rate capability, the host vehicle's maximum variable turning radius capability, a target vehicle's maximum yaw rate capability, or a target vehicle's maximum variable turning radius capability is determined based on sensed weather conditions.

58. The method of claim 47, wherein: The planned navigation maneuver includes at least one of a lane change maneuver, a merging maneuver, a passing maneuver, a parking lot navigation maneuver, or a turning maneuver.

59. The method of claim 47, wherein the at least one processing device is configured to implement a planned navigation action if the determined next-state lateral distance is greater than the sum of the lateral braking distance of the host vehicle and the lateral braking distance of the target vehicle by at least a predetermined minimum distance.

60. The method of claim 59, wherein: The predetermined minimum distance corresponds to a predetermined lateral separation distance to be maintained between the host vehicle and other vehicles.

61. A system for navigating a host vehicle near a crosswalk, the system comprising at least one processing device programmed to: receiving at least one image representative of an environment of a host vehicle from an image capture device; detecting a representation of a crosswalk in the at least one image based on an analysis of the at least one image; determining, based on an analysis of the at least one image, whether a representation of a pedestrian appears in the at least one image; detecting the presence of traffic lights in the host vehicle's environment; determining whether the detected traffic light is associated with the host vehicle and a crosswalk; determining the state of a detected traffic light; When a representation of a pedestrian appears in at least one of the images, determining a proximity of the pedestrian relative to a detected crosswalk; Determining a planned navigation action based on at least one driving strategy, the planned navigation action for navigating the host vehicle relative to the detected crosswalk, wherein the determination of the planned navigation action is further based on the determined state of the detected traffic light and the determined proximity of the pedestrian relative to the detected crosswalk; as well as One or more actuator systems of the host vehicle are caused to implement the planned navigation maneuver.

62. The system of claim 61, wherein: The planned navigation action of the host vehicle includes crossing the crosswalk without pedestrian-induced adjustments when the detected traffic light is in a green state and the determined proximity of the pedestrian relative to the detected crosswalk indicates that the pedestrian is unlikely to enter the crosswalk before the host vehicle intersects the crosswalk.

63. The system of claim 61, wherein: The planned navigation action of the host vehicle includes crossing the crosswalk without pedestrian-induced adjustments when the detected traffic light is in a green state and the determined proximity of the pedestrian relative to the detected crosswalk combined with the detected heading and speed associated with the pedestrian indicate that the pedestrian cannot enter the crosswalk before the host vehicle intersects the crosswalk.

64. The system of claim 61, wherein: The planned navigation action of the host vehicle includes crossing the crosswalk with adjustments caused by the pedestrian when the detected traffic light is in a green state and the determined proximity of the pedestrian relative to the detected crosswalk indicates that the pedestrian can enter the crosswalk before the host vehicle intersects the crosswalk.

65. The system of claim 64, wherein: A pedestrian-induced adjustment is determined based on the detected heading and speed of the pedestrian.

66. The system of claim 64, wherein: The pedestrian-induced adjustment includes lateral movement away from the pedestrian.

67. The system of claim 64, wherein: The pedestrian-induced adjustment includes decelerating the host vehicle before the host vehicle intersects the crosswalk.

68. The system of claim 61, wherein: The planned navigation action of the host vehicle includes: when the traffic light is in the green state and a pedestrian is detected in the crosswalk, stopping the host vehicle before the crosswalk.

69. The system of claim 61, wherein: The determination of whether the detected traffic light is associated with the host vehicle and the crosswalk is based on traffic light relevance information included in a map accessible to the at least one processing device.

70. The system of claim 61, wherein: The detection of the presence of the traffic light in the environment of the host vehicle is based on an analysis of the at least one image.

71. The system of claim 61, wherein: The detection of the presence of the traffic light in the host vehicle's environment is based on one or more signals transmitted by the detected traffic light.

72. The system of claim 61, wherein: The determination of the state of the detected traffic light is based on an analysis of the at least one image.

73. The system of claim 61, wherein: The determination of the state of the detected traffic light is based on one or more signals transmitted by the detected traffic light.

74. A method for navigating a host vehicle near a crosswalk, the method comprising: receiving at least one image representative of an environment of a host vehicle from an image capture device; detecting a representation of a crosswalk in the at least one image based on an analysis of the at least one image; determining, based on an analysis of the at least one image, whether a representation of a pedestrian appears in the at least one image; detecting the presence of traffic lights in the host vehicle's environment; determining whether the detected traffic light is associated with the host vehicle and a crosswalk; determining the state of a detected traffic light; When a representation of a pedestrian appears in at least one of the images, determining a proximity of the pedestrian relative to a detected crosswalk; Determining a planned navigation action based on at least one driving strategy, the planned navigation action for navigating the host vehicle relative to the detected crosswalk, wherein the determination of the planned navigation action is further based on the determined state of the detected traffic light and the determined proximity of the pedestrian relative to the detected crosswalk; as well as One or more actuator systems of the host vehicle are caused to implement the planned navigation maneuver.

75. The method of claim 74, wherein: The detection of the presence of the traffic light in the host vehicle's environment is based on one or more signals transmitted by the detected traffic light.

76. The method of claim 74, wherein: The planned navigation action of the host vehicle includes crossing the crosswalk without pedestrian-induced adjustments when the detected traffic light is in a green state and the determined proximity of the pedestrian relative to the detected crosswalk indicates that the pedestrian is unlikely to enter the crosswalk before the host vehicle intersects the crosswalk.

77. The method of claim 74, wherein: The planned navigation action of the host vehicle includes crossing the crosswalk without pedestrian-induced adjustments when the detected traffic light is in a green state and the determined proximity of the pedestrian relative to the detected crosswalk combined with the detected heading and speed associated with the pedestrian indicate that the pedestrian cannot enter the crosswalk before the host vehicle intersects the crosswalk.

78. The method of claim 74, wherein: The planned navigation action of the host vehicle includes crossing the crosswalk with adjustments caused by the pedestrian when the detected traffic light is in a green state and the determined proximity of the pedestrian relative to the detected crosswalk indicates that the pedestrian can enter the crosswalk before the host vehicle intersects the crosswalk.

79. A method for navigating a vehicle near a crosswalk, the method comprising: receiving at least one image representative of an environment of a host vehicle from an image capture device; Detect the starting and ending positions of the horizontal track; determining, based on an analysis of at least one image, whether a pedestrian is present near the crosswalk; detecting the presence of traffic lights in the host vehicle's environment; Determine if the traffic light is relevant to the host vehicle and the crosswalk; Determine the status of traffic lights; The navigation action of the host vehicle near the detected crosswalk is determined based on: The relevance of traffic lights; the determined state of the traffic light; the presence of pedestrians near the crossing; the shortest distance selected between the pedestrian and either the start or end of the crossing; and The pedestrian's motion vector; and One or more actuator systems of the host vehicle are caused to perform a navigation action.

80. The method of claim 79, wherein: The detection of the presence of the traffic light in the host vehicle's environment is based on one or more signals transmitted by the detected traffic light.

Images