Vehicle navigation system and method

Abstract

Systems and methods for vehicle navigation are provided. In one implementation, at least one processor may receive at least one captured image representing features in the environment of the vehicle from a camera. The processor may identify intersections represented in the image and pedestrians near the intersection. The processor may determine a navigation action of the vehicle relative to the intersection based on routing information of the vehicle; and determine a predicted path of the vehicle relative to the intersection based on the determined navigation action, and determine a predicted path of the pedestrian based on an analysis of the image. The processor may further determine whether the vehicle is expected to collide with a pedestrian based on these predicted paths; and, in response, cause a system associated with the vehicle to implement a collision mitigation action.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 858,082, filed on June 6, 2019, U.S. Provisional Patent Application No. 62 / 864,203, filed on June 20, 2019, and U.S. Provisional Patent Application No. 62 / 878,523, filed on July 25, 2019. The above applications are incorporated herein by reference in their entirety.

[0003] background Technical Field

[0004] The present disclosure generally relates to autonomous vehicle navigation. Additionally, the present disclosure relates to systems and methods for navigating within potential accident liability constraints. Background Art

[0005] As technology continues to advance, the goal of fully autonomous vehicles capable of navigating roads is imminent. Autonomous vehicles may need to consider various factors and make appropriate decisions based on these factors to safely and accurately reach their intended destination. For example, autonomous vehicles may need to process and interpret visual information (e.g., information captured from a camera), information from radio detection and ranging or laser detection and ranging, and may also use information obtained from other sources (e.g., from a GPS device, speed sensor, accelerometer, suspension sensor, etc.). At the same time, in order to navigate to the destination, autonomous vehicles may also need to identify their position within a specific road (e.g., a specific lane within a multi-lane road), navigate alongside other vehicles, avoid obstacles and pedestrians, observe traffic signals and signs, switch from one road to another at appropriate intersections or interchanges, and respond to any other situations that occur or develop during vehicle operation. In addition, the navigation system may need to comply with certain imposed constraints. In some cases, these constraints may be related to the interaction between the host vehicle and one or more other objects (such as other vehicles, pedestrians, etc.). In other cases, the constraints may relate to responsibility rules to be followed when implementing one or more navigation actions for the host vehicle.

[0006] In the field of autonomous driving, there are two important considerations for viable autonomous vehicle systems. The first is the standardization of safety assurance, including the requirements that each autonomous vehicle must meet to ensure safety and how those requirements can be verified. The second is scalability, as cost-inducing engineering solutions will not scale to millions of vehicles and could hinder widespread adoption of autonomous vehicles, or even prevent widespread adoption. Therefore, there is a need for interpretable mathematical models for safety assurance and the design of systems that adhere to safety assurance requirements while being scalable to millions of vehicles. Summary of the Invention

[0007] Embodiments consistent with the present disclosure provide systems and methods for autonomous vehicle navigation. The disclosed embodiments may use cameras to provide autonomous vehicle navigation features. For example, consistent with the disclosed embodiments, the disclosed system may include one, two, or more cameras that monitor the vehicle environment. The disclosed system may provide a navigation response based on, for example, analysis of images captured by one or more of the cameras. The navigation response may also take into account other data, including, for example, global positioning system (GPS) data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data.

[0008] In an embodiment, a navigation system for a host vehicle may include at least one processor. The processor may be programmed to: receive at least one captured image from a camera representing features in the vehicle's environment; identify at least a portion of an intersection represented in the at least one captured image; and identify a representation of a pedestrian near the intersection in the at least one captured image. The processor may be programmed to: determine a navigation maneuver for the vehicle relative to the intersection based at least in part on routing information for the vehicle; determine a predicted path for the vehicle relative to the intersection based on the determined navigation maneuver; and determine a predicted path for the pedestrian based on an analysis of the at least one captured image. The processor may further be programmed to determine whether the vehicle is expected to collide with the pedestrian based at least in part on the predicted path of the vehicle and based on the predicted path of the pedestrian; and, in response to a determination that the vehicle is expected to collide with the pedestrian, cause at least one system associated with the vehicle to implement a collision mitigation action.

[0009] In an embodiment, a method for navigating a host vehicle may include: receiving at least one captured image from a camera representing features in the vehicle's environment; identifying at least a portion of an intersection represented in the at least one captured image; and identifying a representation of a pedestrian near the intersection in the at least one captured image. The method may further include: determining a navigational maneuver for the vehicle relative to the intersection based at least in part on routing information for the vehicle; determining a predicted path for the vehicle relative to the intersection based on the determined navigational maneuver; and determining a predicted path for the pedestrian based on an analysis of the at least one captured image. The method may further include: determining whether the vehicle is expected to collide with the pedestrian based at least in part on the predicted path of the vehicle and based on the predicted path of the pedestrian; and causing at least one system associated with the vehicle to implement a collision mitigation action in response to a determination that the vehicle is expected to collide with the pedestrian.

[0010] Consistent with other disclosed embodiments, a non-transitory computer-readable storage medium may store program instructions executable by at least one processing device to perform any of the steps and / or methods described herein.

[0011] The foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate various disclosed embodiments. In the drawings:

[0013] Figure 1 is a diagrammatic representation of an exemplary system consistent with the disclosed embodiments.

[0014] Figure 2A is a diagrammatic side view representation of an exemplary vehicle including a system consistent with the disclosed embodiments.

[0015] Figure 2B is consistent with the disclosed embodiments Figure 2A A diagrammatic top view representation of the vehicle and systems shown in FIG.

[0016] Figure 2C is a diagrammatic top view representation of another embodiment of a vehicle including a system consistent with the disclosed embodiments.

[0017] Figure 2D is a diagrammatic top view representation of yet another embodiment of a vehicle including a system consistent with the disclosed embodiments.

[0018] Figure 2Eis a diagrammatic top view representation of yet another embodiment of a vehicle including a system consistent with the disclosed embodiments.

[0019] Figure 2F is a diagrammatic representation of an exemplary vehicle control system consistent with the disclosed embodiments.

[0020] Figure 3A is a diagrammatic representation of the interior of a vehicle including a rearview mirror and a user interface for a vehicle imaging system, consistent with the disclosed embodiments.

[0021] Figure 3B is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and against a vehicle windshield, consistent with the disclosed embodiments.

[0022] Figure 3C is consistent with the disclosed embodiments Figure 3B Illustrations of the camera mount shown in from different angles.

[0023] Figure 3D is an illustration of an example of a camera mount configured to be positioned behind a rearview mirror and against a vehicle windshield, consistent with the disclosed embodiments.

[0024] Figure 4 is an exemplary block diagram of a memory configured to store instructions for performing one or more operations consistent with the disclosed embodiments.

[0025] Figure 5A is a flow chart illustrating an exemplary process for eliciting one or more navigation responses based on monocular image analysis, consistent with the disclosed embodiments.

[0026] Figure 5B is a flow chart illustrating an exemplary process for detecting one or more vehicles and / or pedestrians in a set of images consistent with the disclosed embodiments.

[0027] Figure 5C is a flow chart illustrating an exemplary process for detecting road markings and / or lane geometry information in a set of images consistent with the disclosed embodiments.

[0028] Figure 5D is a flow chart illustrating an exemplary process for detecting traffic lights in a set of images consistent with the disclosed embodiments.

[0029] Figure 5E is a flow chart illustrating an exemplary process for eliciting one or more navigation responses based on a vehicle path, consistent with the disclosed embodiments.

[0030] Figure 5Fis a flow chart illustrating an exemplary process for determining whether a leading vehicle is changing lanes, consistent with the disclosed embodiments.

[0031] Figure 6 is a flow chart illustrating an exemplary process for eliciting one or more navigation responses based on stereo image analysis, consistent with the disclosed embodiments.

[0032] Figure 7 is a flow chart illustrating an exemplary process for eliciting one or more navigational responses based on analysis of three sets of images, consistent with the disclosed embodiments.

[0033] Figure 8 is a block diagram representation of modules that may be implemented by one or more specifically programmed processing devices for a navigation system for an autonomous vehicle consistent with the disclosed embodiments.

[0034] Figure 9 is a diagram of navigation options consistent with the disclosed embodiments.

[0035] Figure 10 is a diagram of navigation options consistent with the disclosed embodiments.

[0036] Figure 11A 、 Figure 11B and Figure 11C A schematic representation of navigation options for a host vehicle in a merge zone consistent with the disclosed embodiments is provided.

[0037] Figure 11D A diagrammatic depiction of a dual merge scenario consistent with the disclosed embodiments is provided.

[0038] Figure 11E A diagram is provided of options that may be useful in dual merging scenarios consistent with the disclosed embodiments.

[0039] Figure 12 Illustrations of captured representative images of a host vehicle's environment, along with potential navigation constraints, consistent with the disclosed embodiments are provided.

[0040] Figure 13 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0041] Figure 14 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0042] Figure 15 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0043] Figure 16A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0044] Figure 17A and Figure 17B A diagrammatic illustration of a host vehicle navigating a roundabout consistent with the disclosed embodiments is provided.

[0045] Figure 18 A flow chart of an algorithm for navigating a vehicle consistent with the disclosed embodiments is provided.

[0046] Figure 19 An example of a host vehicle driving on a multi-lane highway is shown, consistent with the disclosed embodiments.

[0047] Figure 20A and Figure 20B An example of a vehicle cutting in front of another vehicle is shown, consistent with the disclosed embodiments.

[0048] Figure 21 An example of a vehicle following another vehicle is shown, consistent with the disclosed embodiments.

[0049] Figure 22 An example of a vehicle exiting a parking lot and merging onto a potentially busy roadway is shown, consistent with the disclosed embodiments.

[0050] Figure 23 A vehicle consistent with the disclosed embodiments is shown traveling on a road.

[0051] Figure 24A 、 Figure 24B 、 Figure 24C and Figure 24D Four example scenarios consistent with the disclosed embodiments are illustrated.

[0052] Figure 25 The diagrams illustrate example scenarios consistent with the disclosed embodiments.

[0053] Figure 26 The diagrams illustrate example scenarios consistent with the disclosed embodiments.

[0054] Figure 27 The diagrams illustrate example scenarios consistent with the disclosed embodiments.

[0055] Figure 28A and Figure 28B An example of a scenario in which a vehicle follows another vehicle is illustrated, consistent with the disclosed embodiments.

[0056] Figure 29A and Figure 29B An example attribution in a cut-in scenario is illustrated, consistent with the disclosed embodiments.

[0057] Figure 30A and Figure 30B An example attribution in a cut-in scenario is illustrated, consistent with the disclosed embodiments.

[0058] Figure 31A 、 Figure 31B 、 Figure 31C and Figure 31D FIGURES illustrate example attribution in a drift scenario consistent with the disclosed embodiments.

[0059] Figure 32A and Figure 32B An example attribution in a two-way traffic scenario is illustrated, consistent with the disclosed embodiments.

[0060] Figure 33A and Figure 33B An example attribution in a route priority scenario consistent with the disclosed embodiments is illustrated.

[0061] Figure 34A and Figure 34B An example attribution in a route priority scenario consistent with the disclosed embodiments is illustrated.

[0062] Figure 35A and Figure 35B An example attribution in a route priority scenario consistent with the disclosed embodiments is illustrated.

[0063] Figure 36A and Figure 36B An example attribution in a route priority scenario consistent with the disclosed embodiments is illustrated.

[0064] Figure 37A and Figure 37B An example attribution in a route priority scenario consistent with the disclosed embodiments is illustrated.

[0065] Figure 38A and Figure 38B An example attribution in a route priority scenario consistent with the disclosed embodiments is illustrated.

[0066] Figure 39A and Figure 39B An example attribution in a route priority scenario consistent with the disclosed embodiments is illustrated.

[0067] Figure 40A and Figure 40B An example attribution in a traffic light scenario is illustrated, consistent with the disclosed embodiments.

[0068] Figure 41A and Figure 41B An example attribution in a traffic light scenario is illustrated, consistent with the disclosed embodiments.

[0069] Figure 42Aand Figure 42B An example attribution in a traffic light scenario is illustrated, consistent with the disclosed embodiments.

[0070] Figure 43A 、 Figure 43B and Figure 43C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is illustrated.

[0071] Figure 44A 、 Figure 44B and Figure 44C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is illustrated.

[0072] Figure 45A 、 Figure 45B and Figure 45C An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is illustrated.

[0073] Figure 46A 、 Figure 46B 、 Figure 46C and Figure 46D An example vulnerable road user (VRU) scenario consistent with the disclosed embodiments is illustrated.

[0074] Figure 47 is an illustration of an example scenario in which collision prediction may be used consistent with the disclosed embodiments

[0075] Figure 48 、 Figure 49 and Figure 50 Example envelopes that may be determined for a host vehicle and / or pedestrians consistent with the disclosed embodiments are illustrated.

[0076] Figure 51 is a diagram illustrating an example technique for determining collision times between circular envelopes, consistent with the disclosed embodiments.

[0077] Figure 52 is a diagram of an example timeline for collision prediction corresponding to a circular envelope consistent with the disclosed embodiments

[0078] Figure 53 is a diagram illustrating an example refined analysis for determining time to collision consistent with the disclosed embodiments.

[0079] Figure 54 is a flow chart illustrating an example process for navigating a host vehicle consistent with the disclosed embodiments. DETAILED DESCRIPTION

[0080] The following detailed description refers to the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the following description to refer to the same or similar parts. Although several illustrative embodiments are described herein, modifications, adaptations, and other implementations are also possible. For example, the components illustrated in the drawings may be replaced, added to, or modified, and the illustrative methods described herein may be modified by replacing, reordering, removing, or adding steps to the disclosed methods. Therefore, the following detailed description is not limited to the disclosed embodiments and examples. Instead, the appropriate scope is defined by the appended claims.

[0081] Autonomous Vehicles Overview

[0082] As used throughout this disclosure, the term "autonomous vehicle" refers to a vehicle that can implement at least one navigation change without driver input. "Navigation change" refers to a change in one or more of the steering, braking, or acceleration / deceleration of the vehicle. To become autonomous, the vehicle does not need to be fully automatic (e.g., fully operating without a driver or without driver input). On the contrary, an autonomous vehicle includes a vehicle that can operate under driver control in certain time periods and operate without driver control in other time periods. An autonomous vehicle may also include a vehicle that only controls some aspects of vehicle navigation, such as steering (e.g., maintaining a vehicle route between vehicle lane constraints) or some steering operations in certain situations (but not in all situations), but may leave other aspects of vehicle navigation to the driver (e.g., braking or braking in certain situations). In some cases, an autonomous vehicle may handle some or all aspects of braking, speed control, and / or steering of the vehicle.

[0083] Because human drivers typically rely on visual cues and observation to control vehicles, transportation infrastructure is constructed accordingly, with lane markings, traffic signs, and traffic lights designed to provide visual information to drivers. Given these design features of transportation infrastructure, autonomous vehicles can include cameras and processing units that analyze visual information captured from the vehicle's environment. This visual information can include, for example, images representing components of the transportation infrastructure observable by the driver (e.g., lane markings, traffic signs, traffic lights, etc.) and other obstacles (e.g., other vehicles, pedestrians, debris, etc.). Additionally, autonomous vehicles can use stored information, such as information that provides a model of the vehicle's environment when navigating. For example, a vehicle can use GPS data, sensor data (e.g., from accelerometers, speed sensors, suspension sensors, etc.), and / or other map data to provide information about its environment while driving, and the vehicle (and other vehicles) can use this information to locate itself on the model. Some vehicles may also be able to communicate with each other, share information, change peer vehicles that pose a hazard or change the vehicle's surroundings, and so on.

[0084] System Overview

[0085] Figure 1 is a block diagram representation of a system 100 consistent with the disclosed exemplary embodiments. System 100 may include various components, depending on the requirements of a particular implementation. In some embodiments, system 100 may include a processing unit 110, an image acquisition unit 120, a position sensor 130, one or more memory units 140, 150, a map database 160, a user interface 170, and a wireless transceiver 172. Processing unit 110 may include one or more processing devices. In some embodiments, processing unit 110 may include an application processor 180, an image processor 190, or any other suitable processing device. Similarly, image acquisition unit 120 may include any number of image acquisition devices and components, depending on the requirements of a particular application. In some embodiments, image acquisition unit 120 may include one or more image capture devices (e.g., cameras, CCDs, or any other type of image sensor), such as image capture device 122, image capture device 124, and image capture device 126. System 100 may also include a data interface 128 that communicatively connects processing unit 110 to image acquisition unit 120. For example, the data interface 128 may include any one or more wired and / or wireless links for transmitting image data acquired by the image acquisition unit 120 to the processing unit 110 .

[0086] The wireless transceiver 172 may include one or more devices configured to exchange transmissions over an air interface to one or more networks (e.g., cellular, Internet, etc.) using radio frequencies, infrared frequencies, magnetic fields, or electric fields. The wireless transceiver 172 may transmit and / or receive data using any known standard (e.g., Wi-Fi, Bluetooth Smart, 802.15.4, ZigBee, etc.). Such transmissions may include communications from the host vehicle to one or more remotely located servers. Such transmissions may also include communications (one-way or two-way) between the host vehicle and one or more target vehicles in the host vehicle's environment (e.g., to facilitate coordination of the host vehicle's navigation with respect to or with target vehicles in the host vehicle's environment), or even broadcast transmissions to unspecified recipients in the vicinity of the transmitting vehicle.

[0087] Both the application processor 180 and the image processor 190 may include various types of hardware-based processing devices. For example, either or both of the application processor 180 and the image processor 190 may include a microprocessor, a preprocessor (such as an image preprocessor), a graphics processor, a central processing unit (CPU), support circuits, a digital signal processor, an integrated circuit, memory, or any other type of device suitable for running applications and suitable for image processing and analysis. In some embodiments, the application processor 180 and / or the image processor 190 may include any type of single-core or multi-core processor, a mobile device microcontroller, a central processing unit, etc. Various processing devices may be used, including, for example, processors available from, for example, processors from manufacturers such as , and various processing devices may include various architectures (e.g., x86 processors, wait).

[0088] In some embodiments, the application processor 180 and / or the image processor 190 may include a processor that can be Any processor chip from the EyeQ series of processor chips available. These processor designs each include multiple processing units with local memory and instruction sets. Such processors may include video inputs for receiving image data from multiple image sensors and may also include video output capabilities. In one example, Using 90nm micron technology operating at 332Mhz. The architecture consists of two floating point hyperthreaded 32-bit RISC CPUs ( cores), five Visual Compute Engines (VCEs), and three vector microcode processors Denali 64-bit mobile DDR controller, 128-bit internal acoustic interconnect, dual 16-bit video input and 18-bit video output controllers, 16-channel DMA, and several peripherals. MIPS34K CPU manages five VCEs, three VMPs TM and DMA, a second MIPS34K CPU and multi-channel DMA and other peripherals. Five VCEs, three and MIPS34K CPU can perform the intensive visual calculations required for multi-function bundled applications. In another example, as a third-generation processor and Six times more powerful Can be used in the disclosed embodiments. In other examples, and / or Of course, any newer or future EyeQ processing devices can also be used with the disclosed embodiments.

[0089] Any of the processing devices disclosed herein can be configured to perform certain functions. Configuring a processing device (such as any of the EyeQ processors or other controllers or microprocessors described) to perform certain functions can include programming computer-executable instructions and making these instructions available to the processing device for execution during operation of the processing device. In some embodiments, configuring the processing device can include programming the processing device directly with architectural instructions. In other embodiments, configuring the processing device can include storing executable instructions on a memory accessible to the processing device during operation. For example, the processing device can access the memory during operation to obtain and execute the stored instructions. In either case, the processing device configured to perform the sensing, image analysis, and / or navigation functions disclosed herein represents a dedicated hardware-based system that controls multiple hardware-based components of a host vehicle.

[0090] Although Figure 1 While processing unit 110 is depicted as including two separate processing devices, more or fewer processing devices may be used. For example, in some embodiments, a single processing device may be used to perform the tasks of application processor 180 and image processor 190. In other embodiments, these tasks may be performed by more than two processing devices. Furthermore, in some embodiments, system 100 may include one or more processing units 110 without including other components, such as image acquisition unit 120.

[0091] The processing unit 110 may include various types of devices. For example, the processing unit 110 may include various devices such as a controller, an image preprocessor, a central processing unit (CPU), support circuits, a digital signal processor, an integrated circuit, memory, or any other type of device for image processing and analysis. The image preprocessor may include a video processor for capturing, digitizing, and processing images from an image sensor. The CPU may include any number of microcontrollers or microprocessors. The support circuits may be any number of circuits generally known in the art, including caches, power supplies, clocks, and input-output circuits. The memory may store software that, when executed by the processor, controls the operation of the system. The memory may include a database and image processing software. The memory may include any number of random access memories, read-only memories, flash memories, disk drives, optical storage, tape storage, removable storage, and other types of storage. In one embodiment, the memory may be separate from the processing unit 110. In another embodiment, the memory may be integrated into the processing unit 110.

[0092] Each memory 140, 150 may include software instructions that, when executed by a processor (e.g., application processor 180 and / or image processor 190), may control the operation of various aspects of system 100. These memory units may include various databases and image processing software, as well as trained systems such as neural networks or deep neural networks, for example. The memory units may include random access memory, read-only memory, flash memory, disk drives, optical storage, tape storage, removable storage, and / or any other type of storage. In some embodiments, the memory units 140, 150 may be separate from the application processor 180 and / or image processor 190. In other embodiments, these memory units may be integrated into the application processor 180 and / or image processor 190.

[0093] Position sensor 130 may include any type of device suitable for determining a location associated with at least one component of system 100. In some embodiments, position sensor 130 may include a GPS receiver. Such a receiver can determine the user's location and velocity by processing signals broadcast by global positioning system satellites. Position information from position sensor 130 may be used by application processor 180 and / or image processor 190.

[0094] In some embodiments, system 100 may include a component, such as a speed sensor (e.g., a speedometer), for measuring the speed of vehicle 200. System 100 may also include one or more accelerometers (single-axis or multi-axis) for measuring the acceleration of vehicle 200 along one or more axes.

[0095] The memory units 140, 150 may include a database indicating the locations of known landmarks or data organized in any other form. Sensor information of the environment (such as images, radio detection and ranging signals, depth information from light detection and ranging or stereo processing of two or more images) may be processed together with position information (such as GPS coordinates, the vehicle's self-motion, etc.) to determine the vehicle's current position relative to the known landmarks and to refine the vehicle's position. Certain aspects of this technology are included in a process called REM. TM The positioning technology is being sold by the assignee of this application.

[0096] The user interface 170 may include any device suitable for providing information to or receiving input from one or more users of the system 100. In some embodiments, the user interface 170 may include user input devices including, for example, a touch screen, a microphone, a keyboard, a pointing device, a track wheel, a camera, knobs, buttons, etc. Using such input devices, a user may be able to provide information input or commands to the system 100 by typing instructions or information, providing voice commands, selecting menu options on a screen using buttons, a pointer, or eye tracking capabilities, or by any other suitable technique for transmitting information to the system 100.

[0097] The user interface 170 may be equipped with one or more processing devices configured to provide information to or receive information from a user and process the information for use, for example, by the application processor 180. In some embodiments, such processing devices may execute instructions for recognizing and tracking eye movements, receiving and interpreting voice commands, recognizing and interpreting touches and / or gestures made on a touch screen, responding to keyboard input or menu selections, etc. In some embodiments, the user interface 170 may include a display, a speaker, a haptic device, and / or any other device for providing output information to a user.

[0098] Map database 160 may comprise any type of database for storing map data useful to system 100. In some embodiments, map database 160 may include data relating to the locations of various items (including roads, water features, geographic features, businesses, points of interest, restaurants, gas stations, etc.) in a reference coordinate system. Map database 160 may store not only the locations of such items but also descriptors associated with these items, including, for example, names associated with any of the stored features. In some embodiments, map database 160 may be physically located with the other components of system 100. Alternatively or additionally, map database 160 or portions thereof may be remotely located relative to other components of system 100 (e.g., processing unit 110). In such embodiments, information from map database 160 may be downloaded via a wired or wireless data connection to a network (e.g., via a cellular network and / or the internet, etc.). In some cases, map database 160 may store a sparse data model comprising a polynomial representation of certain road features (e.g., lane markings) or target trajectories for the host vehicle. The map database 160 may also include stored representations of various identified landmarks that may be used to determine or update the known position of the host vehicle relative to the target trajectory. The landmark representation may include data fields such as landmark type, landmark location, and other potential identifiers.

[0099] Image capture devices 122, 124, and 126 may each comprise any type of device suitable for capturing at least one image from an environment. Furthermore, any number of image capture devices may be used to collect images for input to the image processor. Some embodiments may include only a single image capture device, while other embodiments may include two, three, or even four or more image capture devices. Figure 2B-2E Image capture devices 122, 124, and 126 are further described.

[0100] One or more cameras (e.g., image capture devices 122, 124, and 126) may be part of a sensing block included onboard a vehicle. Various other sensors may be included in the sensing block, and any or all of these sensors may be relied upon to form the vehicle's sensed navigation state. In addition to cameras (forward, side, rear, etc.), other sensors (such as radio detection and ranging, light detection and ranging, and acoustic sensors) may also be included in the sensing block. Additionally, the sensing block may include one or more components configured to transmit and / or receive information related to the vehicle's environment. For example, such components may include wireless transceivers (RF, etc.) that can receive sensor-based information or any other type of information related to the host vehicle's environment from a source remotely located relative to the host vehicle. Such information may include sensor output information or related information received from vehicle systems other than the host vehicle. In some embodiments, such information may include information received from a remote computing device, a centralized server, or the like. Furthermore, cameras may be configured in a variety of different ways: single-camera units, multiple cameras, camera clusters, with long field of view (FOV), short field of view (FOV), wide-angle, fisheye, and the like.

[0101] The system 100 or its various components may be incorporated into a variety of different platforms. In some embodiments, the system 100 may be included on a vehicle 200, such as Figure 2A For example, as shown above with respect to Figure 1 As described, vehicle 200 may be equipped with processing unit 110 and any other components of system 100. While in some embodiments, vehicle 200 may be equipped with only a single image capture device (e.g., a camera), in other embodiments (such as in combination with a plurality of image capture devices), vehicle 200 may be equipped with a plurality of image capture devices (e.g., a camera). Figure 2B-2E In the embodiments discussed above, multiple image capture devices may be used. For example, Figure 2A As shown, either of the image capture devices 122 and 124 of the vehicle 200 may be part of an ADAS (Advanced Driver Assistance System) imaging set.

[0102] The image capture device included on the vehicle 200 as part of the image acquisition unit 120 can be positioned at any suitable location. Figure 2A-2E and Figure 3A-3C As shown, the image capture device 122 can be located near the rearview mirror. This location can provide a line of sight similar to the line of sight of the driver of the vehicle 200, which can assist in determining what is and is not visible to the driver. The image capture device 122 can be positioned anywhere near the rearview mirror, but placing the image capture device 122 on the driver's side of the rearview mirror can further assist in obtaining an image representative of the driver's field of view and / or line of sight.

[0103] Other locations for the image capture device of image acquisition unit 120 can also be used. For example, image capture device 124 can be located on or in the bumper of vehicle 200. Such locations may be particularly suitable for image capture devices with a wide field of view. The line of sight of the image capture device located at the bumper may be different from the line of sight of the driver, and therefore, the bumper image capture device and the driver may not always see the same object. Image capture devices (e.g., image capture devices 122, 124, and 126) can also be located at other locations. For example, the image capture device can be located on or in one or both of the side mirrors of vehicle 200, on the roof of vehicle 200, on the hood of vehicle 200, on the trunk of vehicle 200, on the side of vehicle 200, mounted on any of the windows of vehicle 200, positioned behind or in front of any of the windows of vehicle 200, and mounted in or near the front and / or rear light fixtures of vehicle 200, etc.

[0104] In addition to the image capture device, the vehicle 200 may also include various other components of the system 100. For example, the processing unit 110 may be included on the vehicle 200, integrated with the vehicle's engine control unit (ECU), or separate. The vehicle 200 may also be equipped with a location sensor 130 (such as a GPS receiver) and may also include a map database 160 and memory units 140 and 150.

[0105] As previously discussed, the wireless transceiver 172 can receive data over and / or through one or more networks (e.g., a cellular network, the Internet, etc.). For example, the wireless transceiver 172 can upload data collected by the system 100 to one or more servers and download data from one or more servers. Via the wireless transceiver 172, the system 100 can receive, for example, periodic or on-demand updates to data stored in the map database 160, the memory 140, and / or the storage 150. Similarly, the wireless transceiver 172 can upload any data from the system 100 (e.g., images captured by the image acquisition unit 120, data received by the position sensor 130 or other sensors, the vehicle control system, etc.) and / or any data processed by the processing unit 110 to one or more servers.

[0106] The system 100 can upload data to a server (e.g., to the cloud) based on a privacy level setting. For example, the system 100 can implement privacy level settings to regulate or limit the type of data (including metadata) sent to a server that can uniquely identify a vehicle and / or a driver / owner of the vehicle. Such settings can be set by a user via, for example, the wireless transceiver 172, by factory default settings, or initialized by data received by the wireless transceiver 172.

[0107] In some embodiments, the system 100 may upload data according to a "high" privacy level, and under the setting, the system 100 may transmit data (e.g., location information related to the route, captured images, etc.) without any details about the specific vehicle and / or driver / owner. For example, when uploading data according to the "high" privacy setting, the system 100 may not include the vehicle identification number (VIN) or the name of the vehicle driver or owner, and may instead transmit data such as captured images and / or limited location information related to the route.

[0108] Other privacy levels are also contemplated. For example, the system 100 may transmit data to a server according to an "intermediate" privacy level and include additional information that is not included at a "high" privacy level, such as the make and / or model of the vehicle and / or the type of vehicle (e.g., a passenger car, a sport utility vehicle, a truck, etc.). In some embodiments, the system 100 may upload data according to a "low" privacy level. Under the "low" privacy level setting, the system 100 may upload data and include information sufficient to uniquely identify a particular vehicle, owner / driver, and / or part or all of the route traveled by the vehicle. Such "low" privacy level data may include, for example, one or more of the following: VIN, driver / owner name, starting point of the vehicle before departure, intended destination of the vehicle, make and / or model of the vehicle, type of vehicle, etc.

[0109] Figure 2A is a diagrammatic side view representation of an exemplary vehicle imaging system consistent with the disclosed embodiments. Figure 2B yes Figure 2A A diagrammatic top view of the embodiment shown in FIG. Figure 2B As illustrated, the disclosed embodiments may include a vehicle 200 including a system 100 in the body of the vehicle 200, the system 100 having a first image capture device 122 positioned near a rearview mirror of the vehicle 200 and / or positioned close to a driver of the vehicle 200, a second image capture device 124 positioned on or in a bumper area (e.g., one of the bumper areas 210) of the vehicle 200, and a processing unit 110.

[0110] like Figure 2C As illustrated, both image capture devices 122 and 124 may be positioned near a rearview mirror of vehicle 200 and / or positioned close to the driver of vehicle 200. Additionally, while Figure 2B and Figure 2C Two image capture devices 122 and 124 are shown in FIG, but it should be understood that other embodiments may include more than two image capture devices. Figure 2D and Figure 2E In the illustrated embodiment, a first image capture device 122 , a second image capture device 124 , and a third image capture device 126 are included in the system 100 of a vehicle 200 .

[0111] like Figure 2D As illustrated, image capture device 122 may be positioned near a rearview mirror of vehicle 200 and / or positioned close to the driver of vehicle 200, while image capture devices 124 and 126 may be positioned on or in a bumper area (e.g., one of bumper areas 210) of vehicle 200. Figure 2E As shown, image capture devices 122, 124, and 126 may be positioned near rearview mirrors of vehicle 200 and / or positioned proximate to a driver's seat of vehicle 200. The disclosed embodiments are not limited to any particular number and configuration of image capture devices, and the image capture devices may be positioned at any suitable location within and / or on vehicle 200.

[0112] It should be understood that the disclosed embodiments are not limited to vehicles and can be applied to other scenarios. It should also be understood that the disclosed embodiments are not limited to a specific type of vehicle 200 and can be applicable to all types of vehicles, including cars, trucks, trailers, and other types of vehicles.

[0113] The first image capture device 122 may include any suitable type of image capture device. The image capture device 122 may include an optical axis. In one example, the image capture device 122 may include an Aptina M9V024WVGA sensor with a global shutter. In other embodiments, the image capture device 122 may provide a resolution of 1280x960 pixels and may include a rolling shutter. The image capture device 122 may include various optical elements. In some embodiments, one or more lenses may be included, for example, to provide a desired focal length and field of view for the image capture device. In some embodiments, the image capture device 122 may be associated with a 6 mm lens or a 12 mm lens. In some embodiments, the image capture device 122 may be configured to capture an image having a desired field of view (FOV) 202, such as Figure 2D As shown. For example, the image capture device 122 can be configured to have a conventional FOV, such as in the range of 40 degrees to 56 degrees, including a 46-degree FOV, a 50-degree FOV, a 52-degree FOV, or a larger FOV. Alternatively, the image capture device 122 can be configured to have a narrow FOV in the range of 23 to 40 degrees, such as a 28-degree FOV or a 36-degree FOV. In addition, the image capture device 122 can be configured to have a wide FOV in the range of 100 to 180 degrees. In some embodiments, the image capture device 122 may include a wide-angle bumper camera or a camera with up to 180-degree FOV. In some embodiments, the image capture device 122 may be a 7.2M pixel image capture device having an aspect ratio of approximately 2:1 (e.g., HxV=3800x1900 pixels) and having a horizontal FOV of approximately 100 degrees. Such an image capture device can be used instead of a three-image capture device configuration. Due to significant lens distortion, the vertical FOV of such an image capture device may be significantly less than 50 degrees in implementations where the image capture device uses radially symmetric lenses. For example, such lenses may not be radially symmetric, which would allow the vertical FOV to be greater than 50 degrees while the horizontal FOV is 100 degrees.

[0114] The first image capture device 122 can capture a plurality of first images of a scene associated with the vehicle 200. Each of the plurality of first images can be captured as a series of image scan lines, which can be captured using a rolling shutter. Each scan line can include a plurality of pixels.

[0115] The first image capture device 122 may have a scan rate associated with the acquisition of each image scan line in the first series of image scan lines. The scan rate may refer to the rate at which the image sensor can acquire image data associated with each pixel included in a particular scan line.

[0116] Image capture devices 122, 124, and 126 may include any suitable type and number of image sensors, including, for example, CCD sensors or CMOS sensors. In one embodiment, a CMOS image sensor may be used with a rolling shutter, such that each pixel in a row is read one at a time, and the rows are scanned on a row-by-row basis until the entire image frame is captured. In some embodiments, the rows may be captured sequentially from top to bottom relative to the frame.

[0117] In some embodiments, one or more of the image capture devices disclosed herein (e.g., image capture devices 122, 124, and 126) may constitute a high-resolution imager and may have a resolution greater than 5 M pixels, 7 M pixels, 10 M pixels, or greater.

[0118] The use of a rolling shutter may cause pixels in different rows to be exposed and captured at different times, which may result in skew and other image artifacts in the captured image frame. On the other hand, when image capture device 122 is configured to operate using a global or synchronized shutter, all pixels may be exposed for the same amount of time and within a common exposure period. As a result, the image data in a frame collected from the system using a global shutter represents a snapshot of the entire FOV (such as FOV 202) at a specific time. In contrast, in a rolling shutter application, each row in the frame is exposed at a different time and data is captured at a different time. As a result, moving objects may appear distorted in an image capture device with a rolling shutter. This phenomenon is described in more detail below.

[0119] Second image capture device 124 and third image capture device 126 can be any type of image capture device. Similar to first image capture device 122, each of image capture devices 124 and 126 can include an optical axis. In one embodiment, each of image capture devices 124 and 126 can include an Aptina M9V024 WVGA sensor with a global shutter. Alternatively, each of image capture devices 124 and 126 can include a rolling shutter. Similar to image capture device 122, image capture devices 124 and 126 can be configured to include various lenses and optical elements. In some embodiments, the lenses associated with image capture devices 124 and 126 can provide a FOV (such as FOVs 204 and 206) that is the same as or narrower than the FOV associated with image capture device 122 (such as FOV 202). For example, image capture devices 124 and 126 can have a FOV of 40 degrees, 30 degrees, 26 degrees, 23 degrees, 20 degrees, or less.

[0120] Image capture devices 124 and 126 can capture a plurality of second and third images related to a scene associated with vehicle 200. Each of the plurality of second and third images can be captured as a second series of image scan lines and a third series of image scan lines, which can be captured using a rolling shutter. Each scan line or scan row can have a plurality of pixels. Image capture devices 124 and 126 can have second and third scan rates associated with capturing each of the image scan lines included in the second and third series.

[0121] Each image capture device 122, 124, and 126 can be positioned at any suitable location and orientation relative to vehicle 200. The relative positioning of image capture devices 122, 124, and 126 can be selected to facilitate fusing information collected from the image capture devices. For example, in some embodiments, the FOV associated with image capture device 124 (such as FOV 204) can partially or completely overlap the FOV associated with image capture device 122 (such as FOV 202), and can partially or completely overlap the FOV associated with image capture device 126 (such as FOV 206).

[0122] Image capture devices 122, 124, and 126 may be located at any suitable relative heights on vehicle 200. In one example, there may be height differences between image capture devices 122, 124, and 126, which may provide sufficient parallax information to enable stereo analysis. Figure 2A As shown, the two image capture devices 122 and 124 are at different heights. There may also be lateral displacement differences between the image capture devices 122, 124, and 126, thereby providing additional parallax information for stereo analysis by the processing unit 110, for example. The lateral displacement difference can be expressed as d x Indicates that Figure 2C and Figure 2D In some embodiments, there can be a fore-and-aft displacement (e.g., a range displacement) between image capture devices 122, 124, and 126. For example, image capture device 122 can be positioned 0.5 meters to 2 meters or more behind image capture device 124 and / or image capture device 126. This type of displacement can enable one image capture device to cover a potential blind spot of the other(s).

[0123] Image capture device 122 may have any suitable resolution capability (e.g., the number of pixels associated with the image sensor), and the resolution of the image sensor(s) associated with image capture device 122 may be higher, lower, or the same as the resolution of the image sensor(s) associated with image capture devices 124 and 126. In some embodiments, the image sensor(s) associated with image capture device 122 and / or image capture devices 124 and 126 may have a resolution of 640x480, 1024x768, 1280x960, or any other suitable resolution.

[0124] The frame rate (e.g., the rate at which an image capture device acquires a set of pixel data for one image frame before continuing to capture pixel data associated with the next image frame) can be controllable. The frame rate associated with image capture device 122 can be higher, lower, or the same as the frame rate associated with image capture devices 124 and 126. The frame rate associated with image capture devices 122, 124, and 126 can depend on various factors that can affect the timing of the frame rate. For example, one or more of image capture devices 122, 124, and 126 can include an optional pixel delay period that is applied before or after the acquisition of image data associated with one or more pixels of the image sensors in image capture devices 122, 124, and / or 126. Generally, image data corresponding to each pixel can be acquired according to the clock rate of the device (e.g., one pixel per clock cycle). Additionally, in embodiments including a rolling shutter, one or more of image capture devices 122, 124, and 126 may include an optional horizontal blanking period that is applied before or after the acquisition of image data associated with a row of pixels of an image sensor in image capture devices 122, 124, and / or 126. Furthermore, one or more of image capture devices 122, 124, and / or 126 may include an optional vertical blanking period that is applied before or after the acquisition of image data associated with an image frame of image capture devices 122, 124, and 126.

[0125] These timing controls can enable synchronization of the frame rates associated with image capture devices 122, 124, and 126, even in situations where the line scan rate of each device is different. Additionally, as will be discussed in greater detail below, these optional timing controls can enable synchronization of image capture from areas where the FOV of image capture device 122 overlaps with one or more FOVs of image capture devices 124 and 126, even in situations where the FOV of image capture device 122 is different from the FOVs of image capture devices 124 and 126, in addition to other factors (e.g., image sensor resolution, maximum line scan rate, etc.).

[0126] The frame rate timing in image capture devices 122, 124, and 126 may depend on the resolution of the associated image sensors. For example, assuming similar line scan rates for both devices, if one device includes an image sensor with a resolution of 640x480 and the other device includes an image sensor with a resolution of 1280x960, more time will be required to acquire a frame of image data from the sensor with the higher resolution.

[0127] Another factor that may affect the timing of image data acquisition in image capture devices 122, 124, and 126 is the maximum line scan rate. For example, it will take a certain minimum amount of time to acquire a line of image data from the image sensors included in image capture devices 122, 124, and 126. Assuming no pixel delay period is added, the minimum amount of time to acquire a line of image data will be related to the maximum line scan rate of the particular device. Devices that offer higher maximum line scan rates have the potential to provide higher frame rates than devices with lower maximum line scan rates. In some embodiments, one or more of image capture devices 124 and 126 may have a maximum line scan rate that is higher than the maximum line scan rate associated with image capture device 122. In some embodiments, the maximum line scan rate of image capture devices 124 and / or 126 may be 1.25, 1.5, 1.75, or 2 times or more the maximum line scan rate of image capture device 122.

[0128] In another embodiment, image capture devices 122, 124, and 126 may have the same maximum line scan rate, but image capture device 122 may operate at a scan rate less than or equal to its maximum scan rate. The system may be configured such that one or more of image capture devices 124 and 126 operate at a line scan rate equal to the line scan rate of image capture device 122. In other examples, the system may be configured such that the line scan rate of image capture device 124 and / or image capture device 126 may be 1.25, 1.5, 1.75, or 2 times, or more, greater than the line scan rate of image capture device 122.

[0129] In some embodiments, image capture devices 122, 124, and 126 may be asymmetric. That is, they may include cameras with different fields of view (FOVs) and focal lengths. For example, the fields of view of image capture devices 122, 124, and 126 may include any desired area of ​​the environment relative to vehicle 200. In some embodiments, one or more of image capture devices 122, 124, and 126 may be configured to collect image data from the environment in front of vehicle 200, behind vehicle 200, to the sides of vehicle 200, or a combination thereof.

[0130] Furthermore, the focal length associated with each image capture device 122, 124, and / or 126 can be selectable (e.g., by including an appropriate lens, etc.) so that each device captures images of objects within a desired distance range relative to the vehicle 200. For example, in some embodiments, the image capture devices 122, 124, and 126 can capture images of close-up objects within a few meters from the vehicle. The image capture devices 122, 124, and 126 can also be configured to capture images of objects within a greater range (e.g., 25 m, 50 m, 100 m, 150 m, or longer) from the vehicle. Further, the focal lengths of image capture devices 122, 124, and 126 may be selected such that one image capture device (e.g., image capture device 122) may capture images of objects that are relatively close to the vehicle (e.g., within 10 m or 20 m), while other image capture devices (e.g., image capture devices 124 and 126) may capture images of objects that are farther from the vehicle 200 (e.g., greater than 20 m, 50 m, 100 m, 150 m, etc., from the vehicle).

[0131] According to some embodiments, the FOV of one or more image capture devices 122, 124, and 126 may have a wide angle. For example, having a FOV of 140 degrees may be advantageous, particularly for image capture devices 122, 124, and 126 that may be used to capture images of areas adjacent to vehicle 200. For example, image capture device 122 may be used to capture images of areas to the right or left of vehicle 200, and in such embodiments, it may be desirable for image capture device 122 to have a wide FOV (e.g., at least 140 degrees).

[0132] The field of view associated with each of image capture devices 122, 124, and 126 may depend on the respective focal length. For example, as the focal length increases, the corresponding field of view decreases.

[0133] Image capture devices 122, 124, and 126 can be configured to have any suitable field of view. In one specific example, image capture device 122 can have a horizontal FOV of 46 degrees, image capture device 124 can have a horizontal FOV of 23 degrees, and image capture device 126 can have a horizontal FOV between 23 and 46 degrees. In another example, image capture device 122 can have a horizontal FOV of 52 degrees, image capture device 124 can have a horizontal FOV of 26 degrees, and image capture device 126 can have a horizontal FOV between 26 and 52 degrees. In some embodiments, the ratio of the FOV of image capture device 122 to the FOV of image capture device 124 and / or image capture device 126 can vary between 1.5 and 2.0. In other embodiments, the ratio can vary between 1.25 and 2.25.

[0134] System 100 can be configured such that the field of view of image capture device 122 at least partially or completely overlaps the field of view of image capture device 124 and / or image capture device 126. In some embodiments, system 100 can be configured such that the fields of view of image capture devices 124 and 126, for example, fall within the field of view of image capture device 122 (e.g., are narrower than the field of view of image capture device 122) and share a common center with the field of view of image capture device 122. In other embodiments, image capture devices 122, 124, and 126 can capture adjacent FOVs or can have partial overlap in their FOVs. In some embodiments, the fields of view of image capture devices 122, 124, and 126 can be aligned such that the center of image capture devices 124 and / or 126, having narrower FOVs, can be located in the lower half of the field of view of device 122, having a wider FOV.

[0135] Figure 2F is a diagrammatic representation of an exemplary vehicle control system consistent with the disclosed embodiments. Figure 2FAs shown, the vehicle 200 may include a throttle system 220, a braking system 230, and a steering system 240. The system 100 may provide input (e.g., control signals) to one or more of the throttle system 220, the braking system 230, and the steering system 240 via one or more data links (e.g., any one or more wired or wireless links for transmitting data). For example, based on an analysis of images captured by the image capture devices 122, 124, or 126, the system 100 may provide control signals to one or more of the throttle system 220, the braking system 230, and the steering system 240 to navigate the vehicle 200 (e.g., by causing acceleration, turning, lane deviation, etc.). Further, the system 100 may receive input from one or more of the throttle system 220, the braking system 230, and the steering system 24 indicating the operating condition of the vehicle 200 (e.g., speed, whether the vehicle 200 is braking or turning, etc.). The following is in conjunction with Figure 4-Figure 7 Provide further details.

[0136] like Figure 3A As shown, the vehicle 200 may also include a user interface 170 for interacting with the driver or passengers of the vehicle 200. For example, the user interface 170 in the vehicle application may include a touch screen 320, a knob 330, a button 340, and a microphone 350. The driver or passenger of the vehicle 200 may also use a handle (e.g., a handle located on or near the steering column of the vehicle 200, including, for example, a turn signal handle), a button (e.g., a button located on the steering wheel of the vehicle 200), etc. to interact with the system 100. In some embodiments, the microphone 350 may be positioned adjacent to the rearview mirror 310. Similarly, in some embodiments, the image capture device 122 may be located near the rearview mirror 310. In some embodiments, the user interface 170 may also include one or more speakers 360 (e.g., speakers of the vehicle audio system). For example, the system 100 may provide various notifications (e.g., alarms) via the speakers 360.

[0137] Figure 3B-3D is an illustration of an exemplary camera bracket 370 configured to be positioned behind a rearview mirror (e.g., rearview mirror 310) and against a vehicle windshield, consistent with the disclosed embodiments. Figure 3BAs shown, camera support 370 can include image capture devices 122, 124, and 126. Image capture devices 124 and 126 can be located behind a glare shield 380, which can be flush against the vehicle windshield and include a combination of film and / or anti-reflective material. For example, glare shield 380 can be positioned so that glare shield 380 is aligned against a vehicle windshield having a matching slope. In some embodiments, each of image capture devices 122, 124, and 126 can be located behind glare shield 380, for example, as shown in FIG. Figure 3D The disclosed embodiments are not limited to any particular configuration of image capture devices 122 , 124 , and 126 , camera support 370 , and glare shield 380 . Figure 3C yes Figure 3B An illustration of the camera support 370 shown in FIG. 3 from a front view.

[0138] As will be appreciated by those skilled in the art having the benefit of this disclosure, many variations and / or modifications may be made to the aforementioned disclosed embodiments. For example, not all components are necessary for the operation of the system 100. Further, any component may be located in any appropriate portion of the system 100, and the components may be rearranged into various configurations while providing the functionality of the disclosed embodiments. Therefore, the preceding configurations are examples, and regardless of the configurations discussed above, the system 100 may provide a wide range of functionality to analyze the surroundings of the vehicle 200 and navigate the vehicle 200 in response to the analysis.

[0139] As discussed in further detail below and consistent with the various disclosed embodiments, the system 100 can provide various features related to autonomous driving and / or driver assistance technology. For example, the system 100 can analyze image data, location data (e.g., GPS location information), map data, speed data, and / or data from sensors included in the vehicle 200. The system 100 can collect data from, for example, the image acquisition unit 120, the location sensor 130, and other sensors for analysis. Further, the system 100 can analyze the collected data to determine whether the vehicle 200 should take a certain action, and then automatically take the determined action without human intervention. For example, when the vehicle 200 is navigating without human intervention, the system 100 can automatically control the braking, acceleration, and / or steering of the vehicle 200 (e.g., by sending control signals to one or more of the throttle system 220, the braking system 230, and the steering system 240). Further, the system 100 can analyze the collected data and, based on the analysis of the collected data, issue warnings and / or alerts to the vehicle occupants. Additional details regarding various embodiments provided by system 100 are provided below.

[0140] Forward multi-imaging system

[0141] As discussed above, the system 100 can provide driving assistance functionality using a multi-camera system. The multi-camera system can use one or more cameras facing the direction of travel of the vehicle. In other embodiments, the multi-camera system can include one or more cameras facing the side of the vehicle or the rear of the vehicle. In one embodiment, for example, the system 100 can use a dual-camera imaging system, wherein the first camera and the second camera (e.g., image capture devices 122 and 124) can be positioned at the front and / or side of the vehicle (e.g., vehicle 200). Other camera configurations are consistent with the disclosed embodiments, and the configurations disclosed herein are examples. For example, the system 100 can include a configuration of any number of cameras (e.g., one, two, three, four, five, six, seven, eight, etc.). In addition, the system 100 can include a "cluster" of cameras. For example, a camera cluster (comprising any appropriate number of cameras, e.g., one, four, eight, etc.) can be forward-facing relative to the vehicle, or can face any other direction (e.g., facing back, facing to the side, at an angle, etc.). Thus, system 100 can include multiple camera clusters, with each cluster oriented in a particular direction to capture images from a particular area of ​​the vehicle environment.

[0142] The field of view of the first camera may be larger than, smaller than, or partially overlap the field of view of the second camera. In addition, the first camera may be connected to a first image processor for performing monocular image analysis on the image provided by the first camera, and the second camera may be connected to a second image processor for performing monocular image analysis on the image provided by the second camera. The outputs of the first and second image processors (e.g., processed information) may be combined. In some embodiments, the second image processor may receive images from both the first and second cameras to perform stereo analysis. In another embodiment, the system 100 may use a three-camera imaging system, each camera having a different field of view. Thus, the system can make decisions based on information derived from objects located at different distances in front of and to the sides of the vehicle. References to monocular image analysis may refer to instances of performing image analysis based on images captured from a single viewpoint (e.g., from a single camera). Stereo image analysis may refer to instances of performing image analysis based on two or more images captured with one or more variations of image capture parameters. For example, captured images suitable for performing stereoscopic image analysis may include: images captured from two or more different positions, images captured from different fields of view, images captured using different focal lengths, images captured together with disparity information, etc.

[0143] For example, in one embodiment, system 100 may implement a three-camera configuration using image capture devices 122-126. In such a configuration, image capture device 122 may provide a narrow field of view (e.g., 34 degrees or other values ​​selected from a range of approximately 20 to 45 degrees), image capture device 124 may provide a wide field of view (e.g., 150 degrees or other values ​​selected from a range of approximately 100 to approximately 180 degrees), and image capture device 126 may provide an intermediate field of view (e.g., 46 degrees or other values ​​selected from a range of approximately 35 to approximately 60 degrees). In some embodiments, image capture device 126 may serve as the primary or main camera. Image capture devices 122-126 may be positioned behind rearview mirror 310 and positioned substantially side-by-side (e.g., 6 cm apart). Further, in some embodiments, as discussed above, one or more of image capture devices 122-126 may be mounted behind a glare shield 380 flush with the windshield of vehicle 200. Such shielding may be used to minimize the effect of any reflections from within the vehicle on the image capture devices 122 - 126 .

[0144] In another embodiment, as described above in combination with Figure 3B and Figure 3C As discussed, the wide field of view camera (e.g., image capture device 124 in the above example) can be mounted lower than the narrow field of view and main field of view cameras (e.g., image devices 122 and 126 in the above example). This configuration can provide a clear line of sight from the wide field of view camera. To reduce reflections, the camera can be mounted close to the windshield of the vehicle 200 and a polarizer can be included on the camera to suppress reflected light.

[0145] A three-camera system can provide certain performance characteristics. For example, some embodiments may include the ability to verify the detection of an object by one camera based on the detection results from another camera. In the three-camera configuration discussed above, the processing unit 110 may include, for example, three processing devices (e.g., three EyeQ series processor chips as discussed above), where each processing device is dedicated to processing images captured by one or more of the image capture devices 122-126.

[0146] In a three-camera system, the first processing device can receive images from both the main camera and the narrow field of view camera and perform vision processing on the narrow FOV camera to, for example, detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Furthermore, the first processing device can calculate pixel disparity between the image from the main camera and the image from the narrow camera and create a 3D reconstruction of the environment of the vehicle 200. The first processing device can then combine the 3D reconstruction with 3D map data or with 3D information calculated based on information from another camera.

[0147] The second processing device can receive images from the primary camera and perform visual processing to detect other vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. Additionally, the second processing device can calculate camera displacement and, based on this displacement, calculate pixel disparity between consecutive images and create a 3D reconstruction of the scene (e.g., structure from motion). The second processing device can send the structure from motion-based 3D reconstruction to the first processing device for combination with the stereoscopic 3D image.

[0148] The third processing device can receive images from the wide FOV camera and process these images to detect vehicles, pedestrians, lane markings, traffic signs, traffic lights, and other road objects. The third processing device can further execute additional processing instructions to analyze the images to identify moving objects in the images, such as vehicles changing lanes, pedestrians, etc.

[0149] In some embodiments, independently capturing and processing information based on the image streams may provide an opportunity to provide redundancy in the system. Such redundancy may include, for example, using a first image capture device and images processed from that device to verify and / or supplement information obtained by capturing and processing image information from at least a second image capture device.

[0150] In some embodiments, system 100 may use two image capture devices (e.g., image capture devices 122 and 124) to provide navigation assistance for vehicle 200, and use a third image capture device (e.g., image capture device 126) to provide redundancy and validate the analysis of data received from the other two image capture devices. For example, in such a configuration, image capture devices 122 and 124 may provide images for stereo analysis performed by system 100 to navigate vehicle 200, while image capture device 126 may provide images for monocular analysis performed by system 100 to provide redundancy and validation of information derived from images captured by image capture device 122 and / or image capture device 124. That is, image capture device 126 (and the corresponding processing device) may be considered to provide a redundant subsystem for providing a check on the analysis derived from image capture devices 122 and 124 (e.g., to provide an autonomous emergency braking (AEB) system). In addition, in some embodiments, the redundancy and verification of the received information can be supplemented based on information received from one or more sensors (e.g., radio detection and ranging, light detection and ranging, acoustic sensors, information received from one or more transceivers outside the vehicle, etc.).

[0151] Those skilled in the art will recognize that the above-described camera configurations, camera placements, number of cameras, camera positions, etc. are merely examples. These components, and other components described with respect to the overall system, can be assembled and used in a variety of different configurations without departing from the scope of the disclosed embodiments. Further details regarding the use of a multi-camera system to provide driver assistance and / or autonomous vehicle functionality are provided below.

[0152] Figure 4 1 is an exemplary functional block diagram of memory 140 and / or 150 that may be stored / programmed with instructions for performing one or more operations consistent with the disclosed embodiments. Although reference is made below to memory 140, those skilled in the art will recognize that instructions may be stored in memory 140 and / or 150.

[0153] like Figure 4 As shown, memory 140 may store a monocular image analysis module 402, a stereo image analysis module 404, a velocity and acceleration module 406, and a navigation response module 408. The disclosed embodiments are not limited to any particular configuration of memory 140. Further, application processor 180 and / or image processor 190 may execute instructions stored in any of modules 402-408 included in memory 140. Those skilled in the art will understand that references to processing unit 110 in the following discussion may refer individually or collectively to application processor 180 and image processor 190. Accordingly, the steps of any of the following procedures may be performed by one or more processing devices.

[0154] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis on a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensory information (e.g., information from radio detection and ranging) to perform monocular image analysis. As described below in conjunction with Figures 5A-5D As described, the monocular image analysis module 402 may include instructions for detecting a set of features within a set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other features associated with the vehicle environment. Based on this analysis, the system 100 (e.g., via the processing unit 110) may cause one or more navigation responses in the vehicle 200, such as turns, lane deviations, acceleration changes, etc., as discussed below in conjunction with the navigation response module 408.

[0155] In one embodiment, the monocular image analysis module 402 may store instructions (such as computer vision software) that, when executed by the processing unit 110, perform monocular image analysis on a set of images acquired by one of the image capture devices 122, 124, and 126. In some embodiments, the processing unit 110 may combine information from the set of images with additional sensory information (e.g., information from radio detection and ranging, light detection and ranging, etc.) to perform monocular image analysis. As described below in conjunction with Figures 5A-5D As described, the monocular image analysis module 402 may include instructions for detecting a set of features within a set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, and any other features associated with the vehicle environment. Based on this analysis, the system 100 (e.g., via the processing unit 110) may cause one or more navigational responses in the vehicle 200, such as turns, lane deviations, acceleration changes, etc., as discussed below in conjunction with determining navigational responses.

[0156] In one embodiment, stereo image analysis module 404 may store instructions (such as computer vision software) that, when executed by processing unit 110, perform stereo image analysis on a first set of images and a second set of images captured by a combination of image capture devices selected from image capture devices 122, 124, and 126. In some embodiments, processing unit 110 may combine information from the first set of images and the second set of images with additional sensory information (e.g., information from radio detection and ranging) to perform stereo image analysis. For example, stereo image analysis module 404 may include instructions for performing stereo image analysis based on a first set of images captured by image capture device 124 and a second set of images captured by image capture device 126. As described below in conjunction with Figure 6 As described, the stereo image analysis module 404 may include instructions for detecting a set of features within the first and second sets of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, hazardous objects, etc. Based on the analysis, the processing unit 110 may cause one or more navigation responses in the vehicle 200, such as turns, lane deviations, acceleration changes, etc., as discussed below in conjunction with the navigation response module 408. Furthermore, in some embodiments, the stereo image analysis module 404 may implement techniques associated with a trained system (e.g., a neural network or a deep neural network) or an untrained system.

[0157] In one embodiment, the speed and acceleration module 406 may store software configured to analyze data received from one or more computing and electromechanical devices in the vehicle 200, which are configured to cause changes in the speed and / or acceleration of the vehicle 200. For example, the processing unit 110 may execute instructions associated with the speed and acceleration module 406 to calculate a target speed for the vehicle 200 based on data derived from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include, for example, target position, speed, and / or acceleration, the position and / or speed of the vehicle 200 relative to nearby vehicles, pedestrians, or road objects, position information of the vehicle 200 relative to road lane markings, etc. In addition, the processing unit 110 may calculate the target speed for the vehicle 200 based on sensory input (e.g., information from radio detection and ranging) and input from other systems of the vehicle 200, such as the throttle system 220, the braking system 230, and / or the steering system 240 of the vehicle 200. Based on the calculated target speed, the processing unit 110 can transmit electronic signals to the throttle system 220, the braking system 230 and / or the steering system 240 of the vehicle 200 to trigger a change in speed and / or acceleration by, for example, physically depressing the brakes of the vehicle 200 or releasing the accelerator of the vehicle 200.

[0158] In embodiments, the navigation response module 408 may store software executable by the processing unit 110 to determine a desired navigation response based on data derived from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. Such data may include position and velocity information associated with nearby vehicles, pedestrians, and road objects, target position information for the vehicle 200, and the like. Additionally, in some embodiments, the navigation response may be based (in part or in whole) on map data, a predetermined position of the vehicle 200, and / or a relative velocity or acceleration between the vehicle 200 and one or more objects detected from the execution of the monocular image analysis module 402 and / or the stereo image analysis module 404. The navigation response module 408 may also determine the desired navigation response based on sensory input (e.g., information from radio detection and ranging) and input from other systems of the vehicle 200, such as the throttle system 220, the braking system 230, and the steering system 240 of the vehicle 200. Based on the desired navigation response, the processing unit 110 may transmit electronic signals to the throttle system 220, the brake system 230, and the steering system 240 of the vehicle 200 to trigger the desired navigation response by, for example, turning the steering wheel of the vehicle 200 to achieve a predetermined angle of rotation. In some embodiments, the processing unit 110 may use the output of the navigation response module 408 (e.g., the desired navigation response) as input to execute the speed and acceleration module 406 to calculate the change in the speed of the vehicle 200.

[0159] Furthermore, any of the modules disclosed herein (e.g., modules 402, 404, and 406) may implement techniques associated with trained systems (such as neural networks or deep neural networks) or untrained systems.

[0160] Figure 5A is a flow chart illustrating an exemplary process 500A for eliciting one or more navigation responses based on a monocular image consistent with the disclosed embodiments. At step 510, the processing unit 110 may receive a plurality of images via the data interface 128 between the processing unit 110 and the image acquisition unit 120. For example, a camera included in the image acquisition unit 120 (such as an image capture device 122 having a field of view 202) may capture a plurality of images of an area in front of the vehicle 200 (or, for example, to the side or rear of the vehicle) and transmit these images to the processing unit 110 via a data connection (e.g., digital, wired, USB, wireless, Bluetooth, etc.). The processing unit 110 may execute the monocular image analysis module 402 to analyze the plurality of images at step 520, as described below in conjunction with Figure 5B-5DBy performing the analysis, the processing unit 110 can detect a set of features within a set of images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, and the like.

[0161] The processing unit 110 may also execute the monocular image analysis module 402 in step 520 to detect various road hazards, such as, for example, parts of truck tires, fallen road signs, loose cargo, small animals, and the like. Road hazards may vary in structure, shape, size, and color, which may make detecting such hazards more challenging. In some embodiments, the processing unit 110 may execute the monocular image analysis module 402 to perform multi-frame analysis on multiple images to detect road hazards. For example, the processing unit 110 may estimate camera motion between consecutive image frames and calculate pixel disparity between frames to construct a 3D map of the road. The processing unit 110 may then use the 3D map to detect the road surface and hazards present on the road surface.

[0162] At step 530, the processing unit 110 may execute the navigation response module 408 to perform the navigation response based on the analysis performed at step 520 and the above-mentioned combination. Figure 4 The described techniques cause one or more navigation responses in the vehicle 200. The navigation responses may include, for example, turns, lane deviations, changes in acceleration, etc. In some embodiments, the processing unit 110 may use data derived from the execution of the speed and acceleration module 406 to cause the one or more navigation responses. Furthermore, multiple navigation responses may occur simultaneously, sequentially, or in any combination of these. For example, the processing unit 110 may cause the vehicle 200 to deviate through a lane and then accelerate by, for example, sequentially transmitting control signals to the steering system 240 and the throttle system 220 of the vehicle 200. Alternatively, the processing unit 110 may cause the vehicle 200 to brake while deviating from the lane by, for example, simultaneously transmitting control signals to the braking system 230 and the steering system 240 of the vehicle 200.

[0163] Figure 5Bis a flow chart illustrating an exemplary process 500B for detecting one or more vehicles and / or pedestrians in a set of images consistent with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500B. At step 540, processing unit 110 may determine a set of candidate objects representing possible vehicles and / or pedestrians. For example, processing unit 110 may scan one or more images, compare the images to one or more predetermined patterns, and identify possible locations within each image that may contain objects of interest (e.g., vehicles, pedestrians, or portions thereof). The predetermined patterns may be designed to achieve a high "false hit rate" and a low "miss rate." For example, processing unit 110 may use a low similarity threshold to the predetermined pattern for identifying a candidate object as a possible vehicle or pedestrian. Doing so may allow processing unit 110 to reduce the probability of missing (e.g., not identifying) a candidate object representing a vehicle or pedestrian.

[0164] At step 542, processing unit 110 may filter the candidate object set based on classification criteria to exclude certain candidates (e.g., irrelevant or less relevant objects). Such criteria may be derived from various attributes associated with object types stored in a database (e.g., a database stored in memory 140). Attributes may include object shape, size, texture, location (e.g., relative to vehicle 200), etc. Thus, processing unit 110 may use one or more sets of criteria to reject false candidates from the candidate object set.

[0165] At step 544, the processing unit 110 may analyze the multiple frames to determine whether an object in the set of candidate objects represents a vehicle and / or a pedestrian. For example, the processing unit 110 may track the detected candidate object across consecutive frames and accumulate frame-by-frame data associated with the detected object (e.g., size, position relative to the vehicle 200, etc.). Additionally, the processing unit 110 may estimate parameters of the detected object and compare the frame-by-frame position data of the object with the predicted position.

[0166] In step 546, the processing unit 110 constructs a set of measurements for the detected object. Such measurements may include, for example, position, velocity, and acceleration values ​​(relative to the vehicle 200) associated with the detected object. In some embodiments, the processing unit 110 may construct the measurements based on an estimation technique using a series of time-based observations (such as a Kalman filter or linear quadratic estimation (LQE)) and / or based on available modeling data for different object types (e.g., cars, trucks, pedestrians, bicycles, road signs, etc.). The Kalman filter may be based on measurements of object scale, where the scale measurement is proportional to the time to collision (e.g., the amount of time it takes the vehicle 200 to reach the object). Thus, by executing steps 540-546, the processing unit 110 may identify vehicles and pedestrians that appear in the set of captured images and derive information associated with the vehicles and pedestrians (e.g., position, velocity, size). Based on the identified and derived information, the processing unit 110 may cause one or more navigation responses in the vehicle 200, as described above in conjunction with Figure 5A described.

[0167] At step 548, the processing unit 110 may perform an optical flow analysis on the one or more images to reduce the probability of detecting a "false hit rate" and missing a candidate object representing a vehicle or pedestrian. Optical flow analysis may refer to, for example, analyzing motion patterns relative to the vehicle 200 that are distinct from the motion of the road surface in one or more images associated with other vehicles and pedestrians. The processing unit 110 may calculate the motion of the candidate object by observing the different positions of the object across multiple image frames captured at different times. The processing unit 110 may calculate the motion of the candidate object using position and time values ​​as inputs to a mathematical model. Thus, optical flow analysis may provide another method for detecting vehicles and pedestrians in the vicinity of the vehicle 200. The processing unit 110 may perform the optical flow analysis in conjunction with steps 540-546 to provide redundancy for detecting vehicles and pedestrians and increase the reliability of the system 100.

[0168] Figure 5Cis a flow chart illustrating an exemplary process 500C for detecting road markings and / or lane geometry information in a set of images consistent with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500C. At step 550, processing unit 110 may detect a set of objects by scanning one or more images. In order to detect segments having lane markings, lane geometry information, and other relevant road markings, processing unit 110 may filter the set of objects to exclude those objects determined to be irrelevant (e.g., small potholes, small rocks, etc.). At step 552, processing unit 110 may group together the segments detected in step 550 that belong to the same road marking or lane marking. Based on the grouping, processing unit 110 may form a model, such as a mathematical model, to represent the detected segments.

[0169] At step 554, processing unit 110 may construct a set of measurements associated with the detected segment. In some embodiments, processing unit 110 may create a projection of the detected segment from the image plane to a real-world plane. The projection may be characterized using a cubic polynomial having coefficients corresponding to physical properties such as the position, slope, curvature, and curvature derivatives of the detected road. When generating the projection, processing unit 110 may take into account variations in the road surface and the pitch and roll rates associated with vehicle 200. In addition, processing unit 110 may model the road elevation by analyzing position and motion cues present on the road surface. Further, processing unit 110 may estimate the pitch and roll rates associated with vehicle 200 by tracking a set of feature points in one or more images.

[0170] At step 556, processing unit 110 may perform a multi-frame analysis by, for example, tracking the detected segment across consecutive image frames and accumulating frame-by-frame data associated with the detected segment. As processing unit 110 performs the multi-frame analysis, the set of measurements constructed at step 554 may become more reliable and associated with increasingly higher confidence levels. Thus, by executing steps 550-556, processing unit 110 may identify road markings that appear within the set of captured images and derive lane geometry information. Based on the identified and derived information, processing unit 110 may cause one or more navigation responses in vehicle 200, as described above in conjunction with Figure 5A described.

[0171] At step 558, processing unit 110 may consider additional information sources to further develop a safety model for vehicle 200 within the context of its surroundings. Processing unit 110 may use the safety model to define situations in which system 100 can safely perform autonomous control of vehicle 200. To develop the safety model, in some embodiments, processing unit 110 may consider the positions and motions of other vehicles, detected road edges and obstacles, and / or a general road shape description extracted from map data (such as data from map database 160). By considering additional information sources, processing unit 110 may provide redundancy for detecting road markings and lane geometry and improve the reliability of system 100.

[0172] Figure 5D FIG5 is a flow chart illustrating an exemplary process 500D for detecting traffic lights in a set of images consistent with the disclosed embodiments. Processing unit 110 may execute monocular image analysis module 402 to implement process 500D. At step 560, processing unit 110 may scan the set of images and identify objects that appear at locations in the images that may include traffic lights. For example, processing unit 110 may filter the identified objects to construct a set of candidate objects, excluding those that are unlikely to correspond to traffic lights. Filtering may be performed based on various characteristics associated with traffic lights, such as shape, size, texture, position (e.g., relative to vehicle 200), and so on. Such attributes may be based on multiple examples of traffic lights and traffic control signals and stored in a database. In some embodiments, processing unit 110 may perform multi-frame analysis on a set of candidate objects that reflect possible traffic lights. For example, processing unit 110 may track candidate objects across consecutive image frames, estimate their real-world positions, and filter out objects that are moving (those that are unlikely to be traffic lights). In some embodiments, processing unit 110 may perform color analysis on the candidate objects and identify the relative positions within the possible traffic lights where the detected colors appear.

[0173] At step 562, processing unit 110 may analyze the geometry of the intersection. The analysis may be based on any combination of: (i) the number of lanes detected on either side of vehicle 200, (ii) detected markings on the road (such as arrows), and (iii) a description of the intersection extracted from map data (such as data from map database 160). Processing unit 110 may use information derived from execution of monocular analysis module 402 to perform the analysis. In addition, processing unit 110 may determine a correspondence between the traffic lights detected at step 560 and the lanes present near vehicle 200.

[0174] As the vehicle 200 approaches the intersection, in step 564, the processing unit 110 may update the confidence level associated with the analyzed intersection geometry and the detected traffic lights. For example, the comparison of the number of traffic lights estimated to be present at the intersection with the number of traffic lights actually present at the intersection may affect the confidence level. Therefore, based on the confidence level, the processing unit 110 may delegate control to the driver of the vehicle 200 to improve the safety situation. By executing steps 560-564, the processing unit 110 may identify the traffic lights that appear within the captured image set and analyze the intersection geometry information. Based on the identification and analysis, the processing unit 110 may cause one or more navigation responses in the vehicle 200, as described above in conjunction with Figure 5A described.

[0175] Figure 5E FIG2 is a flow chart illustrating an exemplary process 500E for inducing one or more navigation responses in a vehicle 200 based on a vehicle path consistent with the disclosed embodiments. At step 570, the processing unit 110 may construct an initial vehicle path associated with the vehicle 200. The vehicle path may be represented by a set of points represented by coordinates (x, z), and the distance d between two points in the set of points may be i May fall within the range of 1 to 5 meters. In one embodiment, the processing unit 110 may use two polynomials (e.g., a left road polynomial and a right road polynomial) to construct an initial vehicle path. The processing unit 110 may calculate the geometric midpoint between the two polynomials and offset each point included in the resulting vehicle path by a predetermined offset (e.g., a smart lane offset) - if a predetermined offset is provided (a zero value for the offset may correspond to driving in the middle of the lane). The offset may be in a direction perpendicular to the segment between any two points in the vehicle path. In another embodiment, the processing unit 110 may use a polynomial and an estimated lane width to offset each point of the vehicle path by half the estimated lane width plus a predetermined offset (e.g., a smart lane offset).

[0176] At step 572, the processing unit 110 may update the vehicle path constructed at step 570. The processing unit 110 may reconstruct the vehicle path constructed at step 570 using a higher resolution so that the distance d between two points in the set of points representing the vehicle path is k Smaller than the distance d described above i For example, the distance d kThe processing unit 110 may reconstruct the vehicle path using a parabolic spline algorithm that may produce a cumulative distance vector S corresponding to the total length of the vehicle path (ie, based on the set of points representing the vehicle path).

[0177] At step 574, the processing unit 110 may determine a look-ahead point (expressed as coordinates (x l , z l )). The processing unit 110 can extract a look-ahead point from the accumulated distance vector S, and the look-ahead point can be associated with a look-ahead distance and a look-ahead time. The look-ahead distance can have a lower limit in the range of 10 to 20 meters and can be calculated as the product of the speed of the vehicle 200 and the look-ahead time. For example, as the speed of the vehicle 200 decreases, the look-ahead distance may also decrease (e.g., until the look-ahead distance reaches the lower limit). The look-ahead time ranges from 0.5 seconds to 1.5 seconds and can be inversely proportional to the gain of one or more control loops (such as a heading error tracking control loop) associated with inducing a navigation response in the vehicle 200. For example, the gain of the heading error tracking control loop may depend on the bandwidth of the yaw rate loop, the steering actuator loop, the vehicle lateral dynamics, etc. Therefore, the higher the gain of the heading error tracking control loop, the shorter the look-ahead time.

[0178] At step 576, the processing unit 110 may determine the heading error and yaw rate command based on the look-ahead point determined at step 574. The processing unit 110 may calculate the heading error and yaw rate command by calculating the arc tangent (e.g., arctan(x l / z l )) to determine the heading error. Processing unit 110 may determine the yaw rate command as the product of the heading error and the high-level control gain. If the look-ahead distance is not at the lower limit, the high-level control gain may be equal to: (2 / look-ahead time). Otherwise, the high-level control gain may be equal to: (2*vehicle 200 speed / look-ahead distance).

[0179] Figure 5F is a flow chart illustrating an exemplary process 500F for determining whether a leading vehicle is changing lanes consistent with the disclosed embodiments. In step 580, the processing unit 110 may determine navigation information associated with the leading vehicle (e.g., the vehicle traveling in front of the vehicle 200). For example, the processing unit 110 may use the above-described method in conjunction with Figure 5A and Figure 5B The described techniques may be used to determine the position, velocity (e.g., direction and speed), and / or acceleration of the leading vehicle. The processing unit 110 may also use the above combined Figure 5EThe described techniques determine one or more road polynomials, look-ahead points (associated with vehicle 200 ), and / or snail trails (eg, a set of points describing a path taken by a leading vehicle).

[0180] At step 582, processing unit 110 may analyze the navigation information determined at step 580. In one embodiment, processing unit 110 may calculate the distance between the snail trail and the road polynomial (e.g., along the trail). If the change in this distance along the trail exceeds a predetermined threshold (e.g., 0.1 to 0.2 meters on a straight road, 0.3 to 0.4 meters on a moderately curved road, and 0.5 to 0.6 meters on a road with a sharp curve), processing unit 110 may determine that the leading vehicle may be changing lanes. In the event that multiple vehicles are detected traveling ahead of vehicle 200, processing unit 110 may compare the snail trails associated with each vehicle. Based on this comparison, processing unit 110 may determine that a vehicle whose snail trail does not match the snail trails of the other vehicles may be changing lanes. Processing unit 110 may additionally compare the curvature of the snail trail (associated with the leading vehicle) with the expected curvature of the road segment in which the leading vehicle is traveling. The expected curvature may be extracted from map data (e.g., data from map database 160), from a road polynomial, from snail trails of other vehicles, from prior knowledge about the road, etc. If the curvature of the snail trail differs from the expected curvature of the road segment by more than a predetermined threshold, processing unit 110 may determine that the leading vehicle may be changing lanes.

[0181] In another embodiment, the processing unit 110 may compare the instantaneous position of the leading vehicle with a look-ahead point (associated with the vehicle 200) over a specific time period (e.g., 0.5 seconds to 1.5 seconds). If the distance between the instantaneous position of the leading vehicle and the look-ahead point changes during the specific time period, and the cumulative sum of the changes exceeds a predetermined threshold (e.g., 0.3 meters to 0.4 meters on a straight road, 0.7 meters to 0.8 meters on a moderately curved road, and 1.3 meters to 1.7 meters on a road with a sharp curve), the processing unit 110 may determine that the leading vehicle may be changing lanes. In another embodiment, the processing unit 110 may analyze the geometry of the snail trail by comparing the lateral distance traveled along the snail trail with the expected curvature of the snail trail. The expected curvature radius can be calculated based on (δ z 2 +δ x 2 ) / 2 / (δ x ) to calculate, where δ x represents the lateral travel distance, and δz represents the longitudinal travel distance. If the difference between the lateral travel distance and the expected curvature exceeds a predetermined threshold (e.g., 500 meters to 700 meters), the processing unit 110 may determine that the leading vehicle may be changing lanes. In another embodiment, the processing unit 110 may analyze the position of the leading vehicle. If the position of the leading vehicle occludes the road polynomial (e.g., the leading vehicle is overlaid on the road polynomial), the processing unit 110 may determine that the leading vehicle may be changing lanes. If the position of the leading vehicle is such that another vehicle is detected in front of the leading vehicle and the snail trails of the two vehicles are not parallel, the processing unit 110 may determine that the (closer) leading vehicle may be changing lanes.

[0182] At step 584, the processing unit 110 may determine whether the leading vehicle 200 is changing lanes based on the analysis performed at step 582. For example, the processing unit 110 may make a determination based on a weighted average of the individual analyses performed at step 582. Under such an approach, for example, a determination by the processing unit 110 that the leading vehicle is likely changing lanes based on a particular type of analysis may be assigned a value of "1" (and "0" may be used to indicate a determination that the leading vehicle is unlikely to be changing lanes). Different analyses performed at step 582 may be assigned different weights, and the disclosed embodiments are not limited to any particular combination of analyses and weights. Furthermore, in some embodiments, the analysis may utilize a trained system (e.g., a machine learning or deep learning system) that can, for example, estimate a future path ahead of a vehicle at a current location based on images captured at the current location.

[0183] Figure 6 6 is a flow chart illustrating an exemplary process 600 for inducing one or more navigation responses based on stereo image analysis consistent with the disclosed embodiments. At step 610, processing unit 110 may receive a first and second plurality of images via data interface 128. For example, a camera included in image acquisition unit 120 (such as image capture device 122 having field of view 202 and image capture device 124 having field of view 204) may capture a first and second plurality of images of the area in front of vehicle 200 and transmit them to processing unit 100 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first and second plurality of images via two or more data interfaces. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0184] At step 620, the processing unit 110 may execute the stereo image analysis module 404 to perform stereo image analysis on the first and second plurality of images to create a 3D map of the road ahead of the vehicle and detect features within the image, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, etc. Stereo image analysis may be combined with the above Figure 5A - Figure 5D The steps described above are performed in a similar manner. For example, processing unit 110 can execute stereo image analysis module 404 to detect candidate objects (e.g., vehicles, pedestrians, road signs, traffic lights, road hazards, etc.) within the first and second pluralities of images, filter out a subset of candidate objects based on various criteria, and perform multi-frame analysis, construct measurements, and determine confidence levels for the remaining candidate objects. When performing the above steps, processing unit 110 can consider information from both the first and second pluralities of images, rather than just one set of images. For example, processing unit 110 can analyze differences in pixel-level data (or other subsets of data from the two captured image streams) for candidate objects that appear in both the first and second pluralities of images. As another example, processing unit 110 can estimate the position and / or velocity (e.g., relative to vehicle 200) of a candidate object by observing objects that appear in one of the multiple images but not in the other, or relative to other differences that may exist relative to objects that appear in both image streams. For example, the position, velocity, and / or acceleration relative to the vehicle 200 may be determined based on the trajectory, position, movement characteristics, etc. of features associated with objects appearing in one or both of the image streams.

[0185] At step 630, the processing unit 110 may execute the navigation response module 408 to perform the navigation response based on the analysis performed at step 620 and the above-mentioned combination. Figure 4 The described techniques may be used to cause one or more navigation responses in the vehicle 200. The navigation responses may include, for example, turns, lane deviations, acceleration changes, speed changes, braking, and the like. In some embodiments, the processing unit 110 may use data derived from execution of the velocity and acceleration module 406 to cause one or more navigation responses. Furthermore, multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof.

[0186] Figure 7FIG7 is a flow chart illustrating an exemplary process 700 for inducing one or more navigation responses based on analysis of three image sets consistent with the disclosed embodiments. At step 710, processing unit 110 may receive first, second, and third pluralities of images via data interface 128. For example, cameras included in image acquisition unit 120 (such as image capture device 122 having field of view 202, image capture device 124 having field of view 204, and image capture device 126 having field of view 206) may capture first, second, and third pluralities of images of areas in front of and / or to the sides of vehicle 200 and transmit them to processing unit 110 via a digital connection (e.g., USB, wireless, Bluetooth, etc.). In some embodiments, processing unit 110 may receive the first, second, and third pluralities of images via three or more data interfaces. For example, each of image capture devices 122, 124, 126 may have an associated data interface for transmitting data to processing unit 110. The disclosed embodiments are not limited to any particular data interface configuration or protocol.

[0187] At step 720, the processing unit 110 may analyze the first, second, and third sets of images to detect features within the images, such as lane markings, vehicles, pedestrians, road signs, highway exit ramps, traffic lights, road hazards, etc. The analysis may be similar to the above combined Figure 5A - Figure 5D and Figure 6 For example, the processing unit 110 may perform monocular image analysis on each of the first, second, and third plurality of images (e.g., by executing the monocular image analysis module 402 and based on the above combined Figures 5A-5D Alternatively, the processing unit 110 may perform stereoscopic image analysis on the first and second pluralities of images, the second and third pluralities of images, and / or the first and third pluralities of images (e.g., by executing the stereoscopic image analysis module 404 and based on the above in combination with Figure 6 4 and 5. The processing unit 110 may perform a combination of monocular image analysis and stereo image analysis. For example, the processing unit 110 may perform monocular image analysis on the first plurality of images (e.g., via execution of the monocular image analysis module 402) and perform stereo image analysis on the second and third pluralities of images (e.g., via execution of the stereo image analysis module 404). The configuration of the image capture devices 122, 124, and 126, including their respective positions and fields of view 202, 204, and 206, may affect the type of analysis performed on the first, second, and third pluralities of images. The disclosed embodiments are not limited to a particular configuration of the image capture devices 122, 124, and 126, or the type of analysis performed on the first, second, and third pluralities of images.

[0188] In some embodiments, processing unit 110 may perform testing on system 100 based on the images captured and analyzed in steps 710 and 720. Such testing may provide an indication of the overall performance of system 100 for certain configurations of image capture devices 122, 124, and 126. For example, processing unit 110 may determine a ratio of a "false hit rate" (e.g., instances where system 100 incorrectly determines the presence of a vehicle or pedestrian) to a "miss rate."

[0189] At step 730, the processing unit 110 may cause one or more navigation responses in the vehicle 200 based on information derived from both of the first, second, and third pluralities of images. The selection of both of the first, second, and third pluralities of images may depend on various factors, such as, for example, the number, type, and size of objects detected in each of the plurality of images. The processing unit 110 may also make the selection based on image quality and resolution, the effective field of view reflected in the image, the number of captured frames, the degree to which one or more objects of interest actually appear in the frames (e.g., the percentage of frames in which the object appears, the proportion of each such frame in which the object appears, etc.), and the like.

[0190] In some embodiments, processing unit 110 may select information derived from both of the first, second, and third pluralities of images by determining how consistent the information derived from one image source is with information derived from other image sources. For example, processing unit 110 may combine processed information derived from each of image capture devices 122, 124, and 126 (whether through monocular analysis, stereo analysis, or any combination thereof) and determine visual indicators (e.g., lane markings, detected vehicles and their positions and / or paths, detected traffic lights, etc.) that are consistent across the images captured by each of image capture devices 122, 124, and 126. Processing unit 110 may also exclude information that is inconsistent across the captured images (e.g., vehicles changing lanes, lane models indicating vehicles that are too close to vehicle 200, etc.). Thus, processing unit 110 may select information derived from both of the first, second, and third pluralities of images based on the determination of consistent and inconsistent information.

[0191] The navigation response may include, for example, a turn, a lane deviation, an acceleration change, etc. The processing unit 110 may analyze the navigation response in step 720 and the above-mentioned combination of the above-mentioned steps. Figure 4The processing unit 110 may also use data derived from execution of the velocity and acceleration module 406 to cause one or more navigation responses. In some embodiments, the processing unit 110 may cause one or more navigation responses based on the relative position, relative velocity, and / or relative acceleration between the vehicle 200 and an object detected within any of the first, second, and third pluralities of images. The multiple navigation responses may occur simultaneously, sequentially, or in any combination thereof.

[0192] Reinforcement learning and trained navigation systems

[0193] The following sections discuss autonomous driving and systems and methods for enabling autonomous control of a vehicle, whether that control is fully autonomous (a self-driving vehicle) or partially autonomous (e.g., one or more driver assistance systems or functions). Figure 8 As shown, the autonomous driving task can be divided into three main modules, including a sensing module 801, a driving strategy module 803, and a control module 805. In some embodiments, modules 801, 803, and 805 can be stored in the memory unit 140 and / or the memory unit 150 of the system 100, or modules 801, 803, and 805 (or portions thereof) can be stored remotely from the system 100 (e.g., stored in a server accessible to the system 100 via, for example, the wireless transceiver 172). In addition, any module disclosed herein (e.g., modules 801, 803, and 805) can implement techniques associated with a trained system (such as a neural network or a deep neural network) or an untrained system.

[0194] The sensing module 801, which can be implemented using the processing unit 110, can handle various tasks related to sensing the navigational state of the host vehicle's environment. Such tasks may rely on input from various sensors and sensing systems associated with the host vehicle. These inputs may include images or image streams from one or more recording cameras, GPS location information, accelerometer output, user feedback or input to one or more user interface devices, radio detection and ranging, light detection and ranging, and the like. Sensing information from cameras and / or any other available sensors can be collected, analyzed, and combined with map information to form a "sensed state," which describes information extracted from the scene in the host vehicle's environment. The sensed state can include sensory information related to target vehicles, lane markings, pedestrians, traffic lights, road geometry, lane shape, obstacles, distances to other objects / vehicles, relative speeds, relative accelerations, and any other potential sensory information. Supervised machine learning can be implemented to generate a sensed state output based on the sensory data provided to the sensing module 801. The output of the sensing module can represent the sensed navigational "state" of the host vehicle, which can be passed to the driving strategy module 803.

[0195] Although the sensing state can be formed based on the image data received from one or more cameras or image sensors associated with the main vehicle, any suitable sensor or sensor combination can be used to form the sensing state for use in navigation. In some embodiments, the sensing state can be formed without relying on the captured image data. In fact, any navigation principle in the navigation principles described herein can be applicable to the sensing state formed based on the captured image data and the sensing state formed using other non-image-based sensors. The sensing state can also be determined via a source outside the main vehicle. For example, the sensing state can be formed entirely or in part based on information received from a source away from the main vehicle (e.g., sensor information, processed state information, etc. shared from other vehicles, shared from a central server, or from any other information source relevant to the navigation state of the main vehicle).

[0196] The driving strategy module 803, which is discussed in more detail below and can be implemented using the processing unit 110, can implement a desired driving strategy to determine one or more navigation actions to be taken by the host vehicle in response to the sensed navigation state. If there are no other agents (e.g., target vehicles or pedestrians) in the host vehicle's environment, the sensed state input to the driving strategy module 803 can be handled in a relatively simple manner. When the sensed state requires negotiation with one or more other agents, the task becomes more complex. Techniques for generating the output of the driving strategy module 803 can include reinforcement learning (discussed in more detail below). The output of the driving strategy module 803 can include at least one navigation action for the host vehicle and can include a desired acceleration (which can be converted into an updated speed of the host vehicle), a desired yaw rate of the host vehicle, a desired trajectory, and other potential desired navigation actions.

[0197] Based on the output from the driving strategy module 803, the control module 805 can form control instructions for one or more actuators or controlled devices associated with the host vehicle, which can also be implemented using the processing unit 110. Such actuators and devices can include an accelerator, one or more steering control devices, brakes, signal transmitters, displays, or any other actuators or devices that can be controlled as part of navigation operations associated with the host vehicle. Various aspects of control theory can be used to generate the output of the control module 805. The control module 805 can be responsible for forming and outputting instructions to the controllable components of the host vehicle in order to achieve the desired navigation goals or requirements of the driving strategy module 803.

[0198] Returning to the driving policy module 803, in some embodiments, a trained system trained via reinforcement learning can be used to implement the driving policy module 803. In other embodiments, the driving policy module 803 can be implemented without machine learning methods by "manually" solving various scenarios that may arise during autonomous navigation using a specified algorithm. However, while feasible, this approach can result in a driving policy that is overly simplistic and may lack the flexibility of a trained system based on machine learning. For example, a trained system may be better equipped to handle complex navigation states and may better determine whether a taxi is stopping or stopping to pick up or drop off a passenger; determine whether a pedestrian intends to cross the street in front of the host vehicle; balance the unexpected behavior of other drivers with defensive behavior; negotiate dense traffic involving target vehicles and / or pedestrians; decide when to suspend certain navigation rules or augment others; anticipate unperceived but anticipated conditions (e.g., whether a pedestrian will emerge from behind a car or obstacle); etc. A trained system based on reinforcement learning may also be better equipped to solve continuous and high-dimensional state spaces and continuous action spaces.

[0199] Training a system using reinforcement learning may involve learning a driving policy to map from perception states to navigation actions. A driving policy is a function π: S→A, where S is a set of states and is the action space (e.g., desired velocity, acceleration, yaw command, etc.). The state space is S = S s x S p , where S s is the sensing state and S p Is additional information about the state saved by the strategy. Working in discrete time intervals, at time t, the current state s can be observed t ∈S, and a policy can be applied to obtain the desired action a t =π(s t ).

[0200] The system can be trained by being exposed to various navigation states, causing the system to apply policies and provide rewards (based on a reward function designed to reward ideal navigation behavior). Based on the reward feedback, when producing navigation actions that meet the expectations, the system can "learn" the policy and become trained. For example, the learning system can observe the current state s t ∈S and based on the policy Decide on action a t ∈A. Based on the action decided (and the implementation of the action), the environment will move to the next state s t ∈ S for the learning system to observe. For each action formed in response to the observed state, the feedback to the learning system is the reward signal r1, r2, ...

[0201] The goal of reinforcement learning (RL) is to discover a policy π. It is usually assumed that at time t, there is a measurement in state s t and take action a t The instantaneous quality reward function r t However, at time t, action a is taken t affects the environment and thereby the value of future states. Therefore, when deciding which action to take, not only the current reward but also the future reward should be considered. In some cases, when the system determines that a higher reward may be achieved in the future if a lower reward option is taken now, the system should take a specific action even if it is associated with a lower reward than another available option. To formalize this, observe that the policy π and the initial state s lead to The distribution on which the vector (r1,…,r T ) is the probability that the agent starts in state s0 = s and observes rewards r1,…,r from then on following policy π T The probability of the initial state s can be defined as:

[0202]

[0203] Instead of restricting the time horizon to T, future returns may be discounted to define for some fixed γ∈(0,1):

[0204]

[0205] In any case, the optimal strategy is the solution

[0206]

[0207] where the expected value exceeds the initial state s.

[0208] There are several possible approaches to training a driving policy system. For example, one could use imitation methods (e.g., behavioral cloning), where the system learns from state / action pairs, where an action is the action a good agent (e.g., a human) would choose in response to a particular observed state. Suppose that a human driver is observed. From this observation, many forms (s t ,a t )(where s t is a state, and a t Examples of actions of human drivers) can be obtained, observed, and used as the basis for training a driving policy system. For example, supervised learning can be used to learn a policy π such that π(s t )≈a tThere are many potential advantages to this approach. First, there is no need to define a reward function. Second, learning is supervised and occurs offline (no agent is required during the learning process). A disadvantage of this approach is that different human drivers, and even the same human driver, are nondeterministic in their choice of strategy. Therefore, learning is not very reliable for ||π(s t )-a t || Very small functions are usually not feasible. Moreover, even small errors can accumulate over time to produce large errors.

[0209] Another possible technique is policy-based learning. Here, the policy can be expressed in parameterized form and optimized directly using a suitable optimization technique (e.g., stochastic gradient descent). There are of course many approaches to this problem. One advantage of this approach is that it addresses the problem directly and therefore often leads to good practical results. A potential disadvantage is that it often requires "on-policy" training, i.e., learning π is an iterative process, and at iteration j we have a non-perfect policy π j , and in order to construct the next strategy π j , we must base j Interact with the environment while performing actions.

[0210] The system can also be trained using value-based learning (learning Q or V functions). It is assumed that a good approximation to the optimal value function V* can be learned. The optimal policy can be constructed (for example, by relying on the Bellman equation). Some versions of value-based learning can be implemented offline (called "off-policy" training). Some disadvantages of value-based methods may arise from their strong reliance on the Markov assumption and the required approximation of complex functions (approximating the value function may be more difficult than directly approximating the policy).

[0211] Another technique may include model-based learning and planning (learning the probabilities of state transitions and solving the optimization problem of finding the optimal V). A combination of these techniques can also be used to train a learning system. In this approach, the dynamics of the process can be learned, i.e., using (s t ,a t ) and generate the next state s t+1 Once this function is learned, we can solve an optimization problem to find the policy π with the optimal value. This is called "planning". One advantage of this approach is that the learning part is supervised and can be obtained by observing the triples (s t ,a t ,s t+1) rather than being applied offline. Similar to the “imitation” approach, a drawback of this approach may be that small errors in the learning process may accumulate and produce poorly performing policies.

[0212] Another approach to training the driving policy module 803 can involve decomposing the driving policy function into semantically meaningful components. This allows for manual implementation of portions of the policy, which ensures policy safety, and for implementation of other portions of the policy using reinforcement learning techniques, which enable adaptability to a variety of scenarios, human-like balancing of defensive / aggressive behavior, and human-like negotiation with other drivers. From a technical perspective, reinforcement learning methods can combine several approaches and provide a tractable training procedure, most of which can be performed using recorded data or a home-built simulator.

[0213] In some embodiments, the training of the driving policy module 803 can rely on an "option" mechanism. To illustrate, consider a simple scenario of a driving policy for a two-lane highway. In a direct RL approach, the state is mapped to The policy π is where the first component of π(s) is the desired acceleration command and the second component of π(s) is the yaw rate. In the modified approach, the following policy can be constructed:

[0214] Automatic Cruise Control (ACC) strategy ACC : S→A: This strategy always outputs the yaw rate and only changes the speed, for smooth and accident-free driving.

[0215] ACC+Left Strategy o L :S→A: The longitudinal commands for this strategy are the same as those for ACC. The yaw rate is a direct implementation of centering the vehicle toward the middle of the left lane while ensuring safe lateral movement (e.g., do not move left if there is a car on the left).

[0216] ACC+Right Strategy R :S→A:with o L Same, but the vehicle may center towards the center of the right lane.

[0217] These strategies can be called "options". Based on these "options", we can learn the strategy π of choosing options o : S→O, where O is a set of available options. In one case, O={o ACC , o L , o R}. Option selector strategy π o By setting To define the actual policy π: S→A.

[0218] In practice, the policy function can be decomposed into an option graph 901, such as Figure 9 Another example option chart 1000 is shown as Figure 10 As shown. The option graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There is a special node called the root node 903 of the graph. This node has no incoming nodes. The decision process traverses the graph from the root node until it reaches a "leaf" node, which is a node with no outgoing decision lines. Figure 9 As shown, for example, leaf nodes may include nodes 905, 907, and 909. Upon encountering a leaf node, the driving strategy module 803 may output acceleration and steering commands associated with the desired navigation action associated with the leaf node.

[0219] Internal nodes such as nodes 911, 913, and 915, for example, may cause the implementation of a strategy for selecting a subset of their available options. The available subset of an internal node includes all nodes that are associated with the specific internal node via a decision line. For example, in Figure 9 The internal node 913 designated as “Merge” includes three child nodes 909 , 915 and 917 (“Stop”, “Overtake Right” and “Overtake Left”, respectively), each of which is connected to the node 913 via a decision line.

[0220] By enabling nodes to adjust their position in the hierarchy of the options graph, flexibility in the decision making system can be achieved. For example, any of the nodes can be allowed to declare itself as "critical". Each node can implement a function "iscritical" that will output "true" if the node is in a critical part of its policy implementation. For example, a node responsible for overtaking can declare itself to be critical during a maneuver. This can impose a constraint on the available subset of a node u, which can include all nodes v that are children of node u and for which there is a path from v to a leaf node that passes through all the nodes designated as critical. Such an approach can, on the one hand, allow the declaration of the desired path on the graph at each time step, while on the other hand, maintaining the stability of the policy, especially when the critical part of the policy is being implemented.

[0221] By defining an option graph, the problem of learning a driving policy π: S→A can be decomposed into the problem of defining a policy for each node of the graph, where the policy at the internal node should be selected from the available child nodes. For some nodes, the corresponding policy can be implemented manually (e.g., by specifying a set of actions in response to the observed state through an if-then type algorithm), while for other nodes, a trained system built through reinforcement learning can be used to implement the policy. The choice between manual or trained / learned approaches may depend on the safety aspects associated with the task and on its relative simplicity. The option graph can be implemented in a way that some of the nodes are easy to implement while other nodes can rely on trained models. Such an approach can ensure safe operation of the system.

[0222] The following discussion provides some information on Figure 9 The following further details the role of the option chart within the driving strategy module 803. As discussed above, the input to the driving strategy module is a "sensor state," which summarizes, for example, a map of the environment obtained from available sensors. The output of the driving strategy module 803 is a set of expectations (optionally, along with a set of hard constraints) that define the trajectory that is the solution to the optimization problem.

[0223] As described above, an option graph represents a hierarchical set of decisions organized as a DAG. There is a special node called the "root" of the graph. The root node is the only node with no incoming edges (e.g., decision lines). The decision process starts at the root node and traverses through the graph until it reaches a "leaf" node, i.e., a node with no outgoing edges. Each internal node should implement a strategy for selecting children from its available children. Each leaf node should implement a strategy that defines a set of expectations (e.g., a set of navigation goals for the main vehicle) based on the entire path from the root to the leaf. The set of expectations, together with a set of hard constraints defined directly based on the sensed states, establishes an optimization problem, the solution of which is the trajectory of the vehicle. The hard constraints can be exploited to further improve the safety of the system, and expectations can be used to provide the system with driving comfort and human-like driving behavior. The trajectory provided as the solution to the optimization problem then defines the commands that should be provided to the steering, braking, and / or engine actuators to complete the trajectory.

[0224] Back to Figure 9, option diagram 901 represents an option diagram for a two-lane highway, including merging lanes (meaning that at some point, a third lane merges into the right or left lane of the highway). Root node 903 first determines whether the host vehicle is in a flat road scenario or approaching a merging scenario. This is an example of a decision that can be made based on the sensed state. Flat road node 911 includes three child nodes: stay node 909, left pass node 917, and right pass node 915. Staying refers to a situation where the host vehicle wishes to remain in the same lane. The stay node is a leaf node (no outgoing edges / lines). Therefore, the stay node defines a set of expectations. The first expectation it defines may include a desired lateral position—for example, as close to the center of the current lane as possible. There may also be a desire for smooth navigation (for example, within a predefined or allowed maximum acceleration). The stay node may also define how the host vehicle reacts to other vehicles. For example, the stay node may examine the sensed target vehicles and assign a semantic meaning to each detected target vehicle, which can be converted into a component of the trajectory.

[0225] Various semantics can be assigned to target vehicles in the host vehicle's environment. For example, in some embodiments, the semantics can include any of the following: 1) Not relevant: indicates that the sensed vehicle in the scene is currently irrelevant; 2) Next lane: indicates that the sensed vehicle is in an adjacent lane and should maintain an appropriate offset relative to the vehicle (the exact offset can be calculated in an optimization problem that constructs a trajectory given a desired and hard constraint, and the exact offset is potentially vehicle-dependent - the stay leaf of the option graph sets the semantic type of the target vehicle, which defines the desired behavior relative to the target vehicle); 3) Yield: The host vehicle will attempt to yield to the sensed target vehicle, for example by reducing speed (especially when the host vehicle determines that the target vehicle may be about to cut into the host vehicle's lane); 4) Take: The host vehicle will attempt to take the right of way, for example by increasing speed; 5) Follow: The host vehicle wishes to follow the target vehicle and maintain a smooth driving; 6) Pass Left / Pass Right: This means that the host vehicle wishes to initiate a lane change to the left or right lane. The left overtaking node 917 and the right overtaking node 915 are internal nodes for which expectations have not yet been defined.

[0226] The next node in option graph 901 is select gap node 919. This node may be responsible for selecting a gap between two target vehicles in a particular target lane that the host vehicle desires to enter. By selecting a node of the form IDj for a certain value of j, the host vehicle reaches the desired leaf of the trajectory optimization problem—for example, the host vehicle desires to maneuver to reach the selected gap. Such a maneuver may involve first accelerating / braking in the current lane and then steering toward the target lane at the appropriate time to enter the selected gap. If select gap node 919 cannot find a suitable gap, it moves to abort node 921, which defines the desire to return to the center of the current lane and cancel the overtaking.

[0227] Returning to the merge node 913, as the host vehicle approaches the merge, it has several options that may depend on the specific situation. For example, Figure 11A As shown, the host vehicle 1105 is traveling along a two-lane road, with no other target vehicles detected in either the main lane or the merging lane 1111 of the two-lane road. In this case, upon reaching the merging node 913, the driving strategy module 803 may select the stay node 909. In other words, it may be desirable to remain in its current lane without detecting a target vehicle merging onto the road.

[0228] exist Figure 11B In Figure 1, the situation is slightly different. Here, the host vehicle 1105 senses one or more target vehicles 1107 entering the main lane 1112 from the merging lane 1111. In this case, once the driving strategy module 803 encounters the merging node 913, it may choose to initiate a left passing maneuver to avoid the merging situation.

[0229] exist Figure 11C In Figure 1, host vehicle 1105 encounters one or more target vehicles 1107 entering host lane 1112 from merging lane 1111. Host vehicle 1105 also detects target vehicle 1109 traveling in a lane adjacent to the host vehicle's lane. Host vehicle 1105 also detects one or more target vehicles 1110 traveling in the same lane as host vehicle 1105. In this case, driving strategy module 803 may decide to adjust the speed of host vehicle 1105 to yield to target vehicle 1107 and advance ahead of target vehicle 1115. This can be achieved, for example, by proceeding to select gap node 919, which in turn selects the gap between ID0 (vehicle 1107) and ID1 (vehicle 1115) as the appropriate merging gap. In this case, the appropriate gap for the merging situation defines the objective of the trajectory planner optimization problem.

[0230] As discussed above, nodes of the option graph can declare themselves as "critical", which ensures that the selected option passes through critical nodes. Formally, each node can implement the function IsCritical (is critical). After performing a forward pass from root to leaf on the option graph and solving the trajectory planner's optimization problem, a backward pass from leaf back to root can be performed. Along this backward pass, the IsCritical function of all nodes in the pass can be called, and a list of all critical nodes can be saved. In the forward path corresponding to the next time frame, the driving strategy module 803 may be required to select a path from the root node to the leaf that passes through all critical nodes.

[0231] Figures 11A-11C This can be used to illustrate the potential benefits of this approach. For example, when an overtaking maneuver is initiated and the driving strategy module 803 reaches the leaf corresponding to IDk, it is not desirable to select, for example, a stop node 909 when the host vehicle is in the middle of an overtaking maneuver. To avoid this jump, the IDj node can designate itself as critical. During the maneuver, the success of the trajectory planner can be monitored, and if the overtaking maneuver progresses as expected, the function IsCritical will return a "true" value. This approach can ensure that the overtaking maneuver will continue in the next time frame (rather than jumping to another potentially inconsistent maneuver before completing the originally selected maneuver). On the other hand, if the monitoring of the maneuver indicates that the selected maneuver is not progressing as expected, or if the maneuver has become unnecessary or impossible, the function IsCritical can return a "false" value. This can allow the selection gap node to select a different gap in the next time frame, or to completely abort the overtaking maneuver. This approach can, on the one hand, allow the declaration of the desired path on the option graph at each time step, and on the other hand, can help improve the stability of the strategy while executing in the critical part.

[0232] Hard constraints may be distinguished from navigation expectations, which will be discussed in more detail below. For example, hard constraints can ensure safe driving by applying an added layer of filtering to planned navigation operations. The hard constraints involved can be determined from the sensed state, and these hard constraints involved can be programmed and defined manually rather than by using a trained system based on reinforcement learning. However, in some embodiments, the trained system can learn the applicable hard constraints to apply and follow. Such an approach can promote the driving policy module 803 to reach a selected action that already meets the applicable hard constraints, which can reduce or eliminate the selected action that may need to be modified later to meet the applicable hard constraints. However, as a redundant safety measure, hard constraints can be applied to the output of the driving policy module 803 even if the driving policy module 803 has been trained to take into account predetermined hard constraints.

[0233] There are many examples of potential hard constraints. For example, a hard constraint may be defined in conjunction with a guardrail at the edge of the road. Under no circumstances is the host vehicle allowed to pass the guardrail. Such a rule induces a hard lateral constraint on the trajectory of the host vehicle. Another example of a hard constraint may include a bump in the road (e.g., a speed bump), which may induce a hard constraint on driving speed before and while crossing the bump. Hard constraints may be considered safety critical and, therefore, may be defined manually rather than relying solely on a trained system to learn the constraints during training.

[0234] In contrast to hard constraints, a desired goal may be to achieve or attain comfortable driving. As described above, an example of a desired goal may include a goal to position the host vehicle at a lateral position within the lane corresponding to the host vehicle's lane center. Another desired goal may include the ID of a gap to fit into. Note that the host vehicle is not required to be exactly in the lane center, but instead, a desire to be as close to the lane center as possible may ensure that the host vehicle tends to migrate to the lane center even if it deviates from the lane center. The desired goal may not be safety critical. In some embodiments, the desired goal may require consultation with other drivers and pedestrians. One method for constructing the desired goal may rely on an option graph, and the policies implemented in at least some nodes of the graph may be based on reinforcement learning.

[0235] For nodes of option graph 901 or option graph 1000 implemented as nodes based on learning training, the training process may include decomposing the problem into a supervised learning phase and a reinforcement learning phase. In the supervised learning phase, it is possible to learn t ,a t )arrive A differentiable mapping such that This may be similar to "model-based" reinforcement learning. However, in the forward loop of the network, The actual value s t+1 Instead, the problem of error accumulation is eliminated. The role of is to propagate messages from future actions back to past actions. In this sense, the algorithm may be a combination of "model-based" reinforcement learning and "policy-based learning"

[0236] An important element that can be provided in some scenarios is a differentiable path from future losses / rewards back to the decision on the action. With the option graph structure, the implementation of options involving safety constraints is usually not differentiable. To overcome this problem, the selection of children in the learned policy node can be random. That is, the node can output a probability vector p that assigns the probability of using when selecting each child of the children of a particular node. Suppose a node has k children and makes a (1) ,...,a (k)is the action of each path from child to leaf. The predicted action is thus This may result in a distinguishable path from action to p. In practice, for i~p, action a can be chosen as a (i) , a and The difference between them can be called additive noise.

[0237] For a given s t ,a t In the case of Training, supervised learning can be used with real data. To train a node's policy, a simulator can be used. Afterwards, fine-tuning of the policy can be done using real data. Two concepts can make the simulation more realistic. First, using imitation, an initial policy can be constructed using a "behavioral cloning" paradigm using large real-world datasets. In some cases, the resulting agent may be suitable. In other cases, the resulting agent at least forms a very good initial policy for other agents on the road. Second, using self-play, training can be enhanced using our own policy. For example, given an initial realization of other agents (cars / pedestrians) that may be encountered, the policy can be trained based on a simulator. Some of the other agents may be replaced by the new policy, and the process can be repeated. Therefore, the policy can continue to improve as it should respond to a wider variety of other agents with different levels of complexity.

[0238] Furthermore, in some embodiments, the system can implement a multi-agent approach. For example, the system can consider data from various sources and / or images captured from multiple angles. Furthermore, some disclosed embodiments can provide energy economy because the anticipation of events that do not directly involve the host vehicle but may have an impact on the host vehicle can be considered, or even the anticipation of events that may lead to unpredictable situations involving other vehicles can also be a consideration (e.g., radio detection and ranging may "see through" the lead vehicle and anticipate unavoidable events, or even a high probability of an event that will affect the host vehicle).

[0239] Trained system with imposed navigation constraints

[0240] In the context of autonomous driving, a key concern is ensuring that the learned policies of a trained navigation network are safe. In some embodiments, constraints can be used to train the driving policy system so that the actions selected by the trained system can already account for applicable safety constraints. Additionally, in some embodiments, an additional layer of safety can be provided by passing the selected actions of the trained system through one or more hard constraints implied by the specific sensing scenario in the host vehicle's environment. This approach can ensure that the actions taken by the host vehicle are limited to those actions that are confirmed to satisfy the applicable safety constraints.

[0241] At its core, the navigation system may include a learning algorithm based on a policy function that maps observed states to one or more desired actions. In some implementations, the learning algorithm is a deep learning algorithm. The desired action may include at least one action that is expected to maximize the expected reward of the vehicle. While in some cases, the actual action taken by the vehicle may correspond to one of the desired actions, in other cases, the actual action taken may be determined based on the observed state, one or more desired actions, and non-learned hard constraints (e.g., safety constraints) imposed on the learning navigation engine. These constraints may include a no-driving zone around various types of detected objects (e.g., target vehicles, pedestrians, stationary objects on the roadside or in the road, moving objects on the roadside or in the road, guardrails, etc.). In some cases, the size of the zone may vary based on the detected motion (e.g., speed and / or direction) of the detected object. Other constraints may include a maximum travel speed when passing within the influence zone of a pedestrian, a maximum deceleration (taking into account the distance of the target vehicle behind the host vehicle), mandatory stops at sensed crosswalks or railroad crossings, etc.

[0242] Hard constraints used in conjunction with a system trained via machine learning may provide a degree of safety in autonomous driving that may exceed the degree of safety available based solely on the output of the trained system. For example, a machine learning system may be trained using a desired set of constraints as a training guide, and the trained system may therefore select actions in response to sensed navigation states that take into account and adhere to the limitations of applicable navigation constraints. However, the trained system still has some flexibility in selecting navigation actions, and therefore there may be at least some situations in which the actions selected by the trained system may not strictly adhere to the relevant navigation constraints. Therefore, in order to require that the selected actions strictly adhere to the relevant navigation constraints, the output of the trained system may be combined with, compared with, filtered by, adjusted by, modified by, etc., using non-machine learning components outside of the learning / training framework that ensure the strict application of the relevant navigation constraints.

[0243] The following discussion provides additional details about the trained system and the potential benefits (especially from a security perspective) that may be obtained by combining the trained system with algorithmic components outside the training / learning framework. As discussed, the reinforcement learning objective of the policy can be optimized by stochastic gradient boosting. The objective (e.g., expected reward) can be defined as

[0244] Goals involving expectations can be used in machine learning scenarios. However, such goals that are not subject to navigation constraints may not return actions that are strictly bound by these constraints. For example, consider a reward function for which, in terms of trajectories that represent rare “turning” events (e.g., accidents) to avoid, For the rest of the trajectories, One goal of a learning system might be to learn to perform an overtaking maneuver. Typically, in an accident-free trajectory, rewards successful, smooth overtakes and penalizes staying in the lane without completing the overtake - hence the range [-1, 1]. represents an accident, then the reward -r should provide a sufficiently high penalty to discourage such accidents. The question is what value of r should be to ensure accident-free driving.

[0245] Observe the accident The effect of is the additive term -pr, where p is the probability mass of trajectories with accident events. If this term is negligible, i.e., p < < 1 / r, then the learning system may be more inclined to execute a strategy that leads to accidents (or adopt a generally reckless driving strategy) in order to successfully complete overtaking maneuvers more often than a more defensive strategy that leaves some overtaking maneuvers unsuccessful. In other words, if the probability of an accident is at most p, then r must be set so that r>> 1 / p. Make p very small (e.g., about p = 10 -9 ) may be in line with expectations. Therefore, r should be large. In the policy gradient, it can be estimated The following lemma shows that the random variable The variance of pr 2 As r>>1 / p grows, the variance is larger than r. Therefore, estimating the target may be difficult, and estimating its gradient may be even more difficult.

[0246] Lemma: Let π o Let p and r be scalars, so that with probability p, we get And with probability 1-p, we get but,

[0247]

[0248] The last approximation holds for the case r ≥ 1 / p.

[0249] This discussion shows that the form The objection may not make the function safe without causing variance problems. Baseline subtraction for variance reduction may not provide an adequate remedy for this problem, as the problem will The high variance of is transferred to the equally high variance of the baseline constant, whose estimate will also be affected by numerical instabilities. Moreover, if the probability of an accident is p, then on average at least 1 / p sequences should be sampled before an accident event is obtained. This means that for the The solution to this problem can be found in the architectural design described in this paper, rather than through numerical tuning techniques. The approach here is based on the notion that hard constraints should be injected outside the learning framework. In other words, the policy function can be decomposed into a learnable part and a non-learnable part. Formally, the policy function can be constructed as in maps the (unknowable) state space to a set of expectations (e.g., desired navigation goals, etc.), and π (T) Maps expectations to trajectories (which determine how the car should move over short distances). Function Responsible for driving comfort and responsible for making strategic decisions such as which other vehicles should be overtaken or given way, and the desired position of the host vehicle in its lane. The mapping from the sensed navigation state to the desired state is the strategy The strategy You can learn from experience by maximizing expected returns. The resulting expectation can be converted into a cost function on the driving trajectory. Function π (T) (not the learned function) can be achieved by finding a trajectory that minimizes the cost subject to hard constraints on the safety of the function. This decomposition can ensure the safety of the function while providing comfortable driving.

[0250] like Figure 11D The depicted double merge navigation scenario provides an example to further illustrate these concepts. In a double merge, vehicles approach the merging area 1130 from both the left and right sides. And, from each side, a vehicle, such as vehicle 1133 or vehicle 1135, can decide whether to merge into the lane on the other side of the merging area 1130. Successfully executing a double merge in busy traffic can require considerable negotiation skill and experience, and can be difficult to perform with a heuristic or brute force approach by enumerating all possible trajectories that all agents in the scenario could take. In this double merge example, a set of expectations applicable to the double merge maneuver can be defined. Can be the Cartesian product of the following sets: where [0,v max ] is the desired target speed of the host vehicle, L = {1, 1.5, 2, 2.5, 3, 3.5, 4} is the desired lateral position in lane units, where integers represent lane centers and decimals represent lane boundaries, and {g, t, o} is the classification label assigned to each of the n other vehicles. An other vehicle may be assigned a "g" if the host vehicle will yield to it, a "t" if the host vehicle will take the path relative to the other vehicle, or an "o" if the host vehicle will maintain an offset distance relative to the other vehicle.

[0251] The following is the expected set How to convert it into a description of the cost function on the driving trajectory. The driving trajectory can be represented as (x1, y1), ..., (x k ,y k ), where (x i ,y i) is the (lateral and longitudinal) position of the primary vehicle (in the egocentric cell) at time τ·i. In some experiments, τ = 0.1 seconds and k = 10. Of course, other values ​​can also be chosen. The cost assigned to the trajectory may include a weighted sum of the costs assigned to the desired velocity, lateral position, and labels assigned to the other n vehicles.

[0252] Given the desired velocity v∈[0,v max ]. Then the cost of the trajectory associated with the speed is

[0253]

[0254] Given a desired lateral position l∈L, the cost associated with the desired lateral position is

[0255]

[0256] where dist(x,y,l) is the distance from point (x,y) to lane position l. Regarding the costs incurred by other vehicles, for any other vehicle, (x′1, y′1), ..., (x′ k , y′ k ) can represent other vehicles in the egocentric unit of the main vehicle, and i can be the earliest point, for which there exists j such that (x i ,y i ) and (x' j ,y' j ) is small. If no such point exists, then i can be set to i=∞. If the other car is classified as "give way", then τj>τi+0.5 may be desirable, meaning that the host vehicle will arrive at the same point at least 0.5 seconds after the other vehicle reaches the same point of trajectory intersection. A possible formula to convert the above constraint into a cost is [τ(ij)+0.5] + .

[0257] Likewise, if the other car is classified as “taking the road”, then τj>τi+0.5 may be desirable, which can be converted to a cost [τ(ij)+0.5] + If the other car is classified as "drift", then i = ∞ may be desirable, i = ∞ means that the trajectory of the host vehicle and the trajectory of the drift car do not intersect. This situation can be converted into a cost by penalizing the distance between the trajectories.

[0258] Assigning weights to each of these costs may provide a single objective function π for the trajectory planner (T). Costs that encourage smooth driving may be added to the objective. Also, hard constraints may be added to the objective to ensure functional safety of the trajectory. For example, (x i ,y i ) may be prohibited from leaving the road, and if |i–j| is small, then for any trajectory point (x' j ,y' j ), (x i ,y i ) may be prohibited from approaching (x' j ,y' j ).

[0259] In short, strategy π θ It can be decomposed into a mapping from unknowable states to desired sets, and a mapping from desired sets to actual trajectories. The latter mapping is not based on learning and can be achieved by solving an optimization problem whose cost depends on the desired set and whose hard constraints guarantee the functional safety of the policy.

[0260] The following discussion describes the mapping from unknowable states to desired sets. As described above, in order to comply with functional safety, a system that relies solely on reinforcement learning may suffer from the reward This result can be avoided by using policy gradients to iteratively decompose the problem into a mapping from an (unknowable) state space to a set of desired trajectories, followed by a mapping of the actual trajectories of the system without involving machine learning training.

[0261] For various reasons, decision making can be further broken down into semantically meaningful components. For example, The size of may be large or even continuous. Figure 11D In the double lane merging scenario described, Additionally, the gradient estimator may involve the term In such an expression, the variance may grow with the time range T. In some cases, the value of T may be approximately 250, which may be high enough to cause significant variance. Assuming the sampling rate is in the range of 10 Hz and the merging area 1130 is 100 meters, the preparation for the merge may begin approximately 300 meters before the merging area. If the host vehicle is traveling at a speed of 16 meters per second (approximately 60 kilometers per hour), the value of T for the event may be approximately 250.

[0262] Back to the concept of option charts, Figure 11E The possible representations are shown in Figure 11D. As previously discussed, an option graph can represent a hierarchical set of decisions organized as a directed acyclic graph (DAG). There may be a special node in the graph called a "root" node 1140, which may be the only node with no incoming edges (e.g., decision lines). The decision process starts at the root node and traverses the graph until it reaches a "leaf" node, i.e., a node with no outgoing edges. Each internal node can implement a policy function that selects children from its available children. There may be a set of options that can be traversed from the set on the option graph to the desired set. In other words, a traversal on the option graph may be automatically converted to Given a node v in the graph, the parameter vector θ v A strategy for selecting children of v can be specified. If θ is all θ v The concatenation of It can be defined by traversing the graph from the root to the leaves, using θ at each node v v Defines the strategy to select child nodes.

[0263] exist Figure 11E In the dual merging options graph 1139, the root node 1140 may first determine whether the host vehicle is in the merging area (e.g., Figure 11D area 1130), or alternatively whether the host vehicle is approaching a merging area and needs to prepare for a possible merge. In both cases, the host vehicle may need to decide whether to change lanes (e.g., to the left or to the right) or whether to stay in the current lane. If the host vehicle has already decided to change lanes, the host vehicle may need to decide whether conditions are suitable to continue and perform the lane change maneuver (e.g., at "go" node 1142). If changing lanes is not possible, the host vehicle may attempt to "push" toward the desired lane by aiming at lane markings (e.g., at node 1144 as part of a negotiation with the vehicle in the desired lane). Alternatively, the host vehicle may choose to "stay" in the same lane (e.g., at node 1146). Such a process can determine the lateral position of the host vehicle in a natural manner. For example,

[0264] This can enable determination of a desired lateral position in a natural way. For example, if the host vehicle changes lanes from lane 2 to lane 3, the "go" node can set the desired lateral position to 3, the "stay" node can set the desired lateral position to 2, and the "advance" node can set the desired lateral position to 2.5. Next, the host vehicle can decide whether to maintain the "same" speed (node ​​1148), "speed up" (node ​​1150), or "slow down" (node ​​1152). Next, the host vehicle can enter a "chain" structure 1154 that covers the other vehicles and sets their semantic meaning to the values ​​in the set {g, t, o}. This process can set expectations relative to other vehicles. The parameters of all nodes in the chain can be shared (similar to a recurrent neural network).

[0265] A potential benefit of the option is the interpretability of the results. Another potential benefit is the ability to rely on the set The decomposable structure of , thus, the policy at each node can be selected from a small number of possibilities. Additionally, this structure can allow reducing the variance of the policy gradient estimator.

[0266] As described above, the length of an event in a double merging scenario can be approximately T = 250 steps. Such a value (or any other suitable value, depending on the specific navigation scenario) can provide sufficient time to see the consequences of the host vehicle's actions (e.g., if the host vehicle decides to change lanes in preparation for a merge, the host vehicle will only see the benefits after successfully completing the merge). On the other hand, due to the dynamic nature of driving, the host vehicle must make decisions at a sufficiently high frequency (e.g., 10 Hz in the scenario described above).

[0267] Option graphs can reduce the effective value of T in at least two ways. First, given higher-level decisions, rewards can be defined for lower-level decisions while considering shorter events. For example, when the host vehicle has selected the "Change Lane" and "Go" nodes, a policy for assigning semantic meaning to the vehicle can be learned by looking at 2-3 seconds of events (meaning T becomes 20-30 instead of 250). Second, for high-level decisions (such as whether to change lanes or stay in the same lane), the host vehicle may not need to make a decision every 0.1 seconds. Instead, the host vehicle may be able to make decisions at a lower frequency (e.g., every second) or implement an "option termination" function, and then the gradient is calculated only after each termination of the option. In both cases, the effective value of T may be an order of magnitude smaller than its original value. In summary, the estimator at each node can rely on a value of T that is an order of magnitude smaller than the original 250 steps, which can instantly translate into smaller variance.

[0268] As described above, hard constraints can promote safer driving, and there may be several different types of constraints. For example, static hard constraints can be defined directly from the sensed state. These may include speed bumps, speed limits, road curvatures, intersections, etc. within the host vehicle environment, which may involve one or more constraints on vehicle speed, heading, acceleration, braking (deceleration), etc. Static hard constraints may also include semantic free space, in which the host vehicle is prohibited from leaving the free space and is prohibited from navigating too close to physical obstacles. Static hard constraints can also limit (e.g., prohibit) manipulations that do not conform to various aspects of the kinematic motion of the vehicle. For example, static hard constraints can be used to prohibit manipulations that may cause the host vehicle to overturn, slide, or lose control.

[0269] Hard constraints may also be associated with vehicles. For example, a constraint may be employed that requires a vehicle to maintain a longitudinal distance of at least one meter from other vehicles and a lateral distance of at least 0.5 meters from other vehicles. Constraints may also be employed that require the host vehicle to avoid maintaining a collision course with one or more other vehicles. For example, time τ may be a measurement of time based on a particular scenario. The predicted trajectories of the host vehicle and one or more other vehicles may be considered from the current time to time τ. When two trajectories intersect, We can express the arrival and departure times of vehicle i at an intersection. That is, when the first car of each group passes the intersection, the car will arrive at the point, and a certain amount of time will pass before the last car passes the intersection. This amount of time separates the arrival time from the departure time. Assume (i.e., the arrival time of vehicle 1 is less than the arrival time of vehicle 2), we want to ensure that vehicle 1 has left the intersection before vehicle 2 arrives. Otherwise, a collision will result. Therefore, it is possible to achieve Furthermore, to ensure that vehicle 1 and vehicle 2 do not miss each other by a minimum amount, an additional safety margin can be obtained by including a buffer time (e.g., 0.5 seconds or other suitable value) in the constraint. The hard constraint related to the predicted crossing trajectories of the two vehicles can be expressed as

[0270] The amount of time τ to track the trajectory of the host vehicle and one or more other vehicles can vary. However, in an intersection scenario, where speeds may be lower, τ may be longer, and τ may be defined such that the host vehicle enters and leaves the intersection in less than τ seconds.

[0271] Of course, applying hard constraints to vehicle trajectories requires predicting the trajectories of these vehicles. Trajectory prediction can be relatively straightforward for the host vehicle, as the host vehicle typically already knows and is actually planning its intended trajectory at any given time. Predicting the trajectories of other vehicles relative to other vehicles may be less straightforward. For other vehicles, a baseline calculation for determining the predicted trajectory can rely on the current speed and heading of the other vehicles, for example, based on analysis of image streams captured by one or more cameras and / or other sensors (radio detection and ranging, light detection and ranging, acoustics, etc.) on the host vehicle.

[0272] However, there may be some exceptions that can simplify the problem or at least provide additional confidence in the trajectory predicted for the other vehicle. For example, for structured roads with lane indications and possible give-way rules, the trajectory of the other vehicle can be based at least in part on the position of the other vehicle relative to the lane and on the applicable give-way rules. Therefore, in some cases, when there is observed lane structure, it can be assumed that the next-lane vehicle will respect the lane boundaries. That is, the host vehicle can assume that the next-lane vehicle will stay in its lane unless there is observed evidence (e.g., a signal light, strong lateral movement, movement across a lane boundary) that the next-lane vehicle will cut into the host vehicle's lane.

[0273] Other situations may also provide clues about the intended trajectory of other vehicles. For example, at stop signs, traffic lights, roundabouts, and so on, the primary vehicle may have the right of way, and it can be assumed that other vehicles will obey that right of way. Therefore, unless there is observed evidence of a violation, other vehicles can be assumed to follow a trajectory that obeys the right of way held by the primary vehicle.

[0274] Hard constraints can also be applied relative to pedestrians in the host vehicle's environment. For example, a buffer distance can be established relative to pedestrians, such that the host vehicle is prohibited from navigating closer than a specified buffer distance to any observed pedestrian. The pedestrian buffer distance can be any suitable distance. In some embodiments, the buffer distance can be at least one meter relative to an observed pedestrian.

[0275] Similar to the case of vehicles, hard constraints can also be applied with respect to the relative motion between pedestrians and the host vehicle. For example, the trajectory of a pedestrian (based on heading direction and speed) can be monitored relative to the projected trajectory of the host vehicle. Given a particular pedestrian trajectory, for each point p on the trajectory, t(p) can represent the time required for the pedestrian to reach point p. In order to maintain the required buffer distance of at least 1 meter from the pedestrian, t(p) must be greater than the time it takes for the host vehicle to reach point p (with enough time difference for the host vehicle to pass in front of the pedestrian at a distance of at least one meter), or t(p) must be less than the time it takes for the host vehicle to reach point p (for example, if the host vehicle brakes to give way to the pedestrian). In the latter example, the hard constraint may still require the host vehicle to arrive at point p at a time sufficiently later than the pedestrian so that the host vehicle can pass behind the pedestrian and maintain the required buffer distance of at least 1 meter. Of course, there may be exceptions to the hard constraints for pedestrians. For example, pedestrian hard constraints may be relaxed (e.g., to a smaller buffer of at least 0.75 m or 0.50 m) in situations where the host vehicle has the right of way or at very slow speeds, and where there is no observed evidence of pedestrians yielding to or otherwise navigating toward the host vehicle.

[0276] In some examples, if it is determined that not all constraints can be satisfied, the constraints can be relaxed. For example, if the road is too narrow to leave the required spacing (e.g., 0.5 meters) between two curbs or between a curb and a parked vehicle, if there is a mitigating situation, one or more constraints can be relaxed. For example, if there are no pedestrians (or other objects) on the sidewalk, you can drive slowly at a distance of 0.1 meters from the curb. In some embodiments, if relaxing the constraints will improve the user experience, the constraints can be relaxed. For example, to avoid potholes, constraints can be relaxed to allow vehicles to navigate closer to the edge of the lane, curb, or pedestrians than is normally allowed. In addition, in some embodiments, when determining the constraints to be relaxed, the one or more constraints selected for relaxation are considered to be those with the smallest available negative impact on safety. For example, constraints related to how close a vehicle travels to a curb or concrete guardrail can be relaxed before constraints on proximity to other vehicles are relaxed. In some embodiments, pedestrian constraints may be the last to be relaxed, or in some cases may never be relaxed.

[0277] Figure 12Examples of scenes that can be captured and analyzed during navigation of a host vehicle are shown. For example, as described above, the host vehicle can include a navigation system (e.g., system 100) that can receive a plurality of images representing the host vehicle's environment from a camera associated with the host vehicle (e.g., at least one of image capture device 122, image capture device 124, and image capture device 126). Figure 12 The scene shown in is an example of one of the images that may be captured at time t from the environment of a host vehicle traveling in lane 1210 along predicted trajectory 1212. The navigation system may include at least one processing device (e.g., including any EyeQ processor or other device described above) that is specifically programmed to receive multiple images and analyze the images to determine an action in response to the scene. Specifically, as Figure 8 As shown, at least one processing device may implement a sensing module 801, a driving strategy module 803, and a control module 805. The sensing module 801 may be responsible for collecting and outputting image information collected from a camera, and providing this information in the form of an identified navigation state to the driving strategy module 803, which may constitute a trained navigation system that has been trained through machine learning techniques, such as supervised learning, reinforcement learning, etc. Based on the navigation state information provided by the sensing module 801 to the driving strategy module 803, the driving strategy module 803 (e.g., by implementing the option chart method described above) may generate a desired navigation action for execution by the host vehicle in response to the identified navigation state.

[0278] In some embodiments, at least one processing device can directly convert the desired navigation action into a navigation command using, for example, the control module 805. However, in other embodiments, hard constraints can be applied so that the desired navigation action provided by the driving strategy module 803 is tested against various predetermined navigation constraints that may be involved in the scene and the desired navigation action. For example, in the case where the driving strategy module 803 outputs a desired navigation action that will cause the host vehicle to follow the trajectory 1212, the desired navigation action can be tested against one or more hard constraints associated with various aspects of the host vehicle's environment. For example, the captured image 1201 can reveal the presence of a curb 1213, a pedestrian 1215, a target vehicle 1217, and a stationary object (e.g., an overturned box) in the scene. Each of these constraints may be associated with one or more hard constraints. For example, the curb 1213 can be associated with a static constraint that prohibits the host vehicle from navigating into the curb or past the curb and onto the sidewalk 1214. The curb 1213 may also be associated with a roadside barrier envelope that defines a distance (e.g., a buffer zone) extending away from and along the curb (e.g., 0.1 m, 0.25 m, 0.5 m, 1 m, etc.) that defines a non-navigable zone for the host vehicle. Of course, static constraints may also be associated with other types of roadside boundaries (e.g., guardrails, concrete bollards, traffic cones, pylons, or any other type of roadside barrier).

[0279] It should be noted that distance and ranging can be determined by any suitable method. For example, in some embodiments, distance information can be provided by an onboard radio detection and ranging and / or light detection and ranging system. Alternatively or additionally, distance information can be derived from an analysis of one or more images captured from the environment of the host vehicle. For example, the number of pixels of an identified object represented in an image can be determined and compared to the known field of view and focal length geometry of the image capture device to determine scale and distance. For example, velocity and acceleration can be determined by observing the change in scale between objects from one image to another over a known time interval. This analysis can indicate the direction of motion toward or away from the host vehicle, as well as the speed at which the object is moving away from or approaching the host vehicle. Crossing speed can be determined by analyzing the change in the X coordinate position of the object from one image to another over a known time period.

[0280] Pedestrian 1215 may be associated with a pedestrian envelope that defines a buffer zone 1216. In some cases, a hard constraint may be imposed that prohibits the host vehicle from navigating within 1 meter of pedestrian 1215 (in any direction relative to the pedestrian). Pedestrian 1215 may also define the location of a pedestrian influence zone 1220. Such an influence zone may be associated with a constraint that limits the speed of the host vehicle within the influence zone. The influence zone may extend 5 meters, 10 meters, 20 meters, and so on from pedestrian 1215. Each scale of the influence zone may be associated with a different speed limit. For example, within a zone from 1 to 5 meters from pedestrian 1215, the host vehicle may be limited to a first speed (e.g., 10 miles per hour, 20 miles per hour, etc.), which may be less than the speed limit within the pedestrian influence zone extending from 5 meters to 10 meters. Any scale of influence zone stages may be used. In some embodiments, the first stage may be narrower than 1 to 5 meters and may extend only from one to two meters. In other embodiments, the first phase of the influence zone may extend from 1 meter (the boundary of the no-navigation zone around the pedestrian) to a distance of at least 10 meters. The second phase, in turn, may extend from 10 meters to at least about 20 meters. The second phase may be associated with a maximum travel speed of the host vehicle that is greater than the maximum travel speed associated with the first phase of the pedestrian influence zone.

[0281] The scene detected in the host vehicle's environment may also involve one or more stationary object constraints. For example, in image 1201, at least one processing device may detect a stationary object, such as a box 1219 in the road. The detected stationary object may include various objects, such as at least one of a tree, a pole, a road sign, or an object in the road. One or more predefined navigation constraints may be associated with the detected stationary object. For example, such constraints may include a stationary object envelope, wherein the stationary object envelope defines a buffer zone around the object within which the host vehicle may be prohibited from navigating. At least a portion of the buffer zone may extend a predetermined distance from the edge of the detected stationary object. For example, in the scene represented by image 1201, a buffer zone of at least 0.1 meters, at least 0.25 meters, at least 0.5 meters, or longer may be associated with box 1219, such that the host vehicle will pass at least a distance to the right or left of the box (e.g., a buffer zone distance) to avoid colliding with the detected stationary object.

[0282] The defined hard constraints may also include one or more target vehicle constraints. For example, target vehicle 1217 may be detected in image 1201. To ensure that the host vehicle does not collide with target vehicle 1217, one or more hard constraints may be employed. In some cases, the target vehicle envelope may be associated with a single buffer zone distance. For example, a buffer zone may be defined by a 1-meter distance around the target vehicle in all directions. The buffer zone may define a zone extending at least one meter from the target vehicle into which the host vehicle is prohibited from navigating.

[0283] However, the envelope around the target vehicle 1217 need not be defined by a fixed buffer distance. In some cases, the predefined hard constraints associated with the target vehicle (or any other movable object detected in the host vehicle's environment) may depend on the orientation of the host vehicle relative to the detected target vehicle. For example, in some cases, the longitudinal buffer distance (e.g., the longitudinal buffer distance extending from the target vehicle to the front or rear of the host vehicle, such as when the host vehicle is traveling toward the target vehicle) may be at least 1 meter. The lateral buffer distance (e.g., the distance extending from the target vehicle to either side of the host vehicle - such as when the host vehicle is traveling in the same or opposite direction as the target vehicle so that one side of the host vehicle will pass adjacent to one side of the target vehicle) may be at least 0.5 meters.

[0284] As described above, the detection of pedestrians in the environment of a target vehicle or host vehicle may also involve other constraints. For example, the predicted trajectories of the host vehicle and the target vehicle 1217 may be considered, and if the two trajectories intersect (e.g., at the intersection 1230), a hard constraint may require that or Where the host vehicle is vehicle 1 and the target vehicle 1217 is vehicle 2. Similarly, the trajectory of pedestrian 1215 (based on heading direction and speed) can be monitored relative to the expected trajectory of the host vehicle. Given a particular pedestrian trajectory, each point p on the trajectory, t(p) will represent the pedestrian's arrival at point p (i.e. Figure 12 To maintain the required buffer distance of at least 1 meter behind the pedestrian, t(p) must be greater than the time it takes the host vehicle to arrive at point p (with enough time difference for the host vehicle to pass at least one meter in front of the pedestrian), or t(p) must be less than the time it takes the host vehicle to arrive at point p (for example, if the host vehicle brakes to yield to the pedestrian). In the latter example, the hard constraint will still require the host vehicle to arrive at point p sufficiently later than the pedestrian so that it can pass behind the pedestrian and maintain the required buffer distance of at least 1 meter.

[0285] Other hard constraints may also be employed. For example, in at least some cases, a maximum deceleration of the host vehicle may be employed. Such a maximum deceleration may be determined based on a detected distance to a target vehicle following the host vehicle (e.g., using images collected from a rear-facing camera). Hard constraints may include mandatory stops at sensed crosswalks or railroad crossings, or other applicable constraints.

[0286] When the analysis of the scene in the host vehicle environment indicates that one or more predefined navigation constraints may be involved, these constraints can be applied relative to one or more planned navigation actions of the host vehicle. For example, when the analysis of the scene causes the driving strategy module 803 to return an expected navigation action, the expected navigation action can be tested relative to one or more involved constraints. If the expected navigation action is determined to violate any aspect of the involved constraints (for example, if the expected navigation action will cause the vehicle to be carried within 0.7 meters of a pedestrian 1215, where the predefined hard constraint requires the host vehicle to maintain a distance of at least 1.0 meters from the pedestrian 1215), then at least one modification can be made to the expected navigation action based on one or more predefined navigation constraints. Adjusting the expected navigation action in this way can provide the host vehicle with a navigation action that conforms to the constraints involved in the specific scene detected in the environment of the host vehicle.

[0287] After determining the actual navigational maneuver of the host vehicle, the navigational maneuver can be implemented by causing at least one adjustment of a navigational actuator of the host vehicle in response to the determined actual navigational maneuver of the host vehicle. Such a navigational actuator may include at least one of a steering mechanism, a brake, or an accelerator of the host vehicle.

[0288] Precedence Constraints

[0289] As described above, the navigation system can employ various hard constraints to ensure safe operation of the host vehicle. These constraints can include a minimum safe driving distance relative to pedestrians, target vehicles, road obstacles, or detected objects, a maximum driving speed when passing within the influence zone of a detected pedestrian, or a maximum deceleration of the host vehicle, among others. These constraints can be imposed in conjunction with a trained system based on machine learning (supervised, reinforcement, or a combination thereof), but can also be useful in conjunction with an untrained system (e.g., a system that employs algorithms to directly address expected situations arising in the host vehicle's environment).

[0290] In either case, there may be a hierarchy of constraints. In other words, some navigation constraints may take precedence over others. Therefore, if a situation arises where a navigation action that would result in all involved constraints being satisfied is unavailable, the navigation system may determine the available navigation action that first implements the highest-priority constraint. For example, the system may cause a vehicle to avoid a pedestrian first, even if navigation to avoid the pedestrian would result in a collision with another vehicle or an object detected on the road. In another example, the system may cause a vehicle to drive over a curb to avoid a pedestrian.

[0291] Figure 13 A flow chart illustrating an algorithm for implementing a hierarchy of constraints involved that is determined based on an analysis of a scene in the environment of a host vehicle is provided. For example, at step 1301, at least one processing device associated with a navigation system (e.g., an EyeQ processor, etc.) may receive a plurality of images representing the environment of the host vehicle from a camera mounted on the host vehicle. By analyzing the one or more images representing the scene of the host vehicle's environment at step 1303, a navigation state associated with the host vehicle may be identified. For example, as Figure 12 As shown, among the various attributes of the scene, the navigation state can indicate that the main vehicle is traveling along a two-lane road 1210, that a target vehicle 1217 is passing through an intersection in front of the main vehicle, that a pedestrian 1215 is waiting to cross the road on which the main vehicle is traveling, and that an object 1219 appears in front of the main lane.

[0292] In step 1305, one or more navigation constraints involved in the navigation state of the host vehicle may be determined. For example, after analyzing a scene in the environment of the host vehicle represented by one or more captured images, at least one processing device may determine one or more navigation constraints involved by objects, vehicles, pedestrians, etc. identified by image analysis of the captured images. In some embodiments, the at least one processing device may determine at least a first predefined navigation constraint and a second predefined navigation constraint involved in the navigation state, and the first predefined navigation constraint may be different from the second predefined navigation constraint. For example, the first navigation constraint may involve one or more target vehicles detected in the environment of the host vehicle, and the second navigation constraint may involve pedestrians detected in the environment of the host vehicle.

[0293] In step 1307, at least one processing device may determine a priority associated with the constraints identified in step 1305. In the described example, a second predefined navigation constraint associated with pedestrians may have a higher priority than a first predefined navigation constraint associated with the target vehicle. While the priority associated with a navigation constraint can be determined or assigned based on a variety of factors, in some embodiments, the priority of a navigation constraint may be related to its relative importance from a safety perspective. For example, while it may be important to comply with or satisfy all implemented navigation constraints in as many situations as possible, some constraints may be associated with greater safety risks than others and, therefore, may be assigned a higher priority. For example, a navigation constraint requiring the host vehicle to maintain a separation distance of at least 1 meter from a pedestrian may have a higher priority than a constraint requiring the host vehicle to maintain a separation distance of at least 1 meter from a target vehicle. This may be because a collision with a pedestrian may have more severe consequences than a collision with another vehicle. Similarly, maintaining a separation distance between the host vehicle and the target vehicle may have a higher priority than a constraint requiring the host vehicle to avoid boxes in the road, travel below a certain speed over speed bumps, or expose the host vehicle occupants to no more than a maximum acceleration level.

[0294] While the driving strategy module 803 is designed to maximize safety by satisfying the navigation constraints involved in a particular scenario or navigation state, in some cases, satisfying every involved constraint is simply impossible. In such cases, as shown in step 1309, the priority of each involved constraint can be used to determine which involved constraint should be satisfied first. Continuing with the above example, in a situation where it is impossible to satisfy both the pedestrian gap constraint and the target vehicle gap constraint simultaneously and only one of the constraints can be satisfied, the higher priority of the pedestrian gap constraint may result in satisfying that constraint before attempting to maintain a gap with the target vehicle. Therefore, under normal circumstances, at least one processing device can determine a first navigation action for the host vehicle that satisfies both the first predefined navigation constraint and the second predefined navigation constraint based on the identified navigation state of the host vehicle, where both the first predefined navigation constraint and the second predefined navigation constraint can be satisfied, as shown in step 1311. However, in other situations where not all of the constraints involved can be satisfied, at least one processing device may determine, based on the identified navigation state, a second navigation action for the host vehicle that satisfies a second predefined navigation constraint (i.e., a higher priority constraint) but does not satisfy a first predefined navigation constraint (having a lower priority than the second navigation constraint), wherein the first predefined navigation constraint and the second predefined navigation constraint cannot be satisfied simultaneously, as shown at step 1313.

[0295] Next, in step 1315, to implement the determined navigation action for the host vehicle, the at least one processing device may cause at least one adjustment of a navigation actuator of the host vehicle in response to the first navigation action determined for the host vehicle or the second navigation action determined for the host vehicle. As described in the previous example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0296] Constraint relaxation

[0297] As discussed above, navigation constraints may be imposed for safety purposes. These constraints may include a minimum safe driving distance relative to a pedestrian, target vehicle, road obstacle, or detected object, a maximum driving speed when passing within the influence zone of a detected pedestrian, or a maximum deceleration of the host vehicle, and so on. These constraints may be imposed on a learning or non-learning navigation system. In some cases, these constraints may be relaxed. For example, in a case where the host vehicle slows down or stops near a pedestrian and then moves slowly forward to convey its intention to pass the pedestrian, the pedestrian's response may be detected from the collected image. If the pedestrian's response is to remain still or stop moving (and / or if eye contact with the pedestrian is sensed), it may be understood that the pedestrian recognizes the navigation system's intention to pass the pedestrian. In such cases, the system may relax one or more predefined constraints and implement less stringent constraints (e.g., allowing the vehicle to navigate within 0.5 meters of the pedestrian rather than within a more stringent 1-meter boundary).

[0298] Figure 14 A flow chart for implementing control of a host vehicle based on the relaxation of one or more navigation constraints is provided. At step 1401, at least one processing device may receive a plurality of images representing an environment of the host vehicle from a camera associated with the host vehicle. At step 1403, analysis of the images may enable identification of a navigation state associated with the host vehicle. At step 1405, at least one processor may determine a navigation constraint associated with the navigation state of the host vehicle. The navigation constraint may include a first predefined navigation constraint implicated by at least one aspect of the navigation state. At step 1407, analysis of the plurality of images may reveal the presence of at least one navigation constraint relaxation factor.

[0299] A navigation constraint relaxation factor may include any suitable indicator that indicates that one or more navigation constraints may be suspended, changed, or otherwise relaxed in at least one aspect. In some embodiments, at least one navigation constraint relaxation factor may include determining (based on image analysis) that the pedestrian's eyes are looking in the direction of the host vehicle. In such cases, it can be more safely assumed that the pedestrian is aware of the host vehicle. As a result, the confidence level may be higher that the pedestrian will not engage in unexpected actions that would cause the pedestrian to enter the path of the host vehicle. Other constraint relaxation factors may also be used. For example, at least one navigation constraint relaxation factor may include: a pedestrian determined to be non-moving (e.g., a pedestrian presumed to be unlikely to enter the path of the host vehicle); or a pedestrian determined to have its movement speed slowed down. Navigation constraint relaxation factors may also include more complex actions, such as determining that the pedestrian is non-moving after the host vehicle has stopped and then resuming movement. In this case, the pedestrian may be assumed to understand that the host vehicle has the right of way, and the stopped pedestrian may indicate that the pedestrian intends to yield to the host vehicle. Other situations that may cause one or more constraints to be relaxed include the type of curb (e.g., a low curb or a curb with a gradual slope may allow a relaxed distance constraint), the lack of pedestrians or other objects on the sidewalk, vehicles with engines not running that may have relaxed distances, or situations where pedestrians are facing away and / or moving away from the area toward which the host vehicle is traveling.

[0300] In the event that the presence of a navigation constraint relaxation factor is identified (e.g., in step 1407), a second navigation constraint may be determined or developed in response to the detection of the constraint relaxation factor. The second navigation constraint may be different from the first navigation constraint and may include at least one feature that is relaxed relative to the first navigation constraint. The second navigation constraint may include a newly generated constraint based on the first constraint, wherein the newly generated constraint includes at least one modification that relaxes the first constraint in at least one aspect. Alternatively, the second constraint may constitute a predetermined constraint that is less strict than the first navigation constraint in at least one aspect. In some embodiments, such a second constraint may be retained only for situations where a constraint relaxation factor is identified in the environment of the main vehicle. Regardless of whether the second constraint is newly generated or selected from a fully or partially available set of predetermined constraints, the application of the second navigation constraint in place of the stricter first navigation constraint (which may be applied in the absence of detection of the relevant navigation constraint relaxation factor) may be referred to as constraint relaxation and may be completed in step 1409.

[0301] If at least one constraint relaxation factor is detected at step 1407 and at least one constraint has been relaxed at step 1409, a navigation action of the host vehicle may be determined at step 1411. The navigation action of the host vehicle may be based on the identified navigation state and may satisfy the second navigation constraint. The navigation action may be implemented at step 1413 by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action.

[0302] As described above, the use of navigation constraints and relaxed navigation constraints can be employed in conjunction with a trained (e.g., through machine learning) or an untrained (e.g., a system programmed to respond with a predetermined action in response to a specific navigation state) navigation system. In the case of using a trained navigation system, the availability of relaxed navigation constraints in certain navigation situations can represent a mode switch from a trained system response to an untrained system response. For example, a trained navigation network can determine an original navigation action for a host vehicle based on a first navigation constraint. However, the action taken by the vehicle may be different from the navigation action that satisfies the first navigation constraint. Instead, the action taken can satisfy a second, more relaxed navigation constraint and can be an action developed by an untrained system (e.g., as a response to the detection of a specific condition in the host vehicle environment, such as the presence of a navigation constraint relaxation factor).

[0303] There are many examples of navigation constraints that can be relaxed in response to the detection of a constraint relaxation factor in the host vehicle's environment. For example, when a predefined navigation constraint includes a buffer zone associated with a detected pedestrian and at least a portion of the buffer zone extends a certain distance from the detected pedestrian, the relaxed navigation constraint (newly developed, recalled from memory from a predetermined set, or generated as a relaxed version of a pre-existing constraint) can include a different or modified buffer zone. For example, the different or modified buffer zone may be at a distance from the pedestrian that is less than the distance of the original or unmodified buffer zone relative to the detected pedestrian. Thus, in view of the relaxed constraint, the host vehicle may be allowed to navigate closer to the detected pedestrian if an appropriate constraint relaxation factor is detected in the host vehicle's environment.

[0304] As mentioned above, the relaxation of navigation constraints may include a reduced width in a buffer zone associated with at least one pedestrian. However, the relaxation may also include a reduced width in a buffer zone associated with a target vehicle, a detected object, a roadside obstacle, or any other object detected in the host vehicle environment.

[0305] At least one relaxation feature may also include other types of modifications to navigation constraint characteristics. For example, a relaxation feature may include an increase in speed associated with at least one predefined navigation constraint. A relaxation feature may also include an increase in the maximum allowable acceleration / deceleration associated with at least one predefined navigation constraint.

[0306] As described above, while constraints may be relaxed in some cases, in other cases, navigation constraints may be enhanced. For example, in some cases, the navigation system may determine that conditions justify enhancing the normal set of navigation constraints. Such enhancements may include adding new constraints to a predefined set of constraints or adjusting one or more aspects of the predefined constraints. Adding or adjusting may result in more conservative navigation relative to a predefined set of constraints applicable under normal driving conditions. Conditions that may justify enhancing constraints include sensor failures, adverse environmental conditions (rain, snow, fog, or other conditions associated with reduced visibility or reduced vehicle traction), etc.

[0307] Figure 15 A flow chart for implementing control of a host vehicle based on augmentation of one or more navigation constraints is provided. At step 1501, at least one processing device may receive a plurality of images representing an environment of the host vehicle from a camera associated with the host vehicle. At step 1503, analysis of the images may enable identification of a navigation state associated with the host vehicle. At step 1505, at least one processor may determine a navigation constraint associated with the navigation state of the host vehicle. The navigation constraint may include a first predefined navigation constraint implicated by at least one aspect of the navigation state. At step 1507, analysis of the plurality of images may reveal the presence of at least one navigation constraint augmenting factor.

[0308] The navigation constraints involved may include the above (e.g., Figure 12) or any other suitable navigation constraints. A navigation constraint enhancing factor may include any indicator that one or more navigation constraints may be supplemented / enhanced in at least one aspect. The supplementation or enhancement of navigation constraints may be performed on a set-by-set basis (e.g., by adding a new navigation constraint to a predetermined set of constraints) or on a constraint-by-constraint basis (e.g., by modifying a particular constraint so that the modified constraint is more restrictive than the original constraint, or by adding a new constraint corresponding to a predetermined constraint, wherein the new constraint is more restrictive in at least one aspect than the corresponding constraint). Additionally or alternatively, the supplementation or enhancement of navigation constraints may refer to a selection from a set of predetermined constraints based on a hierarchical structure. For example, a set of enhanced constraints may be selected based on whether a navigation enhancing factor is detected in the environment of or relative to the host vehicle. Under normal conditions where no enhancing factors are detected, the involved navigation constraints may be extracted from the constraints applicable under normal conditions. On the other hand, when one or more constraint enhancing factors are detected, the involved constraints may be extracted from the enhanced constraints generated or predefined relative to the one or more enhancing factors. The enhanced constraints may be more restrictive in at least one aspect than the corresponding constraints applicable under normal conditions.

[0309] In some embodiments, at least one navigation constraint enhancing factor may include detecting (e.g., based on image analysis) the presence of ice, snow, or water on a road surface in the host vehicle environment. Such a determination may be based on, for example, detecting: areas of higher reflectivity than expected for dry road (e.g., indicating ice or water on the road); white areas on the road indicating snow; shadows on the road consistent with the presence of longitudinal grooves on the road (e.g., tire tracks in the snow); water droplets or ice / snow particles on the host vehicle windshield; or any other suitable indicator of the presence of water or ice / snow on the road surface.

[0310] At least one navigation constraint enhancing factor may also include detecting particles on an exterior surface of a host vehicle windshield. Such particles may impair image quality of one or more image capture devices associated with the host vehicle. While described with respect to a host vehicle windshield associated with a camera mounted behind the host vehicle windshield, detection of particles on other surfaces (e.g., a camera lens or lens cover, a headlight lens, a rear windshield, a taillight lens, or any other surface visible to (or detected by) an image capture device associated with the host vehicle) may also indicate the presence of a navigation constraint enhancing factor.

[0311] A navigation constraint enhancing factor may also be detected as a property of one or more image acquisition devices. For example, a detected degradation in image quality of one or more images captured by an image capture device (e.g., a camera) associated with the host vehicle may also constitute a navigation constraint enhancing factor. The degradation in image quality may be associated with a hardware failure or partial hardware failure associated with the image capture device or a component associated with the image capture device. Such degradation in image quality may also be caused by environmental conditions. For example, the presence of smoke, fog, rain, snow, etc. in the air surrounding the host vehicle may also result in a degradation in image quality relative to roads, pedestrians, target vehicles, etc. that may be present in the host vehicle's environment.

[0312] Navigation constraint enhancing factors may also relate to other aspects of the host vehicle. For example, in some cases, a navigation constraint enhancing factor may include the detection of a malfunction or partial malfunction of a system or sensor associated with the host vehicle. Such enhancing factors may include, for example, the detection of a malfunction or partial malfunction of a speed sensor, a GPS receiver, an accelerometer, a camera, radio detection and ranging, light detection and ranging, brakes, tires, or any other system associated with the host vehicle that may affect the host vehicle's ability to navigate relative to the navigation constraints associated with the host vehicle's navigational state.

[0313] Where the presence of a navigation constraint enhancing factor is identified (e.g., at step 1507), a second navigation constraint may be determined or developed in response to the detection of the constraint enhancing factor. The second navigation constraint may be different from the first navigation constraint and may include at least one feature that is enhanced relative to the first navigation constraint. The second navigation constraint may be more restrictive than the first navigation constraint because the detection of the constraint enhancing factor in the environment of or associated with the host vehicle may indicate that the host vehicle may have at least one navigation capability that is reduced relative to normal operating conditions. Such reduced capability may include reduced road traction (e.g., ice, snow, or water on the road; reduced tire pressure, etc.); impaired vision (e.g., rain, snow, dust, smoke, fog, etc. that reduces the quality of captured images); impaired detection capability (e.g., sensor failure or partial failure, reduced sensor performance, etc.); or any other reduction in the ability of the host vehicle to navigate in response to the detected navigation condition.

[0314] If at least one constraint enhancing factor is detected at step 1507 and at least one constraint has been enhanced at step 1509, a navigation action of the host vehicle may be determined at step 1511. The navigation action of the host vehicle may be based on the identified navigation state and may satisfy the second navigation (i.e., enhanced) constraint. The navigation action may be implemented at step 1513 by causing at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action.

[0315] As discussed, the use of navigation constraints and enhanced navigation constraints can be employed in conjunction with a trained (e.g., through machine learning) or an untrained (e.g., a system programmed to respond with a predetermined action in response to a specific navigation state) navigation system. In the case of using a trained navigation system, the availability of enhanced navigation constraints in certain navigation situations can represent a mode switch from a trained system response to an untrained system response. For example, a trained navigation network can determine an original navigation action for a host vehicle based on a first navigation constraint. However, the action taken by the vehicle may be different from the navigation action that satisfies the first navigation constraint. Instead, the action taken can satisfy an enhanced second navigation constraint and can be an action developed by an untrained system (e.g., as a response to the detection of a particular condition in the host vehicle environment, such as the presence of a navigation constraint enhancing factor).

[0316] There are many examples of navigation constraints that can be generated, supplemented, or enhanced in response to constraint-enhancing factors detected in the host vehicle's environment. For example, where a predefined navigation constraint includes a buffer zone associated with a detected pedestrian, object, vehicle, etc., and at least a portion of the buffer zone extends a distance from the detected pedestrian / object / vehicle, an enhanced navigation constraint (newly developed, recalled from memory from a predetermined set, or generated as an enhanced version of a pre-existing constraint) can include a different or modified buffer zone. For example, the different or modified buffer zone can have a distance relative to the pedestrian / object / vehicle that is potentially greater than the original or unmodified buffer zone relative to the detected pedestrian / object / vehicle. Thus, given the enhanced constraint, the host vehicle may be forced to navigate further away from the detected pedestrian / object / vehicle if appropriate constraint-enhancing factors are detected in or relative to the host vehicle's environment.

[0317] The at least one enhanced feature may also include other types of modifications to navigation constraint characteristics. For example, the enhanced feature may include a speed reduction associated with at least one predefined navigation constraint. The enhanced feature may also include a reduction in the maximum allowable deceleration / acceleration associated with at least one predefined navigation constraint.

[0318] Navigation based on long-range planning

[0319] In some embodiments, the disclosed navigation system can determine one or more navigation actions based not only on a detected navigation state in the host vehicle's environment but also based on long-range planning. For example, the system can consider the potential impact of one or more navigation actions on a future navigation state, which navigation actions are available as options for navigating relative to the detected navigation state. Considering the impact of available actions on future states allows the navigation system to determine navigation actions based not only on the currently detected navigation state but also on long-range planning. Navigation using long-range planning techniques may be particularly useful when the navigation system employs one or more reward functions as a technique for selecting navigation actions from available options. The potential reward can be analyzed relative to available navigation actions taken in response to the detected current navigation state of the host vehicle. However, further, the potential reward can also be analyzed relative to actions taken in response to future navigation states that are expected to result from actions available in the current navigation state. As a result, in some cases, the disclosed navigation system can select a navigation action in response to a detected navigation state, even if the selected navigation action may not yield the highest reward among the available actions that could be taken in response to the current navigation state. This may be particularly true if the system determines that a selected action may result in a future navigation state that results in one or more potential navigation actions that offer a higher reward than the selected action (or, in some cases, any available action relative to the current navigation state). This principle can be stated more simply as: take less favorable actions now to yield higher reward options in the future. Thus, the disclosed navigation system capable of long-range planning can select a suboptimal short-term action if a long-term forecast indicates that a short-term loss of reward may result in a long-term gain in reward.

[0320] In general, autonomous driving applications may involve a series of planning problems, in which the navigation system may decide on immediate actions to optimize long-term goals. For example, when a vehicle encounters a merging situation at a roundabout, the navigation system may decide on immediate acceleration or braking commands to initiate navigation into the roundabout. Although the immediate action on the navigation state detected at the roundabout may involve acceleration or braking commands in response to the detected state, the long-term goal is to successfully merge, and the long-term effect of the selected command is the success / failure of the merge. The planning problem can be solved by breaking the problem into two stages. First, supervised learning can be applied to the near future based on current predictions (assuming that the prediction factors are differentiable with respect to the current representation). Second, a recurrent neural network can be used to model the complete trajectory of the agent, where unexplained factors are modeled as (additive) input nodes. This may allow the use of supervised learning techniques and direct optimization on recurrent neural networks to determine the solution to the long-range planning problem. Such methods can also achieve robust policy learning by adding adversarial factors to the environment.

[0321] The two most fundamental elements of autonomous driving systems are sensing and planning. Sensing involves discovering a compact representation of the current state of the environment, while planning involves deciding which actions to take to optimize future goals. Supervised machine learning techniques are very useful for solving sensing problems. Machine learning algorithmic frameworks can also be used for planning, particularly reinforcement learning (RL) frameworks such as those described above.

[0322] RL can be executed in a sequence of consecutive rounds. At round t, the planner (also called agent or driving policy module 803) can observe the state s t ∈S, the state represents the agent and the environment. Then it should decide on an action a t ∈A. After executing the action, the agent receives an immediate reward and move to the new state s t+1 As an example, the host vehicle may include an adaptive cruise control (ACC) system, where the vehicle should autonomously accelerate / brake in order to maintain a sufficient distance from the vehicle ahead while maintaining a smooth ride. This state can be modeled as where x t is the distance to the leading vehicle, and v t is the speed of the main vehicle relative to the speed of the leading vehicle. will be the acceleration command (where, if a t <0, the host vehicle decelerates). The reward can be determined by |a t | (reflects driving stability) and s t(reflecting the function of maintaining a safe distance between the host vehicle and the preceding vehicle). The goal of the planner is to maximize the cumulative reward (possibly over a time horizon or as a discounted sum of future rewards). To do so, the planner may rely on a policy π: S→A that maps states to actions.

[0323] Supervised learning (SL) can be viewed as a special case of RL, where s t Sampling from some distribution on S, and the reward function may have r t =-l(a t ,y t ), where l is the loss function and the learner observes y t The value of y t is the observed state s t There may be some differences between general RL models and the special case of SL that make the general RL problem more challenging.

[0324] In some SL situations, the actions (or predictions) taken by the learner may have no effect on the environment. In other words, s t+1 and a t are independent. This may have two important implications. First, in SL, samples (s1, y1), ..., (s m ,y m ), and only start searching for policies (or predictions) that have good accuracy relative to the sample. In contrast, in RL, the state s t+1 Usually depends on the action taken (which also depends on the previous state), which in turn depends on the policy used to generate the action. This connects the data generation process with the policy learning process. Second, because the action does not affect the environment in SL, choosing a t The contribution to the performance of π is local. Specifically, a t Only affects the value of the immediate reward. In contrast, in RL, the action taken at round t may have long-term effects on the value of rewards in future rounds.

[0325] In SL, the “correct” answer is y t and the return r t =-θ(a t ,y t ) can be a t All possible choices of provide complete knowledge of the payoff, so that the payoff is relative to a tIn contrast, in RL, the "one-time" value of the reward may be all that is observed for a particular choice of action taken. This may be called "bandit" feedback. This is one of the most important reasons why "exploration" is needed as part of long-term navigation planning, because in RL-based systems, if only "bandit" feedback is available, the system may not always know whether the action taken is the best action.

[0326] Many RL algorithms rely, at least in part, on a mathematically elegant model of the Markov decision process (MDP). The Markov assumption is that, given s t and a t In the case of t+1 The distribution of is completely determined. This yields a closed-form expression for the cumulative reward of a given policy in terms of a stationary distribution over the states of the MDP. The stationary distribution of policies can be expressed as the solution to a linear programming problem. This gives rise to two families of algorithms: 1) optimization of the primal problem, which can be called policy search; and 2) optimization of the dual problem, where the variable being optimized is called the value function V. π If the MDP starts from an initial state s and from then on actions are chosen according to π, then the value function determines the expected cumulative reward. The relevant quantity is the state-action value function Q π (s, a), which determines the cumulative reward assuming we start from state s, choose action a at that moment, and from then on choose actions according to π. The Q function can produce a characteristic of the optimal policy (using the Bellman equation). Specifically, the Q function can show that the optimal policy is a deterministic function from S to A (in fact, it can be characterized as a "greedy" policy with respect to the optimal Q function).

[0327] One potential advantage of the MDP model is that it allows the future to be coupled to the present using a Q function. For example, suppose the primary vehicle is now in state s, then Q π The value of (s, a) may indicate the impact of executing a on the future. Therefore, the Q function can provide a local measure of the quality of action a, making the RL problem more similar to the SL scenario.

[0328] Many RL algorithms approximate the V-function or Q-function in some way. Value iteration algorithms (e.g., Q-learning) may rely on the fact that the V- and Q-functions of the optimal policy may be fixed points of some operators derived from the Bellman equation. The actor-critic policy iteration algorithm aims to learn the policy in an iterative manner, where at iteration t, the “critic” estimates And based on this estimate, the "action party" improves the strategy.

[0329] Despite the mathematical simplicity of MDPs and the ease of switching to Q-function notation, this approach can have some limitations. For example, an approximate notion of a Markovian behavior state may be all that can be found in some situations. Furthermore, the transition between states may depend not only on the actions of the agent, but also on the actions of other actors in the environment. For example, in the ACC example mentioned above, while the dynamics of the autonomous vehicle may be Markovian, the next state may depend on the behavior of the driver of the other vehicle, whose behavior is not necessarily Markovian. One possible solution to this problem is to use a partially observed MDP, where a Markovian state is assumed to exist, but observations according to the hidden state distribution are visible.

[0330] A more straightforward approach might consider game-theoretic generalizations of MDPs (e.g., the stochastic games framework). Indeed, algorithms for MDPs can be generalized to multi-agent games (e.g., minimax-Q learning or Nash-Q learning). Other approaches might include explicit modeling of the other parties and vanishing regret learning algorithms. Learning in a multi-agent setting can be more complex than in a single-agent setting.

[0331] A second limitation on the Q-function representation arises by deviating from the table setting. The table setting is when the number of states and actions is small, and therefore Q can be represented as a table with |S| rows and |A| columns. However, if the natural representation of S and A involves Euclidean space, and the state and action spaces are discrete, then the number of states / actions may be exponential in the dimensions. In this case, adopting a table setting may not be practical. Instead, the Q-function can be approximated by a function from a class of parameter hypotheses (e.g., a neural network of a specific architecture). For example, a deep Q-network (DQN) learning algorithm can be used. In a DQN, the state space can be continuous, but the action space can be kept as a small discrete set. There may be ways to handle continuous action spaces, but they may rely on approximating the Q-function. In any case, the Q-function can be complex and sensitive to noise, and therefore can be difficult to learn.

[0332] Another approach might be to use recurrent neural networks (RNNs) to solve RL problems. In some cases, RNNs can be combined with concepts from game theory about multi-agent games and robustness against adversarial environments. Furthermore, this approach might not explicitly rely on any Markov assumptions.

[0333] The following describes a method based on predictive planning navigation in more detail. In this method, it can be assumed that the state space S is The action space A is a subset of A subset of . In many applications, this may be a natural representation. As described above, there may be two key differences between RL and SL: (1) because past actions affect future rewards, future information may need to be propagated back to the past; (2) the “slot machine” nature of rewards obscures the dependency between (state, action) and rewards, which complicates the learning process.

[0334] As a first step in the approach, it can be observed that there are interesting problems where the bandit nature of the rewards is not an issue. For example, the reward values ​​of ACC applications (discussed in more detail below) can be differentiable with respect to the current state and action. In fact, even if the rewards are given in a "bandit" manner, learning differentiable functions And make The problem of may also be a relatively simple SL problem (e.g., a one-dimensional regression problem). Therefore, the first step of the approach may be to define the return as a function that is differentiable with respect to s and a Or use regression learning algorithms to learn differentiable functions The differentiable function Minimize at least some regression loss on samples where the instance vector is And the scalar is r t In some cases, an exploration element may be used in order to create a training set.

[0335] To solve the connection between the past and the future, a similar idea can be used. For example, suppose a differentiable function can be learned so that Learning such a function can be formulated as a SL problem. can be considered as a predictor of the near future. Next, we can use the parameter function π θ :S→A describes the strategy of mapping from S to A. Representing it as a neural network can be achieved using a recurrent neural network (RNN) to represent the events of running the agent in T rounds, where the next state is defined as Here, May be defined by circumstances and may represent unpredictable aspects of the near future. t+1 Depends on s in a differentiable way t and a t The fact that makes it possible to connect future rewards with past behavior. The parameter vector π of the policy function θ It can be learned by backpropagation on the resulting RNN. Note that there is no need to t impose explicit probabilistic assumptions. In particular, there is no requirement for a Markov relation. Instead, one can rely on the recurrent network to propagate “enough” information between the past and the future. Intuitively, may describe the predictable part of the near future, while ν t may express unpredictable aspects that may arise due to the behavior of other actors in the environment. The learning system should learn policies that will be robust to the behavior of other actors. If ||ν t If || is large, the connection between past actions and future rewards may be too noisy for learning meaningful policies. Explicitly expressing the dynamics of the system in a transparent way makes it easier to incorporate prior knowledge. For example, prior knowledge can simplify the definition of problem.

[0336] As discussed above, the learning system may benefit from robustness against adversarial environments, such as the host vehicle's environment, which may include multiple other drivers who may act in unexpected ways. t In the model with the probability assumption imposed on it, we can consider ν t is an environment that is selected in an adversarial manner. In some cases, the constraint may be placed on μ t Otherwise, the attacker may make the planning problem difficult or even impossible. A natural restriction might be to require ||μ t ||Bounded by a constant.

[0337] In autonomous driving applications, robustness against adversarial environments can be useful. t It can even speed up the learning process because it can focus the learning system on a robust optimal strategy. A simple game can be used to illustrate this concept. Action The instantaneous loss function is 0.1|a t |+[|s t |-2] + , where [x] + =max{x,0} is the ReLU (rectified linear unit) function. The next state is s t+1 =s t +a t +v t , where for the environment, v t ∈[-0.5, 0.5] is chosen in an adversarial manner. Here, the optimal strategy can be written as a two-layer network with ReLU: a t =-[s t -1.5] + +[-s t -1.5] + . Observation: When |s t|∈(1.5, 2], the optimal action may have an immediate loss greater than action a=0. Therefore, the system may plan for the future and may not rely solely on the immediate loss. Observation: The loss relative to a t The derivative is 0.1 sign(a t ), relative to s t The derivative is 1[|s t |>2]sign(s t ). In which t ∈(1.5, 2], v t The counter option would be to set v t = 0.5, and therefore, whenever a t >1.5-s t , there may be non-zero loss in round t+1. In this case, the derivative of the loss may be directly back-propagated to a t Therefore, in a t The choice is suboptimal in the case that v t Adversarial selection of can help the navigation system obtain non-zero backpropagation messages. This relationship may help the navigation system choose the current action (even if the action will result in a suboptimal reward or even a loss) based on the expectation that such current action will provide an opportunity for a better action that will result in a higher reward in the future.

[0338] This approach can be applied to almost any navigation situation that may arise. Below is a description of the approach applied to one example: Adaptive Cruise Control (ACC). In the ACC problem, the host vehicle may try to maintain a sufficient distance from the target vehicle ahead (e.g., 1.5 seconds behind the target vehicle). Another goal may be to drive as smoothly as possible while maintaining a desired gap. A model representing this situation can be defined as follows. The state space is The action space is The first coordinate of the state is the speed of the target car, the second coordinate is the speed of the host vehicle, and the last coordinate is the distance between the host vehicle and the target vehicle (e.g., the position of the host vehicle minus the position of the target along the curve of the road). The action that the host vehicle wants to take is to accelerate, which can be expressed as a t The quantity τ may represent the time difference between successive rounds. Although τ may be set to any suitable amount, in one example, τ may be 0.1 seconds. Position s t It can be expressed as The (unknown) acceleration of the target vehicle can be expressed as

[0339] The complete dynamics of the system can be described by the following equation:

[0340]

[0341] This can be described as the sum of two vectors:

[0342]

[0343] The first vector is the predictable part, and the second vector is the unpredictable part. The return of round t is defined as follows:

[0344]

[0345] The first term may result in a penalty for non-zero acceleration, thus encouraging smooth driving. The second term depends on the distance x to the target car. t and expected distance The ratio between the two is defined as the maximum between the 1 meter distance and the 1.5 second interruption distance. In some cases, this ratio may be exactly 1, but as long as the ratio is in the range [0.7, 1.3], the strategy is exempt from any penalty, which can allow the host vehicle some relaxation in navigation, a property that can be important for achieving smooth driving.

[0346] By implementing the methods outlined above, the navigation system of the host vehicle (e.g., through operation of the driving strategy module 803 within the processing unit 110 of the navigation system) can select an action in response to the observed state. The selected action can be based not only on an analysis of the rewards associated with the available responsive actions relative to the sensed navigation state, but also on consideration and analysis of future states, potential actions in response to the future states, and the rewards associated with the potential actions.

[0347] Figure 16 The figure illustrates an algorithmic approach to navigation based on detection and long-range planning. For example, at step 1601, at least one processing device 110 of a navigation system for a host vehicle may receive a plurality of images. These images may capture a scene representing the host vehicle's environment and may be provided by any of the image capture devices described above (e.g., a camera, a sensor, etc.). At step 1603, analysis of one or more of these images may enable the at least one processing device 110 to identify a current navigation state associated with the host vehicle (as described above).

[0348] Various potential navigation actions in response to the sensed navigation state may be determined in steps 1605, 1607, and 1609. These potential navigation actions (e.g., a first navigation action through an Nth available navigation action) may be determined based on the sensed state and a long-range goal of the navigation system (e.g., to complete a merge, smoothly follow a preceding vehicle, pass a target vehicle, avoid an object on the road, slow down for a detected stop sign, avoid cutting in with a target vehicle, or any other navigation operation that may advance the navigation goal of the system).

[0349] For each of the determined potential navigation actions, the system can determine an expected reward. The expected reward can be determined according to any of the techniques described above and can include an analysis of the specific potential action relative to one or more reward functions. Expected rewards 1606, 1608, and 1610 can be determined for each of the potential navigation actions (e.g., the first, second, and Nth navigation actions) determined in steps 1605, 1607, and 1609, respectively.

[0350] In some cases, the host vehicle's navigation system may select from the available potential actions based on the values ​​associated with expected rewards 1606, 1608, and 1610 (or any other type of expected reward indicator). For example, in some cases, the action that yields the highest expected reward may be selected.

[0351] In other cases, particularly when the navigation system is involved in long-range planning to determine the navigation action of the host vehicle, the system may not select the potential action that produces the highest expected reward. Instead, the system can look into the future to analyze whether there is an opportunity to achieve a higher reward in the future if an action with a lower reward is selected in response to the current navigation state. For example, for any or all of the potential actions determined in steps 1605, 1607, and 1609, a future state can be determined. Each future state determined in steps 1613, 1615, and 1617 can represent a future navigation state that is expected to result based on the current navigation state modified by the corresponding potential action (e.g., the potential action determined in steps 1605, 1607, and 1609).

[0352] For each of the future states predicted in steps 1613, 1615, and 1617, one or more future actions (as navigation options available in response to the determined future state) may be determined and evaluated. For example, at steps 1619, 1621, and 1623, a value or any other type of metric (e.g., based on one or more reward functions) of an expected reward associated with the one or more future actions may be developed. The expected reward associated with the one or more future actions may be evaluated by comparing the value of the reward function associated with each future action or by comparing any other metric associated with the expected reward.

[0353] At step 1625, the navigation system of the host vehicle may select a navigation action for the host vehicle based on a comparison of expected rewards not only based on the potential actions identified relative to the current navigation state (e.g., at steps 1605, 1607, and 1609), but also based on the expected rewards determined as a result of the potential future actions available in response to the predicted future state (e.g., determined at steps 1613, 1615, and 1617). The selection at step 1625 may be based on the option and reward analysis performed at steps 1619, 1621, and 1623.

[0354] The selection of a navigation action at step 1625 may be based solely on a comparison of the expected rewards associated with future action options. In such cases, the navigation system may select an action for the current state based solely on a comparison of the expected rewards resulting from the action with the potential future navigation states. The system may select the potential action identified at steps 1605, 1607, or 1609 that is associated with the highest future reward value determined by the analysis at steps 1619, 1621, and 1623.

[0355] The selection of the navigation action at step 1625 may also be based solely on a comparison of the current action options (as described above). In this case, the navigation system may select the potential action associated with the highest expected reward 1606, 1608, or 1610 identified at steps 1605, 1607, or 1609. This selection may be performed with little or no consideration of future navigation states or future expected rewards for available navigation actions responsive to the expected future navigation states.

[0356] On the other hand, in some cases, the selection of the navigation action at step 1625 may be based on a comparison of the expected rewards associated with the future action options and the expected rewards associated with the current action options. In fact, this may be a navigation principle based on long-range planning. For example, the expected rewards for future actions may be analyzed to determine whether any rewards justify selecting a lower-reward action in response to the current navigation state in order to achieve a potentially higher reward in response to a subsequent navigation action that is expected to be available in response to the future navigation state. For example, the value of expected reward 1606 or other indicators may indicate the highest expected reward among rewards 1606, 1608, and 1610. On the other hand, expected reward 1608 may indicate the lowest expected reward among rewards 1606, 1608, and 1610. Analysis of the future state, potential future actions, and future rewards may be used in selecting the navigation action at step 1625, rather than simply selecting the potential action determined at step 1605 (i.e., the action that yields the highest expected reward 1606). In one example, it may be determined that the reward identified in step 1621 (at least one future action in response to the future state determined in step 1615 based on the second potential action determined in step 1607) may be higher than expected reward 1606. Based on this comparison, the second potential action determined in step 1607 may be selected over the first potential action determined in step 1605, despite expected reward 1606 being higher than expected reward 1608. In one example, the potential navigation actions determined in step 1605 may include a merge in front of a detected target vehicle, while the potential navigation actions determined in step 1607 may include a merge behind the target vehicle. While expected reward 1606 for a merge in front of the target vehicle may be higher than expected reward 1608 associated with a merge behind the target vehicle, it may be determined that a merge behind the target vehicle may result in a future state for which action options may exist that, based on available actions responsive to the currently sensed navigation state, yield potential rewards that are even higher than expected rewards 1606, 1608, and other rewards.

[0357] The selection from among the potential actions at step 1625 can be based on any suitable comparison of expected rewards (or any other metric or benefit indicator associated with one potential action compared to other potential actions). In some cases, as described above, if the second potential action is projected to provide at least one future action associated with an expected reward that is higher than the reward associated with the first potential action, then the second potential action can be selected instead of the first potential action. In other cases, more complex comparisons can be employed. For example, a reward associated with an action option responsive to a projected future state can be compared to more than one expected reward associated with a determined potential action.

[0358] In some scenarios, if at least one future action is expected to produce a reward that is higher than any of the expected rewards produced by potential actions for the current state (e.g., expected rewards 1606, 1608, 1610, etc.), then the action and expected reward based on the predicted future state may influence the selection of the potential action for the current state. In some cases, the future action option that produces the highest expected reward (e.g., from the expected rewards associated with the potential actions for the sensed current state and from the expected rewards associated with the potential future action options relative to the potential future navigation state) may be used as a guide for selecting the potential action for the current navigation state. That is, after identifying the future action option that produces the highest expected reward (or a reward above a predetermined threshold, etc.), a potential action may be selected at step 1625 that may result in the future state associated with the identified future action that produces the highest expected reward.

[0359] In other cases, the selection of available actions may be made based on the determined difference between the expected rewards. For example, if the difference between the expected reward associated with the future action determined at step 1621 and the expected reward 1606 is greater than the difference between the expected reward 1608 and the expected reward 1606 (assuming a + difference), then the second potential action determined at step 1607 may be selected. In another example, if the difference between the expected reward associated with the future action determined at step 1621 and the expected reward associated with the future action determined at step 1619 is greater than the difference between the expected reward 1608 and the expected reward 1606, then the second potential action determined at step 1607 may be selected.

[0360] Several examples of selecting from potential actions for the current navigation state have been described. However, any other suitable comparison technique or criteria may be used to select available actions through long-range planning based on action and reward analysis that extends to projected future states. In addition, although Figure 16While two layers of analysis are shown for long-range planning (e.g., a first layer considering rewards arising from potential actions for the current state, and a second layer considering rewards arising from future action options in response to projected future states), analysis based on more layers is possible. For example, rather than basing long-range planning on one or two layers, three, four, or more layers of analysis may be used to select from available potential actions in response to the current navigation state.

[0361] After selecting from the potential actions responsive to the sensed navigational state at step 1627, the at least one processor may cause at least one adjustment of a navigation actuator of the host vehicle in response to the selected potential navigation action. The navigation actuator may include any suitable device for controlling at least one aspect of the host vehicle. For example, the navigation actuator may include at least one of a steering mechanism, a brake, or an accelerator.

[0362] Navigation based on inferred aggressiveness of the other party

[0363] The target vehicle can be monitored by analyzing the collected image stream to determine indicators of driving aggressiveness. Aggression is described in this article as a qualitative or quantitative parameter, but other features can also be used: perceived attention level (driver's potential impairment, distraction - mobile phone, falling asleep, etc.). In some cases, the target vehicle may be considered to have a defensive posture, and in some cases, the target vehicle may be determined to have a more aggressive posture. Navigation actions can be selected or developed based on indicators of aggressiveness. For example, in some cases, the relative speed, relative acceleration, relative acceleration increase, following distance, etc. relative to the host vehicle can be tracked to determine whether the target vehicle is aggressive or defensive. For example, if it is determined that the target vehicle has an aggressiveness exceeding a threshold, the host vehicle may tend to give way to the target vehicle. The aggressiveness of the target vehicle can also be identified based on the determined behavior of the target vehicle relative to the target vehicle path or one or more nearby obstacles (e.g., leading vehicles, obstacles in the road, traffic lights, etc.).

[0364] As an introduction to this concept, an example experiment will be described regarding a host vehicle merging into a roundabout, where the navigation goal is to pass through and exit the roundabout. This scenario may begin with the host vehicle gradually approaching the entrance to the roundabout and may end when the host vehicle reaches the exit of the roundabout (e.g., the second exit). Success can be measured based on whether the host vehicle always maintains a safe distance from all other vehicles, whether the host vehicle completes the route as quickly as possible, and whether the host vehicle adheres to a smooth acceleration strategy. In this description, N TTarget vehicles can be randomly placed on the roundabout. To model a mixture of adversarial and typical behavior, with probability p, the target vehicle can be modeled with an "aggressive" driving strategy such that when the host vehicle attempts to merge in front of the target vehicle, the aggressive target vehicle accelerates. With probability 1-p, the target vehicle can be modeled with a "defensive" driving strategy such that the target vehicle slows down and lets the host vehicle merge. In this experiment, p = 0.5, and the host vehicle's navigation system may not be provided with information about the other driver's type. The other driver's type can be randomly selected at the beginning of this event.

[0365] The navigation state can be represented as the speed and position of the main vehicle (agent), and the position, speed and acceleration of the target vehicles. In order to distinguish between aggressive and defensive drivers based on the current state, it may be important to keep an observation of the target acceleration. All target vehicles can move on a one-dimensional curve that represents the path of the roundabout. The main vehicle can move on its own one-dimensional curve, which intersects the curve of the target vehicle at the merge point, which is the origin of the two curves. In order to model reasonable driving, the absolute value of the acceleration of all vehicles may be bounded by a constant. The speed may also be passed through a ReLU, since backward driving is not allowed. Note that since backward driving is not allowed, long-term planning may become necessary because the agent cannot regret its past actions.

[0366] As described above, the next state s t+1 Can be broken down into predictable parts and the unpredictable part ν t The sum of . can represent the dynamics of the vehicle position and velocity (which can be well defined in a differentiable way), while ν t It can be shown that the acceleration of the target vehicle is It can be expressed as a combination of ReLU functions on affine transformations, so it is relative to s t and a t is differentiable. The vector ν t can be defined by the simulator in a non-differentiable way and can implement aggressive behavior towards some goals and defensive behavior towards others. Figure 17A and Figure 17B Two frames from such a simulator are shown in . In this example experiment, the host vehicle 1701 learns to slow down when approaching the entrance to the roundabout. It also learns to give way to aggressive vehicles (e.g., vehicles 1703 and 1705) and to continue safely when merging in front of defensive vehicles (e.g., vehicles 1706, 1708, and 1710). Figure 17A and Figure 17B In the example shown, the type of target vehicle is not provided to the navigation system of the host vehicle 1701. Instead, the determination of whether a particular vehicle is an attacking vehicle or a defensive vehicle is made through inference based on the observed position and acceleration (e.g., of the target vehicle). Figure 17A In , based on position, speed, and / or relative acceleration, host vehicle 1701 may determine that vehicle 1703 has aggressive tendencies, and therefore, host vehicle 1701 may stop and wait for target vehicle 1703 to pass, rather than attempting to merge in front of target vehicle 1703. However, in Figure 17B , target vehicle 1701 recognizes that target vehicle 1710 traveling behind vehicle 1703 is exhibiting a defensive tendency (again, based on the observed position, speed, and / or relative acceleration of vehicle 1710), and therefore, completes a successful merge in front of target vehicle 1710 and behind target vehicle 1703.

[0367] Figure 18 A flow chart is provided that represents an example algorithm for navigating a host vehicle based on the predicted aggressiveness of other vehicles. Figure 18 In an example, a level of aggressiveness associated with at least one target vehicle can be inferred based on the observed behavior of the target vehicle relative to objects in the target vehicle's environment. For example, at step 1801, at least one processing device of a host vehicle navigation system (e.g., processing device 110) can receive a plurality of images representing the host vehicle's environment from a camera associated with the host vehicle. At step 1803, analysis of one or more of the received images can enable the at least one processor to identify a target vehicle (e.g., vehicle 1703) in the environment of host vehicle 1701. At step 1805, analysis of one or more of the received images can enable the at least one processing device to identify at least one obstacle to the target vehicle in the host vehicle's environment. The objects may include gravel on the road, stop lights / traffic lights, pedestrians, other vehicles (e.g., vehicles traveling in front of the target vehicle, parked vehicles, etc.), boxes in the road, roadblocks, curbs, or any other type of object that may be encountered in the host vehicle's environment. At step 1807 , analysis of one or more of the received images may enable the at least one processing device to determine at least one navigational characteristic of the target vehicle relative to the at least one identified obstacle to the target vehicle.

[0368] Various navigational characteristics can be used to infer the aggressiveness level of a detected target vehicle in order to develop an appropriate navigational response to the target vehicle. For example, such navigational characteristics can include the relative acceleration between the target vehicle and at least one identified obstacle, the distance between the target vehicle and the obstacle (e.g., the following distance of the target vehicle behind another vehicle), and / or the relative speed between the target vehicle and the obstacle.

[0369] In some embodiments, the navigation characteristics of the target vehicle can be determined based on the output of sensors associated with the host vehicle (e.g., radio detection and ranging, speed sensors, GPS, etc.). However, in some cases, the navigation characteristics of the target vehicle can be determined partially or entirely based on an analysis of images in the host vehicle environment. For example, the image analysis techniques described above and the image analysis techniques described in, for example, U.S. Patent No. 9,168,868, incorporated herein by reference, can be used to identify the target vehicle in the host vehicle's environment. Furthermore, monitoring the position of the target vehicle in the captured images over time and / or monitoring the position of one or more features associated with the target vehicle (e.g., taillights, headlights, bumpers, wheels, etc.) in the captured images can enable determination of the relative distance, speed, and / or acceleration between the target vehicle and the host vehicle or between the target vehicle and one or more other objects in the host vehicle environment.

[0370] The aggressiveness level of an identified target vehicle can be inferred from any suitable observed navigational characteristic of the target vehicle, or any combination of observed navigational characteristics. For example, the determination of aggressiveness can be made based on any observed characteristic and one or more predetermined threshold levels, or any other suitable qualitative or quantitative analysis. In some embodiments, if a target vehicle is observed following a host vehicle or another vehicle at a distance less than a predetermined aggressive distance threshold, the target vehicle may be considered an aggressive vehicle. On the other hand, if a target vehicle is observed following a host vehicle or another vehicle at a distance greater than a predetermined defensive distance threshold, the target vehicle may be considered a defensive vehicle. The predetermined offensive distance threshold need not be the same as the predetermined defensive distance threshold. Additionally, one or both of the predetermined offensive distance threshold and the predetermined defensive distance threshold may include a range of values ​​rather than a bright line value. Furthermore, neither the predetermined offensive distance threshold nor the predetermined defensive distance threshold need be fixed. Rather, these values ​​or ranges of values ​​may change over time, and different thresholds / threshold ranges may be applied based on the observed characteristics of the target vehicle. For example, the applied threshold may depend on one or more other characteristics of the target vehicle. Higher observed relative speeds and / or accelerations may warrant the application of larger thresholds / ranges. Conversely, lower relative speeds and / or accelerations (including zero relative speed and / or acceleration) may warrant applying a smaller distance threshold / range when making offensive / defensive inferences.

[0371] Aggressive / defensive inferences may also be based on relative speed and / or relative acceleration thresholds. If the observed relative speed and / or relative acceleration of a target vehicle relative to another vehicle exceeds a predetermined level or range, the target vehicle may be considered aggressive. If the observed relative speed and / or relative acceleration of a target vehicle relative to another vehicle is below a predetermined level or range, the target vehicle may be considered defensive.

[0372] While the determination of aggressiveness / defensiveness may be based solely on any observed navigational characteristic, the determination may also depend on any combination of observed characteristics. For example, as mentioned above, in some cases, a target vehicle may be deemed aggressive based solely on the observation that the target vehicle is following another vehicle at a distance below a certain threshold or range. However, in other cases, the target vehicle may be deemed aggressive if it is following the other vehicle by less than a predetermined amount (which may be the same or different threshold applied when determining based solely on distance) and has a relative speed and / or relative acceleration greater than a predetermined amount or range. Similarly, a target vehicle may be deemed defensive based solely on the observation that the target vehicle is following the other vehicle at a distance greater than a certain threshold or range. However, in other cases, the target vehicle may be deemed defensive if it is following the other vehicle by less than a predetermined amount (which may be the same or different threshold applied when determining based solely on distance) and has a relative speed and / or relative acceleration less than a predetermined amount or range. For example, system 100 may attack / defend if the vehicle accelerates or decelerates at a rate exceeding 0.5G (e.g., a jerk of 5m / s3), the vehicle experiences a lateral acceleration of 0.5G while changing lanes or on a curve, the vehicle causes another vehicle to perform any of the above actions, the vehicle changes lanes and causes another vehicle to yield with a deceleration exceeding 0.3G or a jerk of 3m / s3, and / or the vehicle changes lanes twice without stopping.

[0373] It should be understood that a reference to a quantity exceeding a certain range may indicate that the quantity exceeds all values ​​associated with the range or within the range. Similarly, a reference to a quantity below a certain range may indicate that the quantity is lower than all values ​​associated with the range or within the range. Additionally, although the examples described for making offensive / defensive inferences are described with respect to distance, relative acceleration, and relative speed, any other suitable quantity may be used. For example, a collision time calculation or any indirect indicator of the distance, acceleration, and / or speed of the target vehicle may be used. It should also be noted that although the examples above focus on the situation of the target vehicle relative to other vehicles, offensive / defensive inferences may be made by observing the navigation characteristics of the target vehicle relative to any other type of obstacle (e.g., pedestrians, roadblocks, traffic lights, debris, etc.).

[0374] return Figure 17A and Figure 17BIn the example shown in , when host vehicle 1701 approaches a roundabout, a navigation system (including at least one processing device thereof) may receive an image stream from a camera associated with the host vehicle. Based on analysis of one or more of the received images, any one of target vehicle 1703, target vehicle 1705, target vehicle 1706, target vehicle 1708, and target vehicle 1710 may be identified. Further, the navigation system may analyze navigational features of one or more of the identified target vehicles. The navigation system may identify that the gap between target vehicle 1703 and target vehicle 1705 represents the first opportunity to merge into the roundabout. The navigation system may analyze target vehicle 1703 to determine an aggressiveness indicator associated with target vehicle 1703. If target vehicle 1703 is deemed aggressive, the host vehicle navigation system may choose to yield to vehicle 1703 rather than merging in front of vehicle 1703. On the other hand, if the target vehicle 1703 is deemed a defensive vehicle, the host vehicle navigation system may attempt to complete a merging maneuver in front of the vehicle 1703 .

[0375] As host vehicle 1701 approaches the roundabout, at least one processing device of the navigation system may analyze the captured images to determine navigational characteristics associated with target vehicle 1703. For example, based on these images, it may be determined that vehicle 1703 is following vehicle 1705 at a distance that provides sufficient clearance for host vehicle 1701 to safely enter. In fact, it may be determined that vehicle 1703 is following vehicle 1705 at a distance exceeding an aggressive distance threshold, and therefore, based on this information, the host vehicle navigation system may be inclined to identify target vehicle 1703 as defensive. However, in some cases, as discussed above, more than one navigational characteristic of the target vehicle may be analyzed when making an aggressive / defensive determination. To further analyze this, the host vehicle navigation system may determine that when target vehicle 1703 is following at a non-aggressive distance behind target vehicle 1705, the relative speed and / or relative acceleration of vehicle 1703 relative to target vehicle 1705 exceeds one or more thresholds associated with aggressive behavior. In fact, host vehicle 1701 can determine that target vehicle 1703 is accelerating relative to vehicle 1705 and closing the gap between them. Based on further analysis of relative speed, acceleration, and distance (and even the rate at which the gap between vehicle 1703 and vehicle 1705 is closing), host vehicle 1701 can determine that target vehicle 1703 is behaving aggressively. Thus, while there may be sufficient gap within which the host vehicle can safely navigate, host vehicle 1701 can anticipate that a merge in front of target vehicle 1703 will result in the vehicle directly behind the host vehicle being navigated aggressively. Furthermore, based on observed behavior through image analysis or other sensor output, if host vehicle 1701 merges in front of vehicle 1703, it can be expected that target vehicle 1703 will continue to accelerate toward host vehicle 1701, or will continue to accelerate toward host vehicle 1701 at a non-zero relative speed. This situation may be undesirable from a safety perspective and may also cause discomfort to the passengers of the host vehicle. Figure 17B As shown, host vehicle 1701 may choose to yield to vehicle 1703 and merge into the roundabout behind vehicle 1703 and in front of vehicle 1710, which is deemed defensive based on an analysis of one or more of its navigational features.

[0376] Back to Figure 18At step 1809, at least one processing device of the host vehicle's navigation system may determine a navigation action for the host vehicle (e.g., merging in front of vehicle 1710 and behind vehicle 1703) based on the identified at least one navigation characteristic of the target vehicle relative to the identified obstacle. To implement the navigation action (at step 1811), the at least one processing device may cause at least one adjustment of a navigation actuator of the host vehicle in response to the determined navigation action. For example, at Figure 17A , braking may be applied to yield to the vehicle 1703, and, as Figure 17B As shown, the accelerator may be applied along with steering of the host vehicle's wheels to cause the host vehicle to enter the roundabout behind vehicle 1703 .

[0377] As described in the examples above, the navigation of the host vehicle can be based on the navigation characteristics of the target vehicle relative to another vehicle or object. Additionally, the navigation of the host vehicle can be based only on the navigation characteristics of the target vehicle without specific reference to another vehicle or object. For example, in Figure 18 In step 1807, analysis of the plurality of images captured from the host vehicle's environment may enable determination of at least one navigational characteristic of the identified target vehicle indicative of a level of aggressiveness associated with the target vehicle. The navigational characteristic may include speed, acceleration, and the like, without requiring reference to other objects or target vehicles for making an aggressive / defensive determination. For example, an observed acceleration and / or speed associated with the target vehicle exceeding a predetermined threshold or falling within or exceeding a numerical range may indicate aggressive behavior. Conversely, an observed acceleration and / or speed associated with the target vehicle falling below a predetermined threshold or falling within or exceeding a numerical range may indicate defensive behavior.

[0378] Of course, in some cases, observed navigational characteristics (e.g., position, distance, acceleration, etc.) can be referenced relative to the host vehicle for purposes of making an aggressiveness / defensiveness determination. For example, observed navigational characteristics of a target vehicle that indicate a level of aggressiveness associated with the target vehicle may include an increase in relative acceleration between the target vehicle and the host vehicle, a following distance of the target vehicle behind the host vehicle, a relative speed between the target vehicle and the host vehicle, etc.

[0379] Navigation based on accident responsibility constraints

[0380] As described in the above sections, planned navigation actions can be tested against predetermined constraints to ensure compliance with certain rules. In some embodiments, this concept can be extended to considerations of potential accident liability. As discussed below, a primary goal of autonomous navigation is safety. Since absolute safety may not be possible (e.g., at least because a particular host vehicle under autonomous control cannot control other vehicles around it - it can only control its own actions), using potential accident liability as a consideration in autonomous navigation and, in fact, as a constraint on planned actions, may help ensure that a particular autonomous vehicle does not take any actions that are considered unsafe, e.g., actions that might attribute potential accident liability to the host vehicle. A desired level of accident avoidance (e.g., fewer than 10 km / h of driving) can be achieved if the host vehicle only takes actions that are safe and determined not to result in an accident for which the host vehicle is itself at fault or responsible. -9 Second-rate).

[0381] Challenges posed by most current approaches to autonomous driving include a lack of safety guarantees (or at least an inability to provide the desired level of safety), and a lack of scalability. Consider the problem of ensuring safe driving for multiple agents. Since society is unlikely to tolerate road fatalities caused by machines, an acceptable level of safety is crucial for the acceptance of autonomous vehicles. While the goal may be to provide zero accidents, this may not be possible because an accident typically involves multiple agents, and one might imagine situations where an accident occurs only due to the fault of other agents. For example, Figure 19 As shown, host vehicle 1901 is traveling on a multi-lane highway. While host vehicle 1901 can control its own movements relative to target vehicles 1903, 1905, 1907, and 1909, it cannot control the movements of the target vehicles around it. Therefore, if, for example, vehicle 1905 suddenly cuts into the host vehicle's lane while on a collision course with the host vehicle, host vehicle 1901 may be unable to avoid an accident with at least one of the target vehicles. To address this challenge, a typical response among autonomous vehicle practitioners is to adopt a statistically driven approach, in which safety validation becomes increasingly rigorous as the amount of mileage collected increases.

[0382] To understand the problematic nature of data-driven safety approaches, however, first consider that the probability of a fatality caused by an accident per hour of driving is known to be 10 -6 It is reasonable to assume that if society accepts machines replacing humans in the task of driving, the death rate should decrease by three orders of magnitude, or 10 deaths per hour. -9 The estimated values ​​are similar to the assumed mortality rates for airbags and aviation standards. For example, 10-9 is the probability that the wing will detach from the aircraft in mid-air. However, attempts to guarantee safety using data-driven statistical methods that provide additional confidence in the cumulative miles flown are unrealistic. -9 The amount of data required to calculate the probability of death is its reciprocal (i.e. 10 9 hours of data), which is about 30 billion miles. In addition, multi-agent systems interact with their environment and may not be able to be verified offline (unless there is a realistic simulator that simulates real human driving, which has richness and complexity, such as reckless driving, but the problem of validating a simulator is even harder than creating a safe autonomous vehicle agent). And any change to the planning and control software will require the same amount of data collection, which is obviously intractable and impractical. Furthermore, developing systems through data inevitably suffers from the problem of lack of explainability and interpretability of the actions taken - if an autonomous vehicle (AV) is involved in an accident resulting in a fatality, then we need to know why. Therefore, a model-based safety approach is needed, but the existing "functional safety" and ASIL requirements in the automotive industry are not designed to deal with multi-agent environments.

[0383] The second major challenge in developing safe driving models for autonomous vehicles is the need for scalability. The fundamental premise of AVs goes beyond "building a better world" and is based on the idea that mobility without a driver can be cheaper than mobility with a driver. This premise is inevitably coupled with the concept of scalability—in terms of supporting mass production of AVs (in the millions), and more importantly, in terms of enabling driving in new cities with negligible incremental cost. So, while the cost of computing and sensing is indeed important, the cost of validation and the ability to drive "everywhere" rather than in a select few cities are also necessary to sustain a business if AVs are to be mass-produced.

[0384] The problem with most current approaches lies in a “brute force” mentality across three axes: (i) the required “computational density”; (ii) the way HD maps are defined and created; and (iii) the required specifications for sensors. Brute force approaches work against scalability and shift the focus to a future where unlimited onboard compute is ubiquitous, the cost of building and maintaining HD maps becomes negligible and scalable, and exotic, ultra-advanced sensors can be developed and produced to automotive grade at negligible cost. A future where any one of these is achieved is indeed plausible, but a future where all of them are achieved is likely a low probability event. Therefore, there is a need for a formal model that integrates safety and scalability into an AV program that is socially acceptable and scalable in the sense of supporting millions of vehicles driving anywhere in the developed world.

[0385] The disclosed embodiments represent a solution that can provide the target safety level (and possibly even exceed the safety goal) and can also be scaled to systems including millions (or more) of autonomous vehicles. On the safety side, a model called "Responsibility Sensitive Safety" (RSS) is introduced that formalizes the concept of "accident responsibility", is interpretable and explainable, and embeds a sense of "responsibility" into the actions of robotic agents. The definition of RSS is agnostic to how it is implemented - a key feature that facilitates the goal of creating a convincing global safety model. RSS is derived from observations (e.g. Figure 19 (as shown) agents play asymmetric roles in an accident, where typically only one of the agents is responsible for the accident and is therefore liable for it. The RSS model also includes a formal treatment of "careful driving" under limited sensing conditions, where not all agents are always visible (e.g., due to occlusion). A major goal of the RSS model is to guarantee that an agent never causes an accident for which it is "at fault" or for which it is responsible. The model can only be useful if it has an effective policy that complies with RSS (e.g., a function that maps "sensory states" to actions). For example, actions that seem innocent in the present may lead to catastrophic events in the distant future (a "butterfly effect"). RSS may be useful to construct a set of local constraints for the short-term future that guarantee (or at least virtually guarantee) that no accidents will occur in the future due to the actions of the host vehicle.

[0386] Another contribution revolves around the development of a "semantic" language consisting of units, measurements, and action spaces, and a description of how these fit into the AV's planning, sensing, and actuation. To get a sense of semantics, in this case, the instructions consider how humans take lessons from driving and think about "driving policies." These instructions are not geometric—they are not "drive 13.7 meters at your current speed, then drive at 0.8 m / s." 2 Instead, these instructions are of a semantic nature—“follow the car in front of you” or “pass the car on your left.” The typical language of human driving strategies is in terms of longitudinal and lateral targets, not geometric units of acceleration vectors. Formal semantic languages ​​can be useful in several ways, related to the computational complexity of planning (not growing exponentially with time and the number of agents) and how safety and comfort interact, and in how sensor computations are defined, the specification of sensor modalities, and how they interact in fusion methods. Fusion methods (based on semantic languages) can ensure that the RSS model achieves the required 10 times per hour driven. -9 The probability of death, while only 10 5 Offline validation is performed on a dataset of hours of driving data.

[0387] For example, in a reinforcement learning setting, one can define a Q-function on a semantic space (e.g., a function that evaluates the long-term quality of performing an action a∈A when the agent is in state s∈S; given such a Q-function, a natural choice of action might be to choose the one with the highest quality, π(s) = argmax a Q(s,a)), in which the number of trajectories to examine at any given time is bounded by 104, regardless of the time horizon used for planning. The signal-to-noise ratio in this space can be high, enabling effective machine learning methods to successfully model the Q-function. In the context of sensory computing, semantics can allow for distinguishing between errors that affect safety and errors that affect driving comfort. We define a PAC model (Probably Approximately Correct (PAC)), borrowing Valiant's PAC learning terminology, for sensing that is associated with the Q-function and shows how measurement errors can be incorporated into planning in a manner consistent with RSS while allowing for optimized driving comfort. The semantic language may be important for the success of certain aspects of this model, as standard measures of other errors, such as those in the global coordinate system, may not conform to the PAC sensing model. Furthermore, the semantic language may be an important enabling factor in defining HD maps, which can be constructed using low-bandwidth sensory data and thus can be crowdsourced and support scalability.

[0388] In summary, the disclosed embodiments may include a formal model covering the key components of an AV: sensing, planning, and action. This model helps ensure that, from a planning perspective, no accidents occur that are the AV's fault. Furthermore, through the PAC sensing model, even in the presence of sensing errors, the described fusion approach may only require offline collection of a very reasonable amount of data to comply with the described safety model. Furthermore, the model can link safety and scalability together through a semantic language, thereby providing a complete approach for safe and scalable AVs. Finally, it is worth noting that developing a safety model that is accepted by industry and regulators may be a necessary condition for the success of AVs.

[0389] The RSS model can generally follow the classic sense-plan-act robotic control approach. The sensing system may be responsible for understanding the current state of the host vehicle environment. The planning part, which may be called a "driving policy" and may be implemented by a set of hard-coded instructions, a trained system (e.g., a neural network), or a combination, may be responsible for determining the optimal action for the next step based on the available options for achieving the driving goal (e.g., how to move from the left lane to the right lane to exit the highway). The action part is responsible for implementing the plan (e.g., a system of actuators and one or more controllers for steering, accelerating and / or braking the vehicle to achieve the selected navigation action). The embodiments described below focus primarily on the sensing and planning parts.

[0390] Accidents can result from sensing errors or planning errors. Planning is a multi-agent endeavor because there are other road users (humans and machines) that react to the AV's actions. The RSS model described aims to address the safety of the planning part. This can be called multi-agent safety. In a statistical approach, the estimation of the probability of planning errors can be done "online". That is, after each update of the software, the new version would have to drive billions of miles to provide an acceptable level of estimate of the frequency of planning errors. This is obviously not feasible. As an alternative, the RSS model can provide 100% guarantee (or almost 100% guarantee) that the planning module will not make errors that are the AV's responsibility (formally defining the concept of "liability"). The RSS model can also provide an efficient method whose verification does not rely on online testing.

[0391] Errors in the sensing system may be easier to verify because sensing can be independent of vehicle motion, and thus we can use “offline” data to verify the probability of serious sensing errors. However, even if more than 10 9 Offline driving data for hours of driving is also challenging.As part of the description of the disclosed sensing system, a fusion method is described that can be validated using significantly smaller amounts of data.

[0392] The described RSS system can also scale to millions of cars. For example, the described semantic driving policy and applied safety constraints may meet sensing and mapping requirements that are scalable to millions of cars even with today's technology.

[0393] The fundamental building block for such systems is a comprehensive safety definition—the minimum standards to which AV systems may be expected to adhere. In the following technical lemma, statistical methods for verifying AV systems are shown to be infeasible, even for simple statements such as "the system experiences N accidents per hour." This means that a model-based safety definition is the only viable tool for verifying AV systems.

[0394] Lemma 1 Let X be a probability space, and A be an event with Pr(A)=p1<0.1. Suppose we sample from X independent and identically distributed samples, and assume but

[0395] Pr(z=0)≥e -2

[0396] Prove that we use the inequality 1-x ≥ e -2x (Proof of completeness in Appendix A.1) to obtain

[0397]

[0398] Corollary 1: Assume that AV system AV1 has a small but insufficient probability p1 of being involved in an accident. Any deterministic verification procedure, given a sample size of 1 / p1, will, with constant probability, fail to distinguish between AV1 and a different AV system AV0 that is not involved in an accident.

[0399] To get an idea of ​​typical values ​​for such probabilities, suppose we expect an accident probability of 10- 9 , and a certain AV system only provides 10- 8 Probability. Even if the system gets 10 8 hours of driving time, there is a constant probability that the validation process will fail to indicate that the system is dangerous.

[0400] Finally, note that this difficulty applies to validating a single, specific, and dangerous AV system. A complete solution cannot be considered a single system, as new versions, bug fixes, and updates will be necessary. From the verifier's perspective, every change, even a single line of code, creates a new system. Therefore, a statistically validated solution must be statistically validated online on new samples after each small correction or change to account for changes in the distribution of observed and achieved states of the new system. It is infeasible to repeatedly and systematically acquire such a large number of samples (and even then, fail to validate the system with constant probability).

[0401] Furthermore, any statistical claim must be formalized to be measurable. Claiming that a system has a statistical property of the number of accidents it has caused is much weaker than claiming that it drives in a safe manner. To make this point, one must formally define what safety is.

[0402] Absolute security is impossible

[0403] The action taken by car c may be considered absolutely safe if no accident occurs at some future time after the action. It can be seen that by observing simple driving scenarios (e.g. Figure 19 Absolute safety is not possible (as depicted). From the perspective of vehicle 1901, no action can ensure that none of the surrounding cars will hit it. It is also impossible to solve this problem by prohibiting autonomous vehicles from being in such situations. Since every road with more than two lanes will lead to this situation at some point in time, prohibiting such a scenario is equivalent to asking to stay in the garage. This possible outcome may seem disappointing at first glance. Nothing is absolutely safe. However, as defined above, such a requirement for absolute safety may be too demanding, as evidenced by the fact that human drivers do not comply with absolute safety requirements. Instead, humans behave in accordance with a safety concept that depends on responsibility.

[0404] Responsibility-sensitive safety

[0405] An important aspect missing from the concept of absolute safety is the asymmetric nature of most accidents – usually one of the drivers is responsible and should be blamed. Figure 19 In this example, if left car 1909 suddenly drives into center car 1901, center car 1901 is not to blame. To formalize the fact that AV 1901 lacks responsibility, AV 1901's behavior in its own lane can be considered safe. To do this, we describe a formal concept of "accident attribution," or accident responsibility, which can serve as a prerequisite for a safe driving approach.

[0406] As an example, consider two cars cf, c r On a straight road, one car is driving behind the other on the same road. Suppose the car in front is c f Because of an obstacle on the road, the car brakes suddenly and tries to avoid the obstacle. Unfortunately, r No with c f Keep enough distance to not be able to respond in time and hit the c f Obviously, the blame is on c r It is the responsibility of the vehicle behind to maintain a safe distance from the vehicle ahead and to be prepared for unexpected but reasonable braking.

[0407] Next, consider a broader family of scenarios: driving on a multi-lane road, where cars can freely change lanes, cut into the paths of other cars, travel at varying speeds, and so on. To simplify the discussion below, assume a straight road on a flat surface, with the lateral and longitudinal axes being the x-axis and y-axis, respectively. This can be achieved under mild conditions by defining a homomorphism between actual curved and straight roads. Additionally, consider discrete time space. This definition may help distinguish between two intuitively distinct groups of cases: simple cases, in which no significant lateral maneuvers are performed, and more complex cases, involving lateral maneuvers.

[0408] Definition 1 (Car Corridor) The range of the corridor for car c is [c x，left , c x，ight ]×[±∞], where c x，left 、c x，right It is the position of the leftmost corner and the rightmost corner of c.

[0409] Definition 2 (cut-in) If car c1 ( Figure 20A and Figure 20B The car in 2003) does not intersect the corridor of c0 at time t-1, and intersects the corridor of c0 at time t, then the car c1 ( Figure 20A Car 2001 in Figure 20B enters corridor c0 at time t.

[0410] A further distinction can be made between the front and rear parts of a corridor. The term "cut-in direction" can describe movement in the direction of the relevant road boundary. These definitions can be used to define situations where there is no lateral movement. For simple cases where such a situation does not exist, such as when one car is following another, a safe longitudinal distance is defined:

[0411] Definition 3 (Safe Longitudinal Distance) Car c r (Car 2103) and in c r Another car in the front corridor c f (Car 2105) longitudinal distance 2101 ( Figure 21 ) is safe with respect to the response time p if c f Execute any braking command a(|a| max,brake ), if c r will apply c starting at time p r Maximum braking until complete stop, then c r Will not be with c f collision.

[0412] Lemma 2 below computes d as c r 、c f speed, response time ρ and maximum acceleration a max,brake function of ρ and a max,brake Both are constants and should be adjusted to some reasonable values.

[0413] Lemma 2 Let c r The vertical axis is located at c f The following means of transport. Let a max,brake ,a max,accel is the maximum braking and acceleration command, let ρ be c r The response time of r 、υ f is the longitudinal velocity of the car, let l f , l r is their length. Define υ p,max =υ r +ρ·a max,accel , and define as well as Let L = (l r +l f ) / 2. Then, c r The minimum safe longitudinal distance is:

[0414]

[0415] Proof: Let d t is the distance at time t. To prevent accidents, we must make d​t >L. To construct d min , we need to find the tightest lower bound required on d0. Clearly, d0 must be at least L. As long as the two cars have not stopped after T ≥ ρ seconds, the speed of the previous car will be υ f –T a max,brake , while that of c r will be bounded above by υ ρ,max –(T–ρ)a max,accel . Thus, the lower bound on the distance between the cars after T seconds is:

[0416]

[0417] Note that T r is the time for c r to reach a complete stop (speed = 0), and T f is the time for the other car to reach a complete stop. Note that a max,brake (T r –T f ) = υ ρ,max –υ f +ρa max,brake , so if T r ≤T f , it is sufficient to require d0 > L. If T r >T f , then

[0418]

[0419] it is required that d Tr > L and rearrange the terms to arrive at the proof conclusion.

[0420] Finally, a comparison operator is defined that allows comparison with some concept of "margin": when comparing lengths, speeds, etc., it is necessary to consider very similar quantities as "equal".

[0421] Definition 4 (μ-comparison) The μ-comparison of two numbers a, b is a > μ b if a > b + μ, a < μ b if a < b – μ, and a = μ b if |a – b| ≤ μ.

[0422] The comparisons below (argmin, argmax, etc.) are μ-comparisons for some appropriate μs. Suppose an accident occurs between cars c1 and c2. To consider who should be blamed for the accident, relevant moments to be checked are defined. This is a certain point in time before the accident, intuitively the "point of no return"; after this point, nothing can be done to prevent the accident from occurring.

[0423] Definition 5 (Attributable Time) The attributable time of an accident is the earliest time before the accident, where:

[0424] There are intersections between the corridor of one car in the car and the corridors of other cars in the car, and

[0425] The longitudinal distance is unsafe.

[0426] Obviously, such moments exist, because at the time of the accident, both conditions are met. The time of attribution can be divided into two categories:

[0427] There also occur attributable times of cut-ins, ie these are the first moments at which the corridor of one vehicle intersects with the corridor of another vehicle, with the intersection point being at a non-safe distance.

[0428] The attribution time where no cut-in occurs, i.e., the corridor already has an intersection at a safe longitudinal distance and the distance has changed to unsafe at the attribution time.

[0429] Definition 6 (μ-defeat at lateral velocity): Assume that the cut-in occurs between cars c1 and c2. If the lateral velocity of c1 relative to the cut-in direction is higher than the lateral velocity of c2 relative to the cut-in direction by μ, we say that c1 is defeated at a lateral velocity μ-defeat.

[0430] It should be noted that the direction of the velocity is important: for example, a velocity of -1,1 (two cars colliding) is a draw, but if the velocity is 1,1+μ / 2, then the car driving in the positive direction towards the other car is blamed. Intuitively, this definition would allow us to blame a car that drives very quickly sideways into another car.

[0431] Definition 7 (Winning by Lateral Position (μ1, μ2)): Assume that a cut-in occurs between cars c1 and c2. A cut-in occurs between cars c1 and c2 if c1's lateral position relative to the center of the cut-in lane (the center of the corridor closest to the cut-in) is less than μ1 (in absolute value) and less than c2's lateral position relative to the center of the cut-in lane by μ2.

[0432] Intuitively, if a car is very close to the lane center (μ1) and closer than another car (μ2), then this car will not be blamed.

[0433] Definition 8 (Attribution) The attribution or liability for an accident between cars c1 and c2 is a function of the state at the time of attribution and is defined as follows:

[0434] If the attributable time is not the time of cut-in, the blame lies with the car behind.

[0435] If the attributable time is also the cut-in time, then both cars are attributable, unless, for one of the cars, without loss of generality, c1, for some predefined μs, holds under the following two conditions:

[0436] It will not lose at lateral speed,

[0437] It wins with a sideways position.

[0438] In this case, c1 is excused. In other words, if an unsafe cut occurs, both cars should be blamed unless one of the two cars is not (significantly) faster laterally and that car is (significantly) close to the center of the lane. This way, the desired behavior is captured: if following a car, maintain a safe distance; and if the cut is simply into the corridor of a car traveling in its own lane, cut in at a safe distance. An automated controller-based system that follows the safety guidelines described above should not lead to overly defensive driving, as described below.

[0439] Processing limited sensing

[0440] Having considered the highway example, the second example next addresses the problem of limited sensing. A very common human response, when being blamed for an accident, falls into the category of “but I couldn’t see him.” A lot of the time, this is true. Humans have limited sensing capabilities, sometimes because of unconscious decisions to focus on different parts of the road, sometimes because of carelessness, and sometimes because of physical limitations—it’s impossible to see a pedestrian hiding behind a parked car. Of these human limitations, advanced automated sensing systems may be limited only by the latter: a 360° view of the road and the fact that computers are never careless, which allows automated sensing systems to surpass human sensing capabilities. Returning to the “but I couldn’t see him” example, the appropriate response is “Well, you should have been more careful.” To formalize what to pay attention to relative to limited sensing, consider Figure 22 Consider the scenario depicted in [ 1 ]. Car 2201 ( c0 ) is attempting to exit a parking lot and merge onto a (presumably) busy road, but cannot see if there are any cars on the street because a building 2203 blocks its view. Assume this is a narrow city street with a 30 km / h speed limit. The human driver's behavior is to slowly merge onto the road, gaining more and more field of view until the sensing limitation is removed. A crucial point in time should be defined: the first moment the occluded object is exposed to us; after this exposure, the human will treat it like any other object that the human can sense.

[0441] Definition 9 (Exposure Time) The exposure time of an object is the first time we see it.

[0442] Definition 10 (Fault due to unreasonable speed) Assume that at or after the exposure time, car c1 (car 2205) moves at a speed υ>υ limit If c0 does not drive, then the fault lies solely with c1. We say that c1 is to blame for the unreasonable speed.

[0443] This extension allows c0 to safely leave the parking lot. Using our previous definition of responsibility-sensitive safety, and dynamic υ limit Defined (using road conditions and speed limits, plus a reasonable margin), it is only necessary to check that in the worst case, as shown, the cut-in is at a safe longitudinal distance, assuming that c1 does not exceed υ limit Intuitively, this encourages c0 to drive slower and farther away from the occlusion, slowly increasing its field of view and then allowing a safe merge onto the street.

[0444] After extending the accident liability definition to the basic case of limited sensing, a series of extensions can address similar situations. There are many different definitions of what can be obscured (a potentially fast car cannot be obscured between two closely parked cars, but a pedestrian can) and what is the worst-case maneuver it can perform (a pedestrian's limit Than the car limit The simplistic assumption that drivers are more likely to be in danger of being hit by a car (and therefore less likely to be in danger of being hit by a car) implies a limitation on driving—one must be prepared for the worst-case scenario and be able to react if the moment of exposure suddenly arises. A more detailed example in an urban scenario can be taken from a pedestrian scene that may be obscured by parked vehicles. Accident fault in an accident involving a pedestrian can be defined as:

[0445] Definition 11 (Attribution of Liability for Accidents Involving Pedestrians) An accident involving a pedestrian is always attributable to the automobile unless one of the following three conditions holds:

[0446] The car hits the pedestrian with the side of the car, and the lateral velocity of the car is less than μ relative to the direction of impact.

[0447] The pedestrian's speed at or after the exposure time is greater than υ limit .

[0448] The car came to a complete stop.

[0449] Informally speaking, the car is not blameworthy only if the pedestrian runs into the side of the car and the car that hits the pedestrian is traveling no faster than μ, or if the car is in a stopped position, or if the pedestrian runs superhumanly fast in some direction (not necessarily the direction of the impact).

[0450] While the described system may not guarantee absolute safety, it could lead to scenarios where few, if any, accidents occur in autonomous vehicles. For ex...

Claims

1. A navigation system for a vehicle, the system comprising: at least one processor programmed to: receiving at least one captured image representing a feature in an environment of the vehicle from a camera; identifying at least a portion of an intersection represented in the at least one captured image; identifying a representation in the at least one captured image of a pedestrian proximate the intersection; determining a navigation maneuver of the vehicle relative to the intersection based at least in part on routing information of the vehicle; determining a predicted path of the vehicle relative to the intersection based on the determined navigation action; determining a predicted path of the pedestrian based on an analysis of the at least one captured image; Determining whether the vehicle is expected to collide with the pedestrian, wherein determining whether the vehicle is expected to collide with the pedestrian includes identifying: a predicted intersection between an envelope around the vehicle and a corridor for the pedestrian, the corridor for the pedestrian being determined based on the envelope around the pedestrian and the predicted path of the pedestrian; as well as a predicted intersection between the envelope around the pedestrian and a corridor for the vehicle, the corridor for the vehicle being determined based on the envelope around the vehicle and the predicted path for the vehicle; as well as In response to a determination that the vehicle is expected to collide with the pedestrian, at least one system associated with the vehicle is caused to implement a collision mitigation action. 2 . The navigation system of claim 1 , wherein the collision mitigation action comprises issuing an audible warning or a visual warning to a user of the vehicle. 3 . The navigation system of claim 1 , wherein the collision mitigation action comprises applying brakes of the vehicle. 4 . The navigation system of claim 1 , wherein the collision mitigation action comprises steering the vehicle away from the predicted path of the vehicle.

5. The navigation system of claim 4, wherein the collision mitigation action comprises steering the vehicle toward a path determined to pass behind the pedestrian. 6 . The navigation system of claim 1 , wherein the navigation action of the vehicle comprises one of a left turn, a right turn, or a continuous straight-ahead action relative to the intersection.

7. The navigation system according to claim 1, wherein: The determination of whether the vehicle is expected to collide with the pedestrian is based on a predetermined frontal area of ​​the vehicle. 8 . The navigation system of claim 7 , wherein the determination of whether the vehicle is expected to collide with the pedestrian is further based on an exposure area of ​​the pedestrian.

9. The navigation system of claim 1, wherein the determination of whether the vehicle is expected to collide with the pedestrian is based on an envelope associated with the vehicle.

10. The navigation system of claim 9, wherein the envelope associated with the vehicle comprises a bounding box associated with one or more edges of the vehicle.

11. The navigation system of claim 9, wherein the envelope associated with the vehicle includes a predetermined buffer distance relative to one or more edges of the vehicle, and the determination of whether the vehicle is expected to collide with the pedestrian includes a determination of whether the envelope associated with the vehicle intersects any portion of the pedestrian.

12. The navigation system of claim 11, wherein the predetermined buffer distance is at least one meter.

13. The navigation system of claim 11, wherein the predetermined buffer distance is at least two meters.

14. The navigation system of claim 11, wherein the predetermined buffer distance varies with respect to a speed of the vehicle.

15. The navigation system of claim 1, wherein the determination of whether the vehicle is expected to collide with the pedestrian is based on a plurality of postures of the vehicle at an estimated time of collision.

16. The navigation system of claim 15, wherein the estimated time to collision comprises a time when the vehicle is closest to the pedestrian as the vehicle and the pedestrian travel along their respective predicted paths.

17. The navigation system of claim 15, wherein the plurality of gestures are determined based on at least one of a predicted path of the vehicle or a predicted path of the pedestrian.

18. The navigation system of claim 1, wherein the determination of whether the vehicle is expected to collide with the pedestrian is based on a plurality of postures of the pedestrian at an estimated time of collision.

19. The navigation system of claim 18, wherein the estimated time to collision comprises a time when the vehicle is closest to the pedestrian as the vehicle and the pedestrian travel along their respective predicted paths.

20. The navigation system of claim 19, wherein the plurality of gestures are determined based on at least one of a predicted path of the vehicle or a predicted path of the pedestrian.

21. The navigation system of claim 1, wherein the determination of whether the vehicle is expected to collide with the pedestrian is based on a circular envelope associated with the vehicle and a circular envelope associated with the pedestrian.

22. The navigation system of claim 21, wherein: The determination of whether the vehicle is expected to collide with the pedestrian is further based on a determination of whether a circular envelope associated with the vehicle is expected to intersect a circular envelope associated with the pedestrian due to movement of the circular envelope associated with the vehicle and the circular envelope associated with the pedestrian along their respective predicted paths.

23. The navigation system of claim 22, wherein: The determination of whether the vehicle is expected to collide with the pedestrian further includes a refined analysis of areas of potential intersection between circular envelopes, wherein the refined analysis relies on a refined envelope associated with at least one of the pedestrian or the vehicle.

24. The navigation system of claim 1, wherein determining whether the vehicle is expected to collide with the pedestrian comprises determining whether a predicted path of the vehicle intersects the predicted path of the pedestrian.

25. The navigation system of claim 24, wherein determining whether the vehicle is expected to collide with the pedestrian further comprises determining whether the vehicle and the pedestrian are expected to reach the intersection of the predicted path within a predetermined time window.

26. The navigation system of claim 1, wherein determining the predicted path of the vehicle comprises accessing a map storing a plurality of predetermined target trajectories for the intersection.

27. The navigation system of claim 26, wherein the plurality of predetermined target trajectories are associated with feasible paths for the vehicle along two or more different road segments extending from the intersection.

28. The navigation system of claim 26, wherein determining the predicted path of the vehicle further comprises selecting a target trajectory of the plurality of predetermined target trajectories based on the determined navigation maneuver of the vehicle.

29. The navigation system of claim 28, wherein the predicted path of the vehicle includes at least one difference from the selected target trajectory.

30. The navigation system of claim 29, wherein the at least one difference is determined based on a speed of the vehicle.

31. The navigation system of claim 30, wherein a speed of the vehicle above a predetermined threshold speed causes the predicted path to have a turning radius greater than a turning radius associated with the selected target trajectory.

32. A navigation system as described in claim 1, wherein determining the predicted path of the vehicle includes determining a predetermined straight line first trajectory extending from a current position of the vehicle, a predetermined straight line second trajectory extending from an estimated second position associated with the intersection, and an intermediate curved trajectory extending between the first trajectory and the second trajectory.

33. The navigation system of claim 32, wherein the curved trajectory has a constant radius of curvature.

34. The navigation system of claim 32, wherein the curved trajectory has a variable radius of curvature.

35. The navigation system of claim 32, wherein the curved trajectory has a continuously changing radius of curvature.

36. The navigation system of claim 1, wherein determining the predicted path of the pedestrian comprises determining a predetermined straight-line path of travel relative to a currently detected location of the pedestrian.

37. The navigation system of claim 36, wherein: The determination of the predicted path of the pedestrian is further based on a facing direction of the pedestrian, the facing direction being detected based on an analysis of the at least one captured image.

38. The navigation system of claim 36, wherein the linear path extends perpendicular to at least one drivable path associated with the intersection.

39. The navigation system of claim 1, wherein the at least one captured image comprises a plurality of captured images, wherein determining the predicted path of the pedestrian comprises determining a position of the pedestrian in the plurality of captured images.

40. The navigation system of claim 39, wherein the position of the pedestrian is determined relative to a lower portion of the pedestrian.

41. The navigation system of claim 39, wherein the position of the pedestrian in the plurality of captured images is determined with reference to a common coordinate system.

42. The navigation system of claim 1, wherein the at least one captured image comprises a plurality of captured images, and wherein determining the predicted path of the pedestrian comprises determining a distance from the vehicle to the pedestrian in the plurality of captured images.

43. The navigation system of claim 42, wherein the distance to the pedestrian is determined based on a size of the pedestrian in the plurality of captured images.

44. The navigation system of claim 43, wherein the dimension comprises a distance between at least two tracking points on the representation of the pedestrian in the plurality of captured images.

45. The navigation system of claim 1, wherein the determination of whether the vehicle is expected to collide with the vehicle is further based on an expected speed of the pedestrian and an expected speed of the vehicle.

46. ​​The navigation system of claim 45, wherein the predicted speed of the pedestrian is equal to the detected speed of the pedestrian.

47. The navigation system of claim 46, wherein the at least one captured image comprises a plurality of captured images, and wherein the detected speed of the pedestrian is determined based on analyzing movement of the pedestrian represented by the plurality of captured images.

48. The navigation system of claim 45, wherein the pedestrian's predicted speed is associated with a predetermined average pedestrian speed.

49. The navigation system of claim 45, wherein the predicted speed of the vehicle is equal to the current speed of the vehicle.

50. The navigation system of claim 45, wherein the predicted speed of the vehicle is lower than the current speed of the vehicle.

51. The navigation system of claim 1, wherein determining whether the vehicle is expected to collide with the pedestrian further comprises determining: a first time when an envelope around the vehicle is expected to overlap a corridor around a predicted path of the pedestrian; a second time at which an envelope around the vehicle is expected to no longer overlap a corridor around a predicted path of the pedestrian; a third time at which the envelope around the pedestrian is expected to overlap with the corridor around the predicted path of the vehicle; as well as A fourth time is expected when the envelope around the pedestrian no longer overlaps the corridor around the predicted path of the vehicle.

52. A method for navigating a vehicle, the method comprising: receiving at least one captured image representing a feature in an environment of the vehicle from a camera; identifying at least a portion of an intersection represented in the at least one captured image; identifying a representation in the at least one captured image of a pedestrian proximate the intersection; determining a navigation maneuver of the vehicle relative to the intersection based at least in part on routing information of the vehicle; Determining a predicted path of the vehicle relative to the intersection based on the determined navigation action; determining a predicted path of the pedestrian based on an analysis of the at least one captured image; Determining whether the vehicle is expected to collide with the pedestrian, wherein determining whether the vehicle is expected to collide with the pedestrian includes identifying: a predicted intersection between an envelope around the vehicle and a corridor for the pedestrian, the corridor for the pedestrian being determined based on the envelope around the pedestrian and the predicted path of the pedestrian; as well as a predicted intersection between the envelope around the pedestrian and a corridor for the vehicle, the corridor for the vehicle being determined based on the envelope around the vehicle and the predicted path for the vehicle; as well as In response to a determination that the vehicle is expected to collide with the pedestrian, at least one system associated with the vehicle is caused to implement a collision mitigation action.

Images