Modeling road user reactions to operate autonomous vehicles in the presence of occlusion

Abstract

The present disclosure provides a method for operating an automatic vehicle 100. A method for operating an automatic vehicle includes: detecting a first object and a second object in an environment of the automatic vehicle using a detection system of the automatic vehicle by one or more computing devices; determining, by the one or more computing devices, one or more roadway segments that will be obscured by the first object from the perspective of the automatic vehicle during a set time interval based on a positional relationship between the first object and one or more roadway segments; determining, by the one or more computing devices, a reaction time of the second object based on the obscured one or more roadway segments; and operating the automatic vehicle based on the determined reaction time by the one or more computing devices.

Description

[0001] This application is a divisional application of an invention patent application with an application date (international application date) of December 3, 2019, application number 201980092141.1 (international application number PCT / US2019 / 064163), and invention name “Operating an automatic vehicle based on road user reaction modeling in the event of occlusion”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Application Serial No. 16 / 220,144, filed December 14, 2018, the entire disclosure of which is incorporated herein by reference. Background Art

[0004] Autonomous vehicles, such as vehicles that do not require a human driver, can be used to assist in transporting passengers or items from one location to another. Such vehicles can operate in a fully autonomous mode where the passenger can provide some initial input, such as a pick-up or destination location, and the vehicle maneuvers itself to that location.

[0005] Such vehicles are usually equipped with various types of sensors to detect surrounding objects. For example, an automated vehicle may include lasers, sonars, radars, cameras, and other devices that scan and record data from around the vehicle. Sensor data from one or more of these devices can be used to detect objects and their respective characteristics (position, shape, direction, speed, etc.). These characteristics can be used to predict the trajectory of other objects. These trajectories can define what an object may do in a certain short period of time in the future. These trajectories can then be used to control the vehicle to avoid these objects. Therefore, detection, recognition, and prediction are key functions for the safe operation of automated vehicles. Summary of the invention

[0006] Aspects of the present disclosure provide a method for operating an automated vehicle. The method includes: determining, by one or more computing devices, a plurality of road segments in an environment of the automated vehicle; detecting, by one or more computing devices, a first object and a second object in the environment of the automated vehicle using a detection system of the automated vehicle; determining, by one or more computing devices, a position of the first object relative to the plurality of road segments; determining, by one or more computing devices, one or more road segments of the plurality of road segments to be obscured by the first object from the perspective of the automated vehicle during a set time interval based on the position; determining, by one or more computing devices, a reaction time of the second object based on the obscured one or more road segments; determining, by one or more computing devices, a driving instruction of the automated vehicle based on the determined reaction time of the second object, the driving instruction including a timing of a maneuver of the automated vehicle; and operating the automated vehicle based on the driving instruction by the one or more computing devices.

[0007] In one example, determining the one or more lane segments to be obstructed includes determining a relationship between a position of a first object and a position of the automated vehicle during a set time interval. In this example, determining the relationship between the position of the first object and the position of the automated vehicle during the set time interval includes determining that an area is obstructed from view of the automated vehicle at the location during the set time interval.

[0008] In another example, the second object is a type of vehicle associated with the nominal human actor. In this example, determining the reaction time includes determining whether the nominal human actor associated with the second object is positioned such that the nominal human actor's field of view includes the location of the automated vehicle. Determining the reaction time also optionally includes calculating a linear segment between a chassis of the automated vehicle to the predicted location of the nominal human actor.

[0009] In a further example, determining the reaction time includes: determining a trigger point at which the second object will most likely begin to react to the automated vehicle; and adding the estimated perception time to the trigger point. In yet another example, the method also includes tracking, by the one or more computing devices, the occlusions on the plurality of lane segments by determining one or more second lane segments to be occluded by the first object during a subsequent time interval immediately following the set time interval.

[0010] Other aspects of the present disclosure provide an autonomous driving system for an automatic vehicle. The autonomous driving system includes: a detection system configured to detect objects in the environment of the automatic vehicle, and one or more computing devices communicating with the detection system. The one or more computing devices are configured to: determine a plurality of road segments in the environment of the automatic vehicle; detect a first object and a second object in the environment of the automatic vehicle using the detection system; determine the position of the first object relative to the plurality of road segments; based on the position, determine one or more road segments among the plurality of road segments to be obscured by the first object from the perspective of the automatic vehicle during a set time interval; determine the reaction time of the second object based on the one or more obscured road segments; determine a driving instruction of the automatic vehicle based on the determined reaction time of the second object, the driving instruction including the timing of the manipulation of the automatic vehicle; and operate the autonomous driving system based on the timing of the driving instruction manipulation.

[0011] In one example, one or more lane segments are determined to be occluded based on an area blocked from the field of view of the automated vehicle by a first object during a set time interval. In another example, the second object is a type of vehicle associated with a nominal human actor; and the reaction time is determined based on whether the nominal human actor associated with the second object is positioned such that the field of view of the nominal human actor includes the position of the automated vehicle. In this example, the reaction time is also determined based on a calculated linear segment between the chassis of the automated vehicle to the predicted position of the nominal human actor.

[0012] In another example, the reaction time is determined based on: a stimulus point at which the second object will most likely begin to react to the automated vehicle; and an additional estimated perceived time to the stimulus point. In a further example, the one or more computing devices are further configured to track occlusions on the plurality of lane segments by determining one or more second lane segments to be occluded by the first object during a subsequent time interval immediately following the set time interval. In yet another example, the system further includes an automated vehicle.

[0013] A further aspect of the present disclosure provides a non-transitory, tangible computer-readable storage medium having computer-readable instructions of a program stored thereon. The instructions, when executed by one or more processors, cause the one or more processors to perform a method. The method includes: determining a plurality of road segments in an environment of an automatic vehicle; detecting a first object and a second object in the environment of the automatic vehicle using a detection system of the automatic vehicle; determining a position of the first object relative to the plurality of road segments; based on the position, determining one or more road segments of the plurality of road segments to be obscured by the first object from the perspective of the automatic vehicle during a set time interval; determining a reaction time of the second object based on the obscured one or more road segments; determining a driving instruction of the automatic vehicle based on the determined reaction time of the second object, the driving instruction including a timing of a manipulation of the automatic vehicle; and operating the automatic vehicle based on the driving instruction.

[0014] In one example, determining the one or more lane segments to be obstructed includes determining a relationship between a position of a first object and a position of the automated vehicle during a set time interval. In this example, determining the relationship between the position of the first object and the position of the automated vehicle during the set time interval includes determining that an area is obstructed from view of the automated vehicle at the location during the set time interval.

[0015] In another example, the second object is a type of vehicle associated with a nominal human actor, and determining the reaction time includes determining whether the nominal human actor associated with the second object is positioned such that a field of view of the nominal human actor includes a location of the automated vehicle. In this example, determining the reaction time also includes calculating a linear segment between a chassis of the automated vehicle to the predicted location of the nominal human actor. In a further example, determining the reaction time includes: determining a stimulus point at which the second object will most likely begin to react to the automated vehicle; and adding the estimated perception time to the stimulus point. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a functional diagram of an example vehicle in accordance with aspects of the present disclosure.

[0017] Figure 2 is example map information according to aspects of the present disclosure.

[0018] Figure 3 are example exterior views of a vehicle according to aspects of the present disclosure.

[0019] Figure 4 is an example visual diagram of a system according to aspects of the present disclosure.

[0020] Figure 5 is an example functional diagram of a system according to aspects of the present disclosure.

[0021] Figure 6 is a flow chart 600 of a method for operating a vehicle according to aspects of the present disclosure.

[0022] Figure 7 is an example visual diagram of a scenario 700 in accordance with aspects of the present disclosure.

[0023] Fig. 8A and Figure 8B is a further example visual diagram of a scenario 700 according to aspects of the present disclosure.

[0024] Fig. 9 is a visual diagram of a scenario 900 in accordance with aspects of the present disclosure.

[0025] Fig.10 is a visual diagram of a scenario 700 in accordance with aspects of the present disclosure. DETAILED DESCRIPTION

[0026] Overview

[0027] The present technology relates to using obscured lane segments in a vehicle environment to predict the behavior of road users in the vehicle environment. One or more objects may be detected in the vehicle environment. In order to determine the timing of a vehicle maneuver (such as an unprotected left turn), a predicted behavior of each of the one or more objects may be determined. The predicted behavior may depend on whether the object can see the vehicle and react to the vehicle. Thus, determining the predicted behavior may include detecting whether an object closer to the vehicle will obstruct a lane segment that the object will, or more precisely, is predicted to travel through.

[0028] The above features can provide more accurate predictions of object behavior and lead to safer and more efficient autonomous driving systems that do not need to frequently recalculate paths because objects behave differently than predicted. Additionally, tracking occlusions at the lane segment level as described simplifies the process by using a finite or constant number of lane segments rather than having to solve a complex multi-body problem. Because the system is designed to identify more specific areas where the multi-body problem needs to be solved, less computing power may be required. The autonomous driving system can adjust predictions more quickly and react more quickly to objects in its vicinity.

[0029] Example System

[0030] like Figure 1 As shown in , a vehicle 100 according to one aspect of the present disclosure includes various components. Although certain aspects of the present disclosure are particularly useful for specific types of vehicles, the vehicle can be any type of vehicle, including but not limited to cars, trucks, motorcycles, buses, recreational vehicles, etc. The vehicle can have one or more computing devices, such as a computing device 110 that includes one or more processors 120, a memory 130, and other components typically found in a general-purpose computing device.

[0031] The memory 130 stores information accessible to one or more processors 120, including instructions 132 and data 134 that can be executed or otherwise used by the processor 120. The memory 130 can be of any type capable of storing information accessible to the processor, including computing device readable media, or other media that stores data that can be read by electronic devices, such as hard drives, memory cards, ROM, RAM, DVD or other optical disks, and other writable and read-only memories. The systems and methods may include different combinations of the foregoing, whereby different portions of the instructions and data are stored on different types of media.

[0032] Instructions 132 may be any set of instructions that are directly executed by a processor (such as machine code) or indirectly executed (such as a script). For example, instructions may be stored as computing device code on a computing device readable medium. In this regard, the terms "instructions" and "programs" are used interchangeably herein. Instructions may be stored in an object code format for direct processing by a processor, or in any other computing device language (including a collection of scripts or independent source code modules that are interpreted on demand or precompiled). The functions, methods, and routines of instructions are explained in more detail below.

[0033] The data 134 may be retrieved, stored, or modified by the processor 120 according to the instructions 132. As an example, the data 134 of the memory 130 may store predefined scenarios. A given scenario may identify a set of scenario requirements, including the type of object, the range of the object's position relative to the vehicle, and other factors, such as whether the automated vehicle can maneuver around the object, whether the object is using a turn signal, the condition of the traffic light relative to the object's current position, whether the object is approaching a stop sign, etc. These requirements may include discrete values ​​(such as "right turn signal on" or "in a right turn lane"), or ranges of values ​​(such as "having a heading that is offset from the vehicle's current path by an angle of 30 to 60 degrees"). In some examples, a predetermined scenario may include similar information for multiple objects.

[0034] The one or more processors 120 may be any conventional processor, such as a commercially available CPU. Alternatively, the one or more processors may be dedicated devices, such as an ASIC or other hardware-based processor. Figure 1 The processor, memory, and other elements of the computing device 110 are functionally shown in the same box, but it will be understood by those of ordinary skill in the art that the processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored in the same physical housing. As an example, the internal electronic display 152 may be controlled by a dedicated computing device with its own CPU or other processor, memory, etc., which may interface with the computing device 110 via a high-bandwidth or other network connection. In some examples, the computing device may be a user interface computing device capable of communicating with a user's client device. Similarly, the memory may be a hard drive or other storage medium located in a housing different from the housing of the computing device 110. Therefore, references to a processor or computing device will be understood to include references to a collection of processors or computing devices or memories that may or may not operate in parallel.

[0035] The computing device 110 may have all components typically used in association with a computing device, such as the processor and memory described above, as well as user input 150 (e.g., a mouse, keyboard, touch screen, and / or microphone) and various electronic displays (e.g., a monitor with a screen or any other electrical device operable to display information). In this example, the vehicle includes an internal electronic display 152 and one or more speakers 154 to provide information or an audio-visual experience. In this regard, the internal electronic display 152 may be located within the cabin of the vehicle 100 and may be used by the computing device 110 to provide information to passengers within the vehicle 100. In addition to the internal speakers, the one or more speakers 154 may also include external speakers arranged at different locations on the vehicle to provide audible notifications to objects outside the vehicle 100. The computing device 110 of the vehicle 100 may also receive information from other computing devices or send information to other computing devices, for example, using a wireless network connection 156.

[0036] In one example, computing device 110 may be an autonomous driving computing system incorporated into vehicle 100. Autonomous driving computing system may be capable of communicating with various components of the vehicle. For example, computing device 110 may communicate with various systems of vehicle 100, such as deceleration system 160 (for controlling braking of the vehicle), acceleration system 162 (for controlling acceleration of the vehicle), steering system 164 (for controlling the orientation of the wheels and the direction of the vehicle), signaling system 166 (for controlling turn signals), navigation system 168 (for navigating the vehicle to a location or navigating the vehicle around an object), positioning system 170 (for determining the location of the vehicle), perception system 172 (for detecting objects in the vehicle environment), and power system 174 (e.g., a battery and / or gasoline or diesel powered engine) to control the movement, speed, etc. of vehicle 100 according to instructions 132 of memory 130 in an autonomous driving mode that does not require or need continuous or periodic input from a vehicle occupant. Furthermore, although these systems are shown as being external to the computing device 110 , in practice, these systems may also be incorporated into the computing device 110 , again as an autonomous driving computing system for controlling the vehicle 100 .

[0037] The computing device 110 can control the direction and speed of the vehicle by controlling various components. As an example, the computing device 110 can use data from the map information and navigation system 168 to fully automatically navigate the vehicle to the destination location. The computing device 110 can use the positioning system 170 to determine the vehicle location, and use the perception system 172 to detect objects and respond to objects when necessary to safely reach the location. To this end, the computing device 110 can cause the vehicle to accelerate (e.g., by increasing the fuel or other energy provided to the engine by the acceleration system 162), decelerate (e.g., by reducing the fuel supplied to the engine by the deceleration system 160, shifting gears and / or by applying brakes), change direction (e.g., by turning the front or rear wheels of the vehicle 100 by the steering system 164), and signal such changes (e.g., by illuminating the turn signal of the signaling system 166). Therefore, the acceleration system 162 and the deceleration system 160 can be part of a transmission system, which includes various components between the vehicle engine and the wheels. In addition, by controlling these systems, the computing device 110 can also control the transmission system of the vehicle to automatically manipulate the vehicle.

[0038] As an example, computing device 110 may interact with deceleration system 160 and acceleration system 162 to control the speed of the vehicle. Similarly, steering system 164 may be used by computing device 110 to control the direction of vehicle 100. For example, if vehicle 100 is configured for use on a road, such as a car or truck, the steering system may include components for controlling the angle of the wheels to turn the vehicle. Signaling system 166 may be used by computing device 110, for example, by illuminating turn signals or brake lights when necessary, to signal vehicle intent to other drivers or vehicles.

[0039] The navigation system 168 may be used by the computing device 110 to determine and follow a route to a location. In this regard, the navigation system 168 and / or the data 134 may store map information, such as highly detailed maps that the computing device 110 may use to navigate or control the vehicle. As an example, these maps may identify the shape and elevation of lanes, lane markings, intersections, crosswalks, speed limits, traffic lights, buildings, signs, real-time or historical traffic information, vegetation, or other such objects and information. Lane markings may include features such as solid or dashed double lane lines or solid or dashed single lane lines, solid or dashed lane lines, reflective surfaces, and the like. A given lane may be associated with left and right lane lines or other lane markings that define lane boundaries. Therefore, most lanes may be defined by the left edge of one lane line and the right edge of another lane line. As described above, the map information may store known traffic information or congestion information and / or transportation schedules (trains, buses, etc.) from specific pickup locations at similar times in the past. This information may even be updated in real time via information received by computing device 110 .

[0040] Figure 2is an example of map information 200 including a section of a driving lane of an intersection 230. In this example, map information 200 depicts a portion of map information including information identifying the shapes, locations, and other characteristics of various features. For example, map information 200 includes road 210 and road 220 intersecting at intersection 230. Map information 200 includes lane markings or lane lines 241A and 243A of road 210 on a first side of intersection 230, lane lines 241B and 243B of road 210 on a second side of intersection 230 opposite to the first side. In addition, map information includes lane lines 242 and 246 of road 220 passing through intersection 230 from a third side to a fourth side opposite to the third side, lane line 244A of road 220 on the third side of intersection 230, and lane line 244B of road 220 on the fourth side of intersection 230. The lane markings may be different types of lane markings, such as two-line lane markings 241A, 241B, 244A, and 244B, and dashed lane markings 242, 243A, 243B, and 246. The lane markings may also define various lanes, such as lanes 251, 252, 253, 254, 255, 256, and 258. Lane portions 251A, 253A, and 255A of road 210 are on a first side of intersection 230, while lane portions 251B, 253B, and 255B of road 210 are on a second side of intersection 230 opposite to the first side. Lane portions 252A, 254A, 256A, and 258A of road 220 are on a third side of intersection 230, while lane portions 252B, 254B, 256B, and 258B of road 220 are on a fourth side of intersection 230 opposite the third side. Lanes may be explicitly identified in map information 200 as shown, or may be implied by the width of the road. Map information 200 may also identify bicycle lanes. As shown, map information 200 may also include stop lines 261 and 263 of road 210.

[0041] In addition to these features, the map information 200 may also include information identifying the direction of traffic and speed limits for each lane, as well as information that allows the computing device 110 to determine whether the vehicle has the right of way to complete a particular maneuver (e.g., to complete a turn or to cross a lane of traffic or an intersection). The map information 200 may also include information about traffic signs, such as traffic lights, stop signs, one-way signs, no turn signs, etc. The map information 200 may include information about other environmental features, such as curbs, buildings, parking lots, roadways, waterways, vegetation, etc.

[0042] Although detailed map information is described herein as an image-based map, the map information need not be completely image-based (e.g., raster). For example, the detailed map information may include one or more road maps or a graphical network of information, such as roads, lanes, intersections, and the connections between these features. Each feature may be stored as graphical data and may be associated with information such as a geographic location and whether it is associated with other related features, e.g., a stop sign may be associated with a road and an intersection, etc. In some examples, the associated data may include a grid-based index of the road map to allow efficient lookup of certain road map features.

[0043] The perception system 172 also includes one or more components for detecting objects external to the vehicle, such as other vehicles, obstacles in the road, traffic signals, signs, trees, etc. For example, the perception system 172 may include one or more LIDAR sensors, sonar devices, radar units, cameras, and / or any other detection devices that record data that can be processed by the computing device 110. The sensors of the perception system can detect objects and their characteristics, such as location, orientation, size, shape, type (e.g., vehicle, pedestrian, cyclist, etc.), heading, and speed of movement, etc. The raw data from the sensors and / or the aforementioned characteristics can be quantized or arranged into descriptive functions, vectors, and / or bounding boxes, and periodically and continuously sent to the computing device 110 for further processing as it is generated by the perception system 172. As discussed in further detail below, the computing device 110 can use the positioning system 170 to determine the vehicle location, and use the perception system 172 to detect objects and respond to objects when necessary in order to safely reach the location.

[0044] Figure 3 3 is an example external view of the vehicle 100. In this example, the roof-top housing 310 and the dome housing 312 can include LIDAR sensors and various cameras and radar units. In addition, the housing 320 located at the front end of the vehicle 100 and the housing 330 on the driver's side and the housing 332 on the passenger side can each store the LIDAR sensor. For example, the housing 330 is located in front of the driver's door 350. The vehicle 100 also includes housings 340, 342 for radar units and / or cameras that are also located on the top of the vehicle 100. Additional radar units and cameras (not shown) can be located at the front and rear ends of the vehicle 100 and / or other locations along the roof or roof-top housing 310. The vehicle 100 also includes many features of a typical passenger vehicle, such as doors (350, 352), wheels (360, 362), etc.

[0045] Upon detecting a nearby object, computing device 110 and / or perception system 172 may determine the type of object, e.g., a traffic cone, a pedestrian, a vehicle (such as a passenger car, truck, bus, etc.), a bicycle, etc. Objects may be identified by various models that may take into account various characteristics of the detected object, such as the size of the object, the speed of the object (bicycles tend not to go faster than 40 mph or slower than 0.1 mph), the heat from the bicycle (bicycles tend to have riders that radiate heat from their bodies), etc. Additionally, objects may be classified based on specific attributes of the object, such as information contained on a ticket, bumper sticker, or logo that appears on a vehicle.

[0046] The memory 130 may store various models used by the computing device 110 to determine how to control the vehicle 100. For example, the memory 130 may store one or more object recognition models for identifying road users and objects detected from the sensor data. For another example, the memory 130 may store one or more behavior models for providing probabilities of one or more actions being taken by the detected objects. For another example, the memory 130 may store one or more speed planning models for determining a speed profile of the vehicle 100 based on the map information 200 from the navigation system 168 and the predicted trajectories of other road users detected by the sensor data from the perception system 172.

[0047] Figure 4 and Figure 5 4 are a visual diagram and a functional diagram, respectively, of an example system 400 that includes a plurality of computing devices 410, 420, 430, 440 and a storage system 450 connected via a network 460. The system 400 also includes a vehicle 100 and a vehicle 100A that may be configured similarly to the vehicle 100. Although only a few vehicles and computing devices are depicted for simplicity, a typical system may include significantly more vehicles and computing devices.

[0048] like Figure 4 As shown in , each of computing devices 410, 420, 430, 440 may include one or more processors, memories, data, and instructions. Such processors, memories, data, and instructions may be configured similarly to one or more processors 120, memories 130, data 134, and instructions 132 of computing device 110.

[0049] The network 460 and intermediate nodes may include a variety of configurations and protocols, including short-range communication protocols such as BLUETOOTH(R), Bluetooth LE, the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local area network, a private network using a communication protocol that is proprietary to one or more companies, Ethernet, WiFi, and HTTP, as well as various combinations of the foregoing. Such communications may be facilitated by any device capable of transmitting data to and from other computing devices, such as a modem and a wireless interface.

[0050] In one example, computing device 110 may include a server (e.g., a load-balanced server cluster) having multiple computing devices that exchange information with different nodes of a network for the purpose of receiving data from other computing devices, processing data, and sending data to other computing devices. For example, one or more computing devices 410 may include one or more server computing devices capable of communicating with computing device 110 of vehicle 100 or similar computing devices of vehicle 100A and client computing devices 420, 430, 440 via network 460. For example, vehicles 100 and 100A may be part of a fleet of vehicles that can be dispatched to various locations by server computing devices. In this regard, the vehicles in the fleet may periodically send server computing device location information provided by the vehicles' respective positioning systems, and the one or more server computing devices may track the locations of the vehicles.

[0051] Additionally, server computing device 410 may transmit information using network 460 and present information to users, such as users 422, 432, 442, on displays, such as displays 424, 434, 444 of computing devices 420, 430, 440. In this regard, computing devices 420, 430, 440 may be considered client computing devices.

[0052] like Figure 5 As shown, each client computing device 420, 430, 440 can be a personal computing device intended for use by a user 422, 432, 442, and has all components commonly used in association with a personal computing device, including one or more processors (e.g., CPU), memory (e.g., RAM and internal hard drive) to store data and instructions, a display such as a display 424, 434, 444 (e.g., a monitor with a screen, a touch screen, a projector, a television, or other device operable to display information), and a user input device 426, 436, 446 (e.g., a mouse, keyboard, touch screen, or microphone). The client computing device may also include a camera for recording a video stream, a speaker, a network interface device, and all components for connecting these elements to each other.

[0053] Although each of client computing devices 420, 430, and 440 may include a full-size personal computing device, they may alternatively include a mobile computing device capable of wirelessly exchanging data with a server over a network, such as the Internet. By way of example only, client computing device 420 may be a mobile phone or a device capable of acquiring information via the Internet or other network, such as a wirelessly enabled PDA, tablet PC, wearable computing device or system, or netbook. In another example, client computing device 430 may be a wearable computing system, such as a Figure 4 As an example, the user can input information using a keypad, a keypad, a microphone, using visual signals with a camera, or a touch screen.

[0054] In some examples, client computing device 440 may be a teleoperator workstation used by an administrator to provide teleoperator services to users, such as users 422 and 432. For example, teleoperator 442 may use teleoperator workstation 440 to communicate with users through their respective client computing devices and / or vehicles 100 or 100A via a telephone call or audio connection to ensure safe operation of vehicles 100 and 100A and the safety of the users, as described in further detail below. Figure 4 and Figure 5 Only a single remote operator workstation 440 is shown in FIG. 4 , but any number of such workstations may be included in a typical system.

[0055] The storage system 450 can store various types of information, as described in further detail below. The information can be retrieved or otherwise accessed by a server computing device (such as one or more server computing devices 410) to perform some or all of the features described herein. For example, the information can include user account information, such as credentials that can be used to identify a user of one or more server computing devices (e.g., a username and password in the case of traditional single-factor authentication, and other types of credentials commonly used in multi-factor authentication, such as random identifiers, biometrics, etc.). The storage system 450 can also store routing data for generating and evaluating routes between locations. For example, routing information can be used to estimate how long it will take for a vehicle at a first location to reach a second location. In this regard, the routing information can include map information, not necessarily as specific as the detailed map information described above, but including roads, and information about these roads, such as direction (one-way, two-way, etc.), orientation (north, south, etc.), speed limits, and traffic information identifying expected traffic conditions, etc. The storage system 450 can also store map information, such as map information 200.

[0056] The storage system 450 may also store various models used for routing and controlling the vehicle 100, such as object recognition models, behavior models, speed planning models, etc. In this regard, the storage system 450 may store data used to train some or all of these models. For example, the training data may include images that are manually labeled with recognized objects, videos of trajectories taken by road users under various traffic patterns. The storage system 450 may store other data required to use these models, such as parameters and values.

[0057] Storage system 450 may also store information that can be provided to a client computing device for display to a user. For example, storage system 450 may store predetermined distance information for determining areas where a vehicle may stop for a given pickup location or destination location. Storage system 450 may also store graphics, icons, and other items that may be displayed to a user as described below.

[0058] As with memory 130, storage system 450 may be any type of computerized storage device capable of storing information accessible to server computing device 410, such as a hard drive, memory card, ROM, RAM, DVD, writable memory, and read-only memory. In addition, storage system 450 may include a distributed storage system in which data is stored on multiple different storage devices that may be physically located in the same or different geographic locations. Storage system 450 may be configured as follows: Figure 4 4 is shown connected to a computing device via network 460, and / or may be directly connected to or incorporated into any computing device among computing devices 110, 410, 420, 430, 440, etc.

[0059] Example Method

[0060] In addition to the systems described above and shown in the accompanying drawings, various operations will now be described. The computing device 110 can use the occluded roadway segments in the vehicle environment as described below to predict the behavior of road users in the vehicle environment. Figure 6 , the flowchart 600 shown according to various aspects of the present disclosure may be executed by the computing device 110. Figure 6 The blocks are shown in a particular order, but the order may be varied and multiple operations may be performed simultaneously. Additionally, operations may be added or omitted.

[0061] At box 602, the computing device 110 determines a plurality of lane segments in a vehicle environment. The determination may be made by detecting traffic control features of a road within a detection range of the vehicle 100 to identify lanes on the road. For example, traffic control features for identifying lanes on a road may include lane lines or lane markings. Lane segments may be defined as a fixed length of a particular lane, such as a 10-foot lane. The length is the distance between two endpoints. The length may also be defined as a set area, such as a rectangle or other shape. Multiple lane segments for a particular lane may be adjacent to each other along the length of the particular lane. In some implementations, determining a plurality of lane segments in the vehicle environment may include determining a location of the vehicle 100 and accessing map data stored in the vehicle memory 130 that identifies lane segments in an area surrounding the vehicle location. In Figure 7 In the scenario 700 depicted in FIG. 1 , the vehicle 100 may be parked in lane portion 253A near the intersection 230 at the stop line 261. The vehicle 100 may have a planned maneuver 702 to cross one or more lanes of the road 220. The vehicle's planned maneuver 702, as shown by the dashed line, includes a left turn from lane portion 253A into lane portion 256A of the road 220. In the maneuver 702, the vehicle 100 needs to cross lanes 252 and 254 of the road 220.

[0062] for Figure 7 , defines twelve (12) lane segments, as shown by a plurality of boxed areas arranged end-to-end along lane 252. The shape of the lane segments can be other shapes or linear segments. Portions of lanes 254, 256, and 258 also have approximately twelve defined lane segments, also shown by a plurality of boxed areas arranged end-to-end. These lane segments can be defined by computing device 110, or can be predetermined and stored in the memory 130 of the vehicle.

[0063] At block 604, computing device 110 detects a plurality of objects in the vehicle environment using, for example, sensor data from perception system 172. The sensor data may also include characteristics of each object, such as the size, shape, speed, orientation, direction, etc. of the object. The plurality of objects may include moving and / or stationary objects. In particular, the plurality of objects may include other road users, such as vehicles, bicycles, or pedestrians, or may include other types of obstacles, such as buildings, posts, trees, or construction tools. Figure 7As shown in FIG. 1 , a truck 710 and a van 720 are in the vehicle environment. Computing device 110 can use perception system 172 to detect truck 710 and van 720 traveling in lanes 252 and 254, respectively, and classify both truck 710 and van 720 as road users. Computing device 110 can also use perception system 172 to identify the approximate dimensions and speed of truck 710 and van 720.

[0064] At box 606, the computing device 110 determines location information of each object in the plurality of objects in relation to the plurality of road segments. The location information may include the location of each object and the orientation of each object. For a first object among the plurality of road users, the computing device 110 may detect the location of the first object relative to the detected traffic control feature. In some other implementations, the location of the first object relative to the vehicle location may be detected and identified using map data stored in a vehicle memory. In some implementations, the location information may also include a predicted behavior of each object, such as a trajectory. For example, the predicted behavior may include a trajectory of the first object determined based on a speed, orientation, and / or direction of the first object detected by the perception system 172. When the first object is a stationary object, the predicted behavior may be that the first object remains stationary.

[0065] In scenario 700, computing device 110 may determine that truck 710 is in lane portion 252A based on the position of the truck being between lane line 242 and the edge of road 220 and / or the position of the truck being on lane segment 712. Computing device 110 may also determine that truck 710 has an orientation toward lane portion 252B. Similarly, computing device 110 may determine that van 720 is in lane portion 254A based on the position of the van being between lane lines 242 and 244A and / or the position of the van being on lane segment 722. The computing device may also determine that van 720 has an orientation toward lane portion 254B. With respect to predicted behavior, the predicted behavior of the truck includes driving along lane portion 252A toward intersection 230 at a constant speed or at a speed within 5 miles per hour of the current speed during a first time interval. The predicted behavior of the van includes driving along lane portion 254A toward intersection 230 at a constant speed or at a speed within 5 miles per hour of the current speed during a first time interval. The speed of the van 720 during the first time interval may be greater than the speed of the truck 710 .

[0066] At box 608, computing device 110 also determines that one or more lane segments of the plurality of lane segments are or will be obscured by at least one of the plurality of objects for a first time interval. The first time interval may be set to, for example, five (5) seconds, or more or less. Determining that one or more lane segments are or will be obscured for a first time interval may include determining a relationship between the position of each object (including, for example, the first object) and the vehicle position during the first time interval. The start of the first time interval may be a current time or a future time. When determining that one or more lane segments will be obscured at a future point in time, a predicted position of the given object may be determined based on the position and predicted behavior of the given object.

[0067] The occlusion may include one or more areas that are blocked from the field of view of the vehicle 100 by multiple objects during the first time interval. These blocked areas can be identified based on the relationship between the locations. In particular, the blocked area may include an area that falls behind the current position or predicted position of a given object in the multiple objects from the viewpoint of the vehicle 100. The field of view of the vehicle 100 may be defined by the sensors of the perception system 172 and the sensor data generated by the perception system 172. The field of view may include areas in the sensor data within an angular range extending from the location of the vehicle 100 and within a certain distance from the location of the vehicle 100. Any roadway segment in the one or more areas may be determined to be blocked during the first time interval.

[0068] In scenario 700 , computing device 110 determines one or more roadway segments that are obscured by truck 710 . Figure 7 and Fig. 8A The positions of truck 710 and van 720 are shown at the beginning of a first time interval. As shown, truck 710 and van 720 are driving adjacent to each other, with van 720 being closer to the rear of truck 710 than to the front of the truck. Figure 8B The predicted future positions of truck 710 and van 720 at the end of the first time interval are shown. As shown, both truck 710 and van 720 are predicted to be driving along their respective lane portions 252A and 254A, and van 720 is predicted to be driving at a faster speed than truck 710, such that van 720 will be closer to the front of truck 710 and begin to pass truck 710 by a length at the end of the first time interval.

[0069] Computing device 110 may project the field of view as truck 710 and van 720 move between the beginning and the end of the first time interval. At the beginning of the first time interval, three (3) lane segments in lane portion 254A and two (2) lane segments in lane portion 256A are obscured by truck 710. Fig. 8A70 and the shading of the obscured lane segments. Additional lane segments in other map information may also be determined to be obscured. At the end of the first time interval, six (6) lane segments in lane portion 254A, five (5) lane segments in lane portion 256A, and four (4) lane segments in lane portion 258A are obscured by truck 710. Figure 8B 804 and the shaded portion of lane segments obscured from view in the first time interval. The obscured lane segments for the first time interval are all lane segments that are blocked from view during the entire time interval. Computing device 110 also determines lane segments obscured by other objects in the plurality of objects, such as van 720. For example, when van 720 passes truck 710, an additional two (2) lane segments in lane portion 256A obscured by van 720, such as Figure 8B The field of view portion 806 and the shaded lines of the blocked road segment are shown.

[0070] At block 610, computing device 110 then determines a reaction time of at least one object (such as the second object) of the plurality of objects based on the obscured one or more road segments. A reaction time is a point in time at which an object can adjust its behavior (such as by slowing down or speeding up) in response to a current behavior or a planned behavior of vehicle 100. The at least one object for which a reaction time is determined may be one or more objects classified as road users in the vehicle environment, such as truck 710 and van 720. Additionally, the at least one object may be one or more objects predicted to be in the obscured one or more road segments during the first time interval based on the predicted behavior of the at least one object, such as Fig. 8A and Figure 8B A van 720 is shown in FIG. 7 as being predicted to travel through the obstructed road segment.

[0071] The reaction time may be a predicted point in time and / or location at which a given object may be expected to become aware of the automated vehicle 100 and begin to react to the automated vehicle 100. Awareness of a given object may be assessed based on the position and orientation of the given object or a nominal human actor associated with the given object relative to the vehicle 100. The nominal human actor associated with the given object may be an operator of a road user, such as a driver of a vehicle or a cyclist on a bicycle, or a pedestrian. For example, a given object is determined to be aware of the vehicle 100 when the given object is not in an obstructed roadway segment and is oriented such that the associated nominal human actor's field of view includes the vehicle position. The nominal human actor's field of view may be limited to a forward-facing angular range of less than 180°.

[0072] For objects in one or more obstructed lane segments, determining the reaction time may include determining whether the associated nominal human actor is positioned so that the nominal human actor's field of view includes the vehicle position, regardless of whether it is in the obstructed lane segment. Even when an object (such as a car) is in the obstructed lane behind another object, a human actor (such as a driver of a vehicle) can still be in a position where he or she can see the automated vehicle 100 and react to the automated vehicle 100. The determination may include detecting that the position of the nominal human actor associated with a given object in the obstructed lane segment is in the field of view of the vehicle 100, and the given object is oriented so that the associated nominal human actor's field of view includes the vehicle position. Alternatively, the determination may include determining a minimum amount of the associated nominal human actor's field of view that can include the vehicle 100. The minimum amount may be 30%, 50%, or 70% of the surface area of ​​the vehicle 100. The location of the nominal human actor may be detected using the perception system 172 of the vehicle 100, or may be determined based on, for example, the average height of the nominal human actor's eyes above the road surface while driving. The location of the nominal human actor may also be determined based on the predicted position of the driver in the vehicle given the position and orientation of the vehicle. Fig. 9 In the scene 900 shown in FIG. 1 , where the positions of the truck 710 and the van are interchanged with those in the scene 700 , the computing device 110 may detect that the truck driver is in the rightmost quarter of the truck 710 relative to the orientation of the vehicle, and determine that the vehicle 100 is in the field of view of the truck driver. For example, a linear segment 902 may be calculated between the detected position of the truck driver and the position of the vehicle 100, and the linear segment 902 may be in the estimated field of view of the truck driver. Alternatively, the amount of the vehicle in the estimated field of view of the truck driver may be estimated to be greater than the above minimum amount.

[0073] In other implementations, determining whether the associated nominal human actor is positioned so that the nominal human actor's field of view includes the vehicle position may include calculating whether a linear segment between the lowest point of the body of the vehicle 100 and the position of the nominal human actor associated with a given object in the obscured one or more roadway segments intersects with any obstacles. The chassis of the vehicle 100 may be the lowest point of the body of the vehicle 100. When the obstacle intersects the linear segment, the given object is determined to be unaware of the vehicle 100 and unable to react to the vehicle 100. In addition, when the linear segment between the lowest point of the body of the vehicle 100 and the position of the nominal human actor is not within the nominal human actor's field of view, the given object is also determined to be unaware of the vehicle 100 and unable to react to the vehicle. Alternatively, when the portion of the vehicle within the nominal human actor's field of view is less than a minimum amount of the vehicle, the given object may be determined to be unaware of the vehicle 100 and unable to react to the vehicle.

[0074] Fig.10 is with Fig. 8A Another schematic diagram of scene 700 corresponding to the beginning of the first time shown in , wherein van 720 is in the obscured lane segment. A linear segment 1002 can be calculated between the chassis of vehicle 100 and the predicted position of the nominal human driver of van 720. The chassis is determined to be the center of the bottom of the vehicle body. Considering the orientation of van 720 along lane portion 252A to lane portion 252B, the predicted position of the nominal human driver is determined to be in the rightmost quarter of the van 720 as seen from the perspective of vehicle 100. Linear segment 1002 intersects with the body of truck 710. Therefore, computing device 110 can determine that van 720 is unaware of vehicle 100 at the beginning of the first time interval and the reaction time of van 720 is after the beginning of the first time interval.

[0075] Determining the reaction time may include adjusting the predicted behavior of the given object when the given object is traveling in the obstructed one or more road segments. In particular, it may be assumed that the one or more objects will not react to the vehicle 100 while traveling in the obstructed one or more road segments during the first time interval. For example, a second object, such as a van 720, may maintain the same or similar speed while traveling through the one or more obstructed road segments during the first time interval because the second object most likely will not see the vehicle 100 to slow down or speed up for the vehicle 100.

[0076] Determining the reaction time may also include determining the stimulus point at which each of the plurality of objects will most likely begin to react to the vehicle 100, and adding the estimated perception time. The stimulus point for a given object (such as a second object) may be determined as the time point at which the given object reaches one end of one or more obscured lane segments or reaches the minimum distance through one or more obscured lane segments. The minimum distance may be where the minimum amount of vehicle 100 becomes visible to the nominal human actor associated with the second object. Perception time, such as 0.5 seconds, 1.5 seconds or other time, may also be added to the determined stimulus point to account for the time taken by the nominal human actor to process the visual stimulus and then react to the visual stimulus (such as moving a foot and applying the brakes). Perception time may also include the amount of time taken by the nominal human actor to notice the vehicle 100. If the given object does not reach one end of one or more obscured lane segments during the first time interval, the stimulus point will not be determined for the given object.

[0077] In scenario 700, computing device 110 may determine the reaction time of van 720. Because van 720 is in a lane segment that is obscured by truck 710 for most of the first interval, Fig. 8A and Figure 8B , so van 720 may be predicted to travel through the obscured lane segment without any reaction to vehicle 100 (such as maintaining a constant speed). At the end of the first interval, the length by which van 720 exceeds truck 710 may be equal to the minimum distance at which vehicle 100 becomes visible to the nominal human actor in van 720. Therefore, the end of the first interval may be determined by computing device 110 as the trigger point for van 720. The reaction time of van 720 is then determined to be 1.5 seconds after the end of the first time interval.

[0078] At box 612, based on the determined reaction times of the multiple objects, the computing device 110 determines the driving instructions for the vehicle 100. The driving instructions for the vehicle 100 include the timing of the maneuver of the vehicle 100 depending on the reaction time of one or more of the multiple objects. For example, the maneuver may be an unprotected left turn across a single lane or a multi-lane road. This maneuver depends on the reaction time of the objects in the multi-lane road that the vehicle 100 needs to cross to make a left turn. The maneuver may be another type of unprotected maneuver, such as crossing an intersection or a right turn. In order to determine the timing of the maneuver, the computing device 110 may select the timing of the maneuver using the predicted behavior for each of the multiple objects after the corresponding reaction time when the vehicle 100 has enough time to complete the maneuver taking into account the trajectory and predicted behavior of the one or more objects. In some implementations, the predicted behavior is determined only for a subset of objects with trajectories that may intersect with the maneuver trajectory.

[0079] In scenario 700, computing device 110 determines the timing of planned maneuver 702 for vehicle 100 to make an unprotected left turn through intersection 230 onto multi-lane road 220. Since the reaction time of van 720 is predicted to be the end of the first time interval, computing device 110 determines the timing of planned maneuver 702 to be after the first time interval. In particular, the timing may be after both truck 710 and van 720 pass through intersection 230, rather than before, because van 720 will likely arrive at the intersection without seeing and reacting to vehicle 100.

[0080] In a further example, computing device 110 may continue tracking the obscured roadway segment by determining that the next one or more roadway segments will be obscured by at least one of the plurality of objects during a second time interval immediately following the first time interval. The second time interval may be the same length of time as the first time interval, or more or less. Determining the next one or more roadway segments may be based on a relationship between the predicted position of each object and the vehicle position during the second time interval. Determination of obscured roadway segments may thus be repeated as needed until a driving instruction is determined for the vehicle.

[0081] In some other implementations, a reaction time may be determined for a hypothetical third object in one or more lane segments that are obscured. When the obscured lane segment is partially or completely undetectable by the perception system 172 of the vehicle 100, it may be advantageous for the computing device 110 to determine the reaction time of a hypothetical object that may be in the obscured lane segment. In doing so, the computing device 110 may be able to plan for the vehicle 100 a maneuver that takes into account objects that may not be detected. The hypothetical third object may be assumed to have an associated nominal human actor and be in at least one lane segment in the obscured lane segment (with traffic traveling). The hypothetical third object may be assumed to have average characteristics of vehicles traveling in a lane that includes at least one obscured lane segment, such as an average speed or acceleration. The reaction time of the hypothetical third object may be determined as described for an object in the obscured lane segment in block 610.

[0082] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to obtain unique advantages. Since these and other variations and combinations of the above-mentioned features can be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be made by way of illustration, rather than by limiting the subject matter defined by the claims. In addition, the provision of examples described herein, as well as clauses phrased as "such as", "including", etc., should not be interpreted as limiting the subject matter of the claims to specific examples; on the contrary, these examples are intended to illustrate only one embodiment of many possible embodiments. In addition, the same reference numerals in different figures may identify the same or similar elements.

[0083] Translation of attached pictures

[0084] Figure 1

[0085] 110 Computing equipment

[0086] 120 processors

[0087] 130 Memory

[0088] 132 Instructions

[0089] 134 Data

[0090] 150 User input

[0091] 152 Internal electronic display

[0092] 154 Speakers

[0093] 156 Wireless network connection

[0094] 160 Speed ​​reduction system

[0095] 162 Acceleration System

[0096] 164 Steering system

[0097] 166 Signaling System

[0098] 168 Navigation System

[0099] 170 Positioning System

[0100] 172 Perception System

[0101] 174 Power system (engine)

[0102] Figure 4

[0103] Network

[0104] Figure 5

[0105] 100 Vehicles

[0106] 100A Vehicle 410

[0108] Server Computing Device

[0109] Processor(s)

[0110] Memory

[0111] Instructions

[0112] Data

[0113] 420 / 430

[0114] Client Computing Device

[0115] Processor(s)

[0116] Memory

[0117] Instructions

[0118] Data

[0119] 424 / 434 Display

[0120] 426 / 436 User input

[0121] 440 Network

[0122] Concierge workstation

[0123] Processor(s)

[0124] Memory

[0125] Instructions

Claims

1. A method for operating an automated vehicle, the method comprising: detecting, by one or more computing devices, a first object and a second object in an environment of the automated vehicle using a detection system of the automated vehicle; determining, by the one or more computing devices, one or more road segments that will be obscured by the first object from the perspective of the automated vehicle during a set time interval based on a positional relationship between the first object and the one or more road segments; determining, by one or more computing devices, a reaction time of a second object based on the one or more obscured roadway segments, wherein the reaction time of the second object indicates a point in time at which the second object is expected to be able to react to the automated vehicle; as well as The automated vehicle is operated by one or more computing devices based on the determined reaction time.

2. The method according to claim 1, wherein: The positional relationship includes a predicted trajectory of the first object during the set time interval.

3. The method according to claim 1, wherein: Determining one or more lane segments to be obscured includes identifying areas to be obscured from a field of view of the automated vehicle based on the positional relationship.

4. The method according to claim 3, wherein: Identifying an area obscured from the field of view of the automated vehicle based on the positional relationship includes: projecting the autonomous vehicle's field of view from the autonomous vehicle's position; and One or more road segments behind the first object relative to the projected field of view are determined.

5. The method according to claim 1, wherein: The second object is a vehicle type associated with the human; and Determining the reaction time includes predicting when a human associated with the second object will react to the automated vehicle.

6. The method according to claim 5, wherein: Predicting when the human associated with the second object will react to the automated vehicle includes determining when a field of view of the human includes a minimum amount of the automated vehicle.

7. The method according to claim 1, wherein: Determining reaction time includes: determining a trigger point at which the second subject will most likely begin to react to the automated vehicle; and Add the estimated perceived time to the incentive point.

8. A system for operating an automated vehicle, comprising: a detection system configured to detect objects in an environment of the automated vehicle, the detected objects comprising a first object and a second object; as well as One or more computing devices in communication with the detection system, the one or more computing devices being configured to: determining, based on a positional relationship between the first object and one or more road segments, one or more road segments that will be obscured by the first object from the perspective of the automated vehicle during a set time interval; determining a reaction time of a second object based on the obscured one or more lane segments, wherein the reaction time of the second object indicates a point in time when the second object is expected to be able to react to the automated vehicle; and Operate an automated vehicle based on reaction time.

9. The system according to claim 8, wherein: The positional relationship includes a predicted trajectory of the first object during a set time interval.

10. The system according to claim 8, wherein: The one or more computing devices are configured to determine one or more lane segments to be obscured based on the identified areas obscured from the field of view of the detection system according to the positional relationship.

11. The system of claim 8, wherein: The second object is a vehicle type associated with the human; and The reaction time is determined based on a predicted point in time when a human associated with the second object will react to the automated vehicle.

12. The system of claim 11, wherein the reaction time is further determined based on when the human's field of view includes a minimum amount of the automated vehicle.

13. The system of claim 8, wherein the reaction time is determined based on: The trigger point at which the second subject will most likely begin to react to the automated vehicle; and Additional estimated perceived time to the stimulus point.

14. The system of claim 8, further comprising an automated vehicle.

15. A non-transitory tangible computer-readable storage medium having stored thereon computer-readable instructions of a program, which, when executed by one or more processors, cause the one or more processors to perform a method comprising: detecting a first object and a second object in an environment of the autonomous vehicle using a detection system of the autonomous vehicle; determining, based on a positional relationship between the first object and the one or more road segments, one or more road segments that will be obscured by the first object from the perspective of the automated vehicle during a set time interval; determining a reaction time of a second object based on the obscured one or more lane segments, wherein the reaction time of the second object indicates a point in time when the second object is expected to be able to react to the automated vehicle; and The automated vehicle is operated based on the determined reaction time.

16. The medium according to claim 15, wherein The positional relationship includes a predicted trajectory of the first object during a set time interval.

17. The medium of claim 15, wherein determining one or more lane segments to be obscured comprises identifying areas to be obscured from a field of view of the automated vehicle based on the positional relationship.

18. The medium of claim 17, wherein identifying an area obscured from the field of view of the automated vehicle based on the positional relationship comprises: projecting a field of view of the autonomous vehicle from the position of the autonomous vehicle; as well as One or more lane segments behind the first object relative to the projected field of view are determined.

19. The medium of claim 15, wherein: The second object is a vehicle type associated with the human; and Determining the reaction time includes predicting when a human associated with the second object will react to the automated vehicle.

20. The medium of claim 19, wherein predicting when the human associated with the second object will react to the automated vehicle comprises determining when a field of view of the human includes a minimum amount of the automated vehicle.

21. A method for operating an automated vehicle, the method comprising: detecting, by one or more computing devices, an object in an environment of the automated vehicle using a detection system of the automated vehicle; determining, by the one or more computing devices, one or more lane segments that will be obscured from the perspective of the automated vehicle during a set time interval; determining, by the one or more computing devices, a reaction time of the subject based on the obstructed one or more roadway segments and a field of view of a human associated with the subject, wherein the reaction time of the subject indicates a point in time at which the subject is expected to be able to react to the automated vehicle; and An automated vehicle is operated by the one or more computing devices based on the determined reaction time.

22. The method of claim 21, further comprising predicting, by the one or more computing devices, that the object will travel through at least one of the one or more obscured road segments during the set time interval.

23. The method according to claim 21, wherein: Determining the reaction time includes predicting when a human associated with the subject will react to the automated vehicle during or after a set time interval.

24. The method according to claim 23, wherein: Predicting when the human associated with the object will react to the automated vehicle includes determining when a field of view of the human includes a minimum amount of the automated vehicle.

25. The method according to claim 23, wherein: Predicting when the human associated with the object will react to the automated vehicle includes determining that a position of the human associated with the object in the obscured one or more roadway segments is within a field of view of a detection system of the automated vehicle during or after the set time interval.

26. The method according to claim 25, wherein: Determining that the location of the human is in the field of view of the automated vehicle's detection system during or after the set time interval includes determining whether a linear segment of road between the lowest point of the automated vehicle and the location of the human intersects an obstacle.

27. The method of claim 21, further comprising determining, by the one or more computing devices, a field of view of the human based on a position of the human in an object detected by the detection system.

28. The method of claim 21, wherein: Determining reaction time is also based on the position and orientation of the object.

29. The method according to claim 21, wherein: Determining reaction time is also based on the human's position and orientation.

30. The method of claim 21, wherein determining the reaction time comprises determining a predicted behavior of the object when the object travels in one or more obscured road segments.

31. The method of claim 21, wherein determining the reaction time comprises: determining the trigger point at which the subject will most likely begin to react to the automated vehicle; as well as Add the estimated perceived time to the incentive point.

32. A system for operating an automated vehicle, comprising: a detection system configured to detect objects in an environment of the automated vehicle; as well as One or more computing devices in communication with the detection system, the one or more computing devices being configured to: determining one or more lane segments that will be obscured from the perspective of the automated vehicle during a set time interval; determining a reaction time of the subject based on the obscured one or more roadway segments and a field of view of a human associated with the subject, wherein the reaction time of the subject indicates a point in time at which the subject is expected to be able to react to the automated vehicle; and Operate an automated vehicle based on reaction time.

33. The system of claim 32, wherein: The one or more computing devices are configured to predict that the object will travel through at least one of the one or more obstructed roadway segments during the set time interval.

34. The system of claim 32, wherein: The one or more computing devices are configured to determine a reaction time based on a predicted time when a human associated with the object will react to the automated vehicle during or after a set time interval.

35. The system of claim 34, wherein: The one or more computing devices are configured to determine the predicted time based on when the human's field of view includes a minimum amount of the automated vehicle.

36. The system of claim 34, wherein: The one or more computing devices are configured to determine the predicted time based on when a position of the human associated with the object at the obscured one or more roadway segments is within a field of view of a detection system of the automated vehicle during or after the set time interval.

37. The system of claim 36, wherein: The one or more computing devices are configured to determine when the human's position is in the field of view of the automated vehicle's detection system during or after the set time interval based on whether the linear segment between the automated vehicle's lowest point and the human's position intersects an obstacle.

38. The system of claim 32, wherein: The one or more computing devices are configured to determine a field of view of the human based on a position of the human in an object detected by the detection system.

39. The system of claim 32, further comprising an automated vehicle.

40. A non-transitory tangible computer-readable storage medium having stored thereon computer-readable instructions of a program, which, when executed by one or more computing devices, cause the one or more computing devices to perform a method comprising: detecting an object in an environment of an autonomous vehicle using a detection system of the autonomous vehicle; determining one or more lane segments that will be obscured from the perspective of the automated vehicle during a set time interval; determining a reaction time of the subject based on the obscured one or more roadway segments and a field of view of a human associated with the subject, wherein the reaction time of the subject indicates a point in time at which the subject is expected to be able to react to the automated vehicle; and The automated vehicle is operated based on the determined reaction time.

Images