Transmission for Autonomous Vehicles

Abstract

To provide a method for facilitating communications from an autonomous vehicle (100) to a user.SOLUTION: For instance, a method may include: while attempting to pick up a user and prior to the user entering a vehicle, inputting a current location of the vehicle and map information (200) into a model in order to identify a type of communication action for communicating a location of the vehicle to the user; enabling a first communication based on the type of the communication type; determining whether the user has responded to the first communication from received sensor data; and enabling a second communication based on the determination of whether the user has responded to the communication.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Application No. 16 / 351,124, filed March 12, 2019, the disclosure of which is incorporated herein by reference. [Background technology]

[0002] Autonomous vehicles, such as vehicles that do not require a human driver, may be used to assist in the transportation of passengers or goods from one location to another. Such vehicles may operate in a fully autonomous mode, where the passenger can provide some initial input, such as a pickup location or destination, and the vehicle will navigate itself to the location.

[0003] When a person (or user) desires to be physically transported between two locations via a vehicle, they may use any number of taxi services. To date, these services typically involve a human driver who is given dispatch instructions to locations to pick up and drop off the user. Typically, these locations are known through a physical cue (i.e., raising a hand to have the driver stop), a phone call explaining where the user is actually located, or a face-to-face discussion between the driver and the user. While these services are convenient, they typically fail to provide the user with precise information regarding where the pick-up or drop-off will occur. Summary of the Invention

[0004] Aspects of the present disclosure provide a method for facilitating communication from an autonomous vehicle to a user. The method includes inputting, by one or more processors of the vehicle, a current location and map information of the vehicle into a model while attempting to pick up the user and before the user enters the autonomous vehicle, identifying a type of communication action to communicate the location of the vehicle to the user using the model, enabling, by the one or more processors, a first communication based on the type of communication action, and, after enabling the first communication, determining, by the one or more processors, whether the user is moving toward the vehicle from received sensor data.

[0005] In one example, the type of communication action is automatically generating an audible communication by the vehicle, and enabling the first communication includes instructing the vehicle to make the audible communication. In this example, the first communication is honking the vehicle's horn. In another example, the type of communication action is automatically surfacing an option on the user's client computing device to enable the user to have the vehicle generate the audible communication. In another example, the type of communication action is automatically generating a visual communication by the vehicle, and enabling the first communication includes the vehicle making the visual communication. In this example, the first communication is flashing the vehicle's headlights. In another example, the type of communication action is automatically surfacing an option on the user's client computing device to enable the user to have the vehicle generate the visual communication. In another example, the received sensor data includes location information generated by the user's client computing device. In another example, the received sensor data includes data generated by a perception system of the vehicle, and the perception system includes at least one sensor. In another example, the method also includes determining a type of communication action for the second communication using the model of escalated communication and enabling, by the one or more processors, the second communication based on a determination of whether the user is moving toward the vehicle, where the type of communication action for the second communication is further used to enable the second communication. In this example, the type of communication action for the second communication is automatically surfacing an option on the user's client computing device to enable the user to have the vehicle generate an audible communication. Alternatively, the type of communication action for the second communication is automatically surfacing an option on the user's client computing device to enable the user to have the vehicle generate a visual communication. In another example, the first communication includes the vehicle automatically flashing its lights, and the second communication includes the vehicle automatically honking its horn.In another example, the first communication includes the vehicle automatically honking the vehicle horn and the second communication includes the vehicle automatically requesting a customer service representative to connect to the user's client computing device. In another example, the model is a machine learning model.

[0006] Another aspect of the present disclosure provides a method for training a model for facilitating communication from an autonomous vehicle to a user, the method including: receiving, by one or more computing devices, training data including a first training input indicating a location of the vehicle, a second training input indicating map information, a third training input indicating a location of the user, a fourth training input characterizing sensor data identifying one or more objects in an environment of the vehicle, and a target output indicating a type of communication; training, by the one or more computing devices, the model on the training data according to current values ​​of parameters of the model to generate a set of output values ​​indicating a level of suitability for the type of communication; determining a difference value using the target output and the set of output values; and adjusting the current values ​​of the parameters of the model using the difference value.

[0007] In one example, the training data corresponds to a request by a user to have the vehicle perform a type of communication to communicate with the user. In another example, the type of communication is an auditory communication. In another example, the type of communication is a visual communication. In another example, the training data further includes ambient lighting conditions. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a functional diagram of an exemplary vehicle in accordance with an exemplary embodiment. [Figure 2] 1 is an example of map information according to an aspect of the present disclosure. [Figure 3] 1 is an exemplary exterior view of a vehicle according to an aspect of the present disclosure. [Figure 4] FIG. 1 is an illustration of an exemplary system according to aspects of the present disclosure. [Figure 5]FIG. 5 is a functional diagram of the system of FIG. 4 according to an embodiment of the present disclosure. [Figure 6] 1 is an example of a client computing device and displayed options according to an aspect of the present disclosure. [Figure 7] 1 is an example of a client computing device and displayed options according to an aspect of the present disclosure. [Figure 8] 1 is an example of a client computing device and displayed options according to an aspect of the present disclosure. [Figure 9] 1 is an example of map information according to an aspect of the present disclosure. [Figure 10] 1 is an example of map information according to an aspect of the present disclosure. [Figure 11] FIG. 1 is an exemplary flow diagram according to aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] overview This technology relates to facilitating the pickup and drop-off of passengers (or users) or cargo in autonomous vehicles using auditory and / or visual communications, or indeed any situation where a pedestrian needs to reach the vehicle. In many situations, an autonomous vehicle does not have a human driver who can communicate with a person to help them find the vehicle or the correct drop-off location (i.e., for pickup). As such, an autonomous vehicle can proactively attempt to communicate with a person in a useful and effective manner using various auditory and / or visual communications. For example, models can be generated to enable the vehicle to determine when to provide auditory and / or visual communications to a person and / or whether to surface an option for the person to do so.

[0010] To generate the model, a person may be provided with an option, for example, via an application on the person's computing device (e.g., a cell phone or other client computing device), to have the vehicle provide an auditory transmission. When the person uses the option, this data may be recorded. Each time the option is used, a message may be provided to the vehicle causing the vehicle to provide the transmission. This message may include information such as the date and time the request was generated, the type of transmission to be made, and the person's location. This message and other information may also be transmitted, for example, by the vehicle and / or the client computing device to a server computing system.

[0011] The messages and other information may then be processed by the server computing device to generate a model to enable the vehicle's computing device to better communicate with the person. For example, the model may be trained to indicate whether a particular type of communication is appropriate. If so, the type of communication may be made available as an option in an application on the person's client computing device and / or automatically generated by the vehicle's computing device.

[0012] To train the model, the person's location, other information, and map information can be used as training inputs, and the type of communication (from the message) can be used as the training output. The more training data used to train the model, the more accurate the model will be in determining when to provide a communication or options for providing a communication. The model can be trained to distinguish between situations where a visual communication is appropriate and situations where an auditory communication is appropriate.

[0013] In some cases, depending on the amount of training data available, the model may be trained for a specific purpose, for example, a model may be trained for a particular person or group of people based on characteristics of the person's or group's history of using the service.

[0014] The trained model can then be provided to one or more vehicle computing devices to enable them to better communicate with people. When a vehicle is approaching or waiting at a pickup or drop-off location, the vehicle computing device 110 can use the model to determine whether communication is appropriate, and if so, what type. This can be done, for example, based on the vehicle's environment, whether the person (or potential passenger) has a clear line of sight to the vehicle, or vice versa.

[0015] In one aspect, the model may be used to determine whether the options described above need to be surfaced in the application. Additionally, if the model output indicates that visual communication is more appropriate than auditory communication, the surfaced options may only allow visual communication. Similarly, if the model output indicates that auditory communication is more appropriate than visual communication, the surfaced options may only allow visual communication. In another aspect, rather than providing the user with options for auditory or visual communication, the model may be used to determine whether the vehicle should automatically communicate auditory or visually. Additionally or alternatively, the model output may be used to determine an initial action, and subsequent actions may be automatically performed depending on the initial action.

[0016] The user's response to the subsequent action may be used to build a model of escalated transmission. For example, the results may be tracked for each situation in which the subsequent action was used. This information may then be analyzed to identify patterns that increase the likelihood that the user will respond to the vehicle transmission and enter the vehicle more quickly. The model of escalated transmission may be trained to determine, based on the previous or initial action, what the next action should be to best facilitate the user reaching the vehicle. Again, the more training data used to train the model, the more accurately the model will determine how to escalate from the previous action. Similar to the first model, the trained model of escalated transmission may be provided to one or more vehicles to enable their computing devices to better communicate with people, including potential passengers.

[0017] The features described herein may enable autonomous vehicles to improve passenger pickup and drop-off. For example, a user can have the vehicle visually and / or audibly communicate with the user, either on their own or by prompting them to use surfaced options. This allows the user to more easily locate the vehicle. Additionally or alternatively, the vehicle may use the model to predetermine whether and how to communicate with the user, and how to escalate those communications over time.

[0018] Exemplary System 1, a vehicle 100 according to one embodiment of the present disclosure includes various components. While some embodiments of the present disclosure are particularly useful in connection with particular types of vehicles, the vehicle may be any type of vehicle, including, but not limited to, an automobile, a truck, a motorcycle, a bus, a recreational vehicle, etc. The vehicle may have one or more computing devices, such as a computing device 110, which may include one or more processors 120, a memory 130, and other components typically present in a general-purpose computing device.

[0019] Memory 130 stores information accessible by one or more processors 120, including instructions 134 and data 132 that can be executed or otherwise used by processor(s) 120. Memory 130 may be any type of memory capable of storing information accessible by a processor, including computing device-readable media or other media that store data that can be read using an electronic device, such as a hard drive, memory card, ROM, RAM, DVD or other optical disk, and other writable and read-only memory. Systems and methods may include different combinations of the above, whereby different portions of the instructions and data are stored on different types of media.

[0020] The instructions 134 may be any series of instructions executed directly (e.g., machine code) or indirectly (e.g., script) by a processor. For example, the instructions may be stored as computing device code on a computing device-readable medium. In that regard, the terms "instructions" and "program" may be used interchangeably herein. The instructions may be stored in object code format for direct processing by a processor, or in any other computing device language, including a script or a collection of independent source code modules that are interpreted on demand or pre-compiled. The functions, methods, and routines of the instructions are described in further detail below.

[0021] Data 132 may be retrieved, stored, or modified by processor 120 according to instructions 134. For example, although claimed subject matter is not limited to any particular data structure, data may be stored in a computing device register, in a relational database as a table with multiple different fields and records, an XML document, or a flat file. Data may also be formatted in any computing device-readable format.

[0022] The one or more processors 120 may be any conventional processor, such as a commercially available CPU or GPU. Alternatively, the one or more processors may be dedicated devices, such as an ASIC or other hardware-based processor. While FIG. 1 functionally illustrates the processor, memory, and other elements of computing device 110 as being within the same block, those skilled in the art will understand that a processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be housed within the same physical enclosure. For example, memory may be a hard drive or other storage medium located in a different enclosure than that of computing device 110. Thus, reference to a processor or computing device will be understood to include reference to a collection of processors or computing devices or memories that may or may not operate in parallel.

[0023] Computing device 110 may include all components typically used in connection with a computing device, such as the processor and memory described above, as well as user input devices 150 (e.g., a mouse, keyboard, touchscreen, and / or microphone) and various electronic displays (e.g., a monitor having a screen, or any other electrical device operable to display information). In this example, the vehicle includes one or more speakers 154 as well as an electronic display 152 to provide information or an audiovisual experience. In this regard, electronic display 152 may be located within the interior of vehicle 100 and may be used by computing device 110 to provide information to passengers within vehicle 100. In some cases, electronic display 152 may be an interior display visible to persons outside the vehicle through a vehicle window or other transparent vehicle enclosure and / or may be capable of projecting images through a window or other transparent vehicle enclosure to provide information to persons outside the vehicle. Alternatively, electronic display 152 may be an externally mounted display capable of projecting information to passengers within the vehicle (i.e., on the underside of a roof pod viewable through a glass roof) and / or providing information to persons outside the vehicle.

[0024] Computing device 110 also includes one or more wireless network connections 156 to facilitate communication with other computing devices, such as client and server computing devices, described in more detail below. Wireless network connections may include short-range communication protocols such as Bluetooth, Bluetooth Low Energy (LE), cellular connections, and various configurations and protocols including the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local network, a private network using one or more enterprise-specific communication protocols, Ethernet, WiFi, and HTTP, as well as various combinations of the above.

[0025] In one example, computing device 110 may be part of a communication system of an autonomous driving computing system integrated into vehicle 100. In this regard, the communication system may be included or configured to transmit a signal that causes an audible communication to be played through speaker 154. The communication system may also be configured to transmit a signal that causes a visual communication to be made, for example, by flashing or otherwise controlling the vehicle's headlights 350, 352 (shown in FIG. 3 ) or by displaying information on internal electronic display 152.

[0026] Autonomous control system 176 may include various computing devices, configured similarly to computing device 110, that can communicate with various components of the vehicle to control the vehicle in an autonomous driving mode. For example, returning to FIG. 1 , autonomous control system 176 may communicate with various systems of vehicle 100, such as deceleration system 160, acceleration system 162, steering system 164, routing system 166, planner system 168, positioning system 170, and perception system 172, to control the movement, speed, etc. of vehicle 100 in accordance with instructions 134 in memory 130 in an autonomous driving mode.

[0027] As an example, the computing devices of autonomous control system 176 may interact with deceleration system 160 and acceleration system 162 to control the speed of the vehicle. Similarly, steering system 164 may be used by autonomous control system 176 to control the direction of vehicle 100. For example, if vehicle 100 is configured for use on a road like a car or truck, the steering system may include components that control the angle of the wheels to turn the vehicle. Autonomous control system 176 may also use a traffic light system to signal the vehicle's intentions to other drivers or vehicles, for example, by activating turn signals or brake lights as needed.

[0028] Routing system 166 may be used by autonomous control system 176 to generate a route to a destination. Planner system 168 may be used by computing device 110 to follow a route. In this regard, planner system 168 and / or routing system 166 may store detailed map information, such as highly detailed maps that identify road shapes and elevations, lane lines, intersections, crosswalks, speed limits, traffic signals, buildings, signs, real-time traffic information, pull-over spot vegetation, or other such objects and information.

[0029] 2 is an example of map information 200 for a section of roadway including an intersection 202 adjacent to a parking lot 210 for a building 220. The map information 200 may be a local version of the map information stored in memory 130 of the computing device 110. Other versions of the map information may also be stored in storage system 450, discussed further below. In this example, the map information 200 includes information identifying the shape, location, and other characteristics of lane lines 230, 232, 234, 236, lanes 240, 242, 244, 246, stop signs 250, 252, 254, 256, etc. In this example, the map information 200 also includes information identifying features of the parking lot 210 and the building 220, such as parking spaces 260, 262, 264, 266, 268, drivable areas 270, 272, 274, 276, etc. Additionally, in this example, the map information identifies entrances and exits 282, 284, 286 of building 220. Although only a few features are shown in map information 200 in Figure 2, map information 200 may include significantly more features and details to enable vehicle 100 to be controlled in an autonomous driving mode.

[0030] Although map information is depicted herein as an image-based map, map information need not be entirely image-based (e.g., raster). For example, map information may include one or more road graphs, or graph networks of information such as roads, lanes, intersections, and connections between these features, which may be represented by road segments. Each feature may be stored as graph data and associated with information such as geographic location, whether or not linked to other related features, e.g., stop signs may be linked to roads and intersections, etc. In some examples, the associated data may include a grid-based index of the road graph to enable efficient searching of specific road graph features.

[0031] Positioning system 170 may be used by autonomous control system 176 to determine the vehicle's relative or absolute location on a map or Earth. For example, positioning system 170 may include a GPS receiver to determine the device's latitude, longitude, and / or altitude location. Other location systems, such as laser-based location systems, inertial-aided GPS, or camera-based location, may also be used to identify the vehicle's location. In addition to absolute geographic location information, such as latitude, longitude, and altitude, the vehicle's location may also include relative location information, such as its position relative to other vehicles in the immediate vicinity, which can often be determined with less noise than absolute geographic location.

[0032] Positioning system 170 may also include other devices in communication with computing device autonomous control system 176, such as an accelerometer, gyroscope, or another direction / speed detection device for determining the vehicle's direction and speed, or changes thereto. By way of example only, an acceleration device may determine the vehicle's pitch, yaw, or roll (or changes thereto) relative to the direction of gravity or a plane perpendicular to gravity. The device may also track increases or decreases in speed, and the direction of such changes. Providing device position and orientation data as described herein may be automatically provided to computing device 110, other computing devices, and combinations of the above.

[0033] Perception system 172 also includes one or more components for detecting objects external to the vehicle, such as other vehicles, obstacles in the road, traffic signals, signs, trees, etc. For example, perception system 172 may include lasers, sonar, radar, cameras, and / or any other detection devices that record data that can be processed by the computing device of autonomous control system 176. If the vehicle is a passenger vehicle, such as a minivan, the minivan may include lasers or other sensors mounted on the roof or other convenient location. For example, FIG. 3 is an exemplary exterior view of vehicle 100. In this example, roof-top housing 310 and dome-shaped housing 312 may include a LIDAR sensor and various camera and radar units. Additionally, housing 320 located at the front end of vehicle 100, and housings 330, 332 on the driver's and passenger's sides of the vehicle may each store a LIDAR sensor. For example, housing 330 is located in front of driver door 360. Vehicle 100 also includes housings 340, 342 for radar units and / or cameras also located on the roof of vehicle 100. Additional radar units and cameras (not shown) may be located at the front and rear ends of vehicle 100 and / or on the roof or other locations along roof-top housing 310.

[0034] Autonomous control system 176 may be capable of communicating with various components of the vehicle to control the movement of vehicle 100 in accordance with primary vehicle control code in the memory of autonomous control system 176. For example, returning to FIG. 1 , autonomous control system 176 may include various computing devices that communicate with various systems of vehicle 100, such as deceleration system 160, acceleration system 162, steering system 164, routing system 166, planner system 168, positioning system 170, perception system 172, and power system 174 (i.e., the vehicle's engine or motor), to control the movement, speed, etc. of vehicle 100 in accordance with instructions 134 in memory 130.

[0035] Various systems of the vehicle may function using autonomous vehicle control software to determine and control how to control the vehicle. As an example, the perception system software modules of perception system 172 may use sensor data generated by one or more sensors of the autonomous vehicle, such as a camera, LIDAR sensor, radar unit, or sonar unit, to detect and identify objects and their features. These features may include location, type, heading, orientation, speed, acceleration, change in acceleration, size, shape, etc. In some cases, the features may be input into a motion prediction system software module that uses various motion models based on the object type to output predicted future motion of the detected object. In other examples, the features may be input into one or more detection system software modules, such as a traffic light detection system software module configured to detect known traffic light conditions, a construction zone detection system software module configured to detect construction zones from sensor data generated by one or more sensors of the vehicle, and an emergency vehicle detection system software module configured to detect emergency vehicles from sensor data generated by the sensors of the vehicle. Each of these detection system software modules may use various models to output the likelihood that a construction zone or object is an emergency vehicle. Detected objects, predicted future behavior, various probabilities from the detection system software module, map information identifying the vehicle's environment, location information from positioning system 170 identifying the vehicle's location and orientation, the vehicle's destination, and feedback from various other systems in the vehicle may be input to a planner system software module in planner system 168. The planner system can use this input to generate a trajectory for the vehicle to follow over a short period of time in the future based on the route generated by the routing module in routing system 166. A control system software module in autonomous control system 176 may be configured to control the movement of the vehicle by, for example, controlling the braking, acceleration, and steering of the vehicle to follow the trajectory.

[0036] Autonomous control system 176 can control the vehicle in an autonomous driving mode by controlling various components. For example, autonomous control system 176 can use detailed map information and data from planner system 168 to navigate the vehicle to a destination fully autonomously. Autonomous control system 176 can determine the vehicle's location using positioning system 170 and detect and respond to objects using perception system 172 as needed to safely reach that location. Again, to do so, computing device 110 can generate trajectories, cause the vehicle to follow these trajectories, accelerate the vehicle (e.g., by providing fuel or other energy to engine or power system 174 via acceleration system 162), decelerate the vehicle (e.g., by reducing fuel provided to engine or power system 174, changing gears, and / or applying the brakes via deceleration system 160), change direction (e.g., by turning the front or rear wheels of vehicle 100 via steering system 164), and signal such changes (e.g., by activating a turn signal). Thus, acceleration system 162 and deceleration system 160 may be part of a drivetrain that includes various components between the vehicle's engine and the vehicle's wheels. Again, by controlling these systems, autonomous control system 176 may also control the vehicle's drivetrain in order to steer the vehicle autonomously.

[0037] Computing device 110 of vehicle 100 can also receive or transfer information to or from other computing devices, such as computing devices that are part of a transportation service as well as other computing devices. Figures 4 and 5 are pictorial and functional diagrams, respectively, of an exemplary system 400, which includes multiple computing devices 410, 420, 430, 440 and a storage system 450 connected via a network 460. System 400 also includes vehicle 100 and vehicles 100A and 100B, which may be configured the same as or similar to vehicle 100. For simplicity, only a few vehicles and computing devices are shown, although a typical system may include many more.

[0038] 5, each of computing devices 410, 420, 430, 440 may include one or more processors, memory, data, and instructions, which may be configured similarly to one or more processors 120, memory 130, data 132, and instructions 134 of computing device 110.

[0039] The network 460 and intermediary nodes may include a variety of configurations and protocols, including short-range communication protocols such as Bluetooth, Bluetooth LE, the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local network, a private network using one or more company-proprietary communication protocols, Ethernet, WiFi, and HTTP, as well as various combinations of the above. Such communication may be facilitated by any device capable of transmitting data to and from other computing devices, such as a modem and a wireless interface.

[0040] In one example, the one or more computing devices 410 may include one or more server computing devices having multiple computing devices, such as a load-balancing server farm, that exchange information with different nodes of a network for the purpose of receiving, processing, and transmitting data to and from other computing devices. For example, the one or more computing devices 410 may include one or more server computing devices that are capable of communicating with computing device 110 of vehicle 100, or a similar computing device of vehicle 100A, as well as computing devices 420, 430, and 440, via network 460. For example, vehicles 100 and 100A may be part of a fleet of vehicles that may be dispatched to various locations by the server computing device. In this regard, server computing device 410 may function as a dispatch server computing system that may be used to dispatch vehicles such as vehicle 100 and vehicle 100A to different locations to pick up and drop off passengers. Additionally, server computing device 410 may use network 460 to transmit and present information to users, such as users 422, 432, 442, on displays, such as displays 424, 434, 444, of computing devices 420, 430, 440. In this regard, computing devices 420, 430, 440 may be considered client computing devices.

[0041] 5, each client computing device 420, 430, 440 may be a personal computing device intended for use by a user 422, 432, 442 and may have all the components typically used in connection with a personal computing device, including one or more processors (e.g., a central processing unit (CPU)), memory (e.g., RAM and an internal hard drive) for storing data and instructions, a display such as displays 424, 434, 444 (e.g., a monitor with a screen, a touchscreen, a projector, a television, or other device operable to display information), and user input devices 426, 436, 446 (e.g., a mouse, keyboard, touchscreen, or microphone). The client computing devices may also include a camera for recording video streams, speakers, a network interface device, and all of the components used to connect these elements to each other.

[0042] Client computing devices 420, 430, and 440 may each comprise a full-sized personal computing device, or alternatively, a mobile computing device capable of wirelessly exchanging data with a server over a network such as the Internet. By way of example only, client computing device 420 may be a mobile phone, or a device such as a wireless-enabled PDA, tablet PC, wearable computing device or system, or netbook capable of obtaining information over the Internet or other network. In another example, client computing device 430 may be a wearable computing system depicted as a wristwatch, as shown in FIG. 4. As an example, a user may input information using a miniature keyboard, keypad, microphone, visual signals using a camera, or a touch screen.

[0043] Similar to memory 130, storage system 450 may be any type of computerized storage device capable of storing information accessible by server computing device 410, such as a hard drive, memory card, ROM, RAM, DVD, CD-ROM, writable memory, and read-only memory. In addition, storage system 450 may include a distributed storage system in which data is stored on multiple different storage devices that may be physically located in the same or different geographic locations. Storage system 450 may be connected to the computing devices via network 460, as shown in Figures 4 and 5, and / or may be directly connected to or incorporated into any of the computing devices 110, 410, 420, 430, 440, etc.

[0044] Storage system 450 may store various types of information. For example, storage system 450 may also store the autonomous vehicle control software described above that is used by a vehicle, such as vehicle 100, to operate the vehicle in an autonomous driving mode. This autonomous vehicle control software stored in storage system 450 includes various revoked and validated versions of the autonomous vehicle control software. Once validated, the autonomous vehicle control software may be sent to memory 130 of vehicle 100 to be used by the vehicle's computing device, for example, to control the vehicle in an autonomous driving mode.

[0045] Storage system 450 can store various types of information, as described in more detail below. This information can be retrieved or otherwise accessed by a server computing device, such as one or more server computing devices 410, to perform some or all of the features described herein. For example, the storage system can store various models and their parameter values, which can be updated through training, as described further below. Storage system 450 can also store log data. This log data can include, for example, sensor data generated by a perception system, such as perception system 172 of vehicle 100. A perception system can include multiple sensors that generate sensor data. By way of example, the sensor data can include not only raw sensor data but also data identifying defining characteristics of perceived objects (including other road users), such as the shape, position, orientation, and speed of objects, such as vehicles, pedestrians, bicycles, vegetation, curbs, lane markings, sidewalks, crosswalks, buildings, and the like. The log data may also include “event” data identifying the vehicle's environment and / or various types of auditory communications generated by the vehicle in response to requests, as described further below.

[0046] Exemplary Methods In addition to the operations described above and illustrated in the figures, various operations are described herein. It should be understood that the following operations do not have to be performed in the exact order described below. Rather, various steps may be processed in a different order or simultaneously, and steps may also be added or omitted.

[0047] To generate and train the model, a user of the service may be provided with an option, for example, via an application on the user's computing device (i.e., a mobile phone), to request that a vehicle provide a communication. In this regard, using the option may cause the vehicle to provide a communication. When the user uses the option, this data may be recorded. FIG. 6 is an example diagram of a client computing device 420, including options 610, 620 displayed on the display 424. In this example, option 610 may enable the client computing device to send a request, for example, via the network 460 or other wireless connection, to the vehicle to generate an audible communication by honking the horn or playing a corresponding sound through the speaker 154. Option 620 may enable the client computing device to send a request, for example, via the network 460 or other wireless connection, to the vehicle to generate a visual communication, for example, by flashing the headlights 350, 352 and / or displaying information on the electronic display 152. In some examples, option 630 may be provided to allow the user to not request a transmission, such as if the user is confident that they have identified their vehicle.

[0048] For example, a user may use option 620 to have the autonomous vehicle flash its headlights in a dark parking lot. As another example, in a well-lit parking lot with few other pedestrians, a user may use option 610 to have the vehicle honk the horn or provide some other audible communication. When more pedestrians are present, a user may select option 620 over option 610. As another example, a user may use option 610 to have the vehicle honk the horn when in a large parking lot or near a large building. As yet another option, a user may use option 620 to have the vehicle flash its headlights when there are multiple autonomous vehicles nearby. Alternatively, rather than flashing headlights, another type of visual communication option may be provided, such as displaying information on electronic display 152.

[0049] Each time one of the options, such as options 610, 620, is used to request a transmission, a message may be provided to the vehicle, causing the vehicle's computing device 110 to make or generate the transmission. This message may include information such as the date and time the request was made, the type of transmission to be made, and the user's location. This message, as well as other message information, may be sent, for example, by the vehicle and / or the user's client computing device to a server computing system, such as server computing system 410, which may store the message in storage system 450. By way of example, the other message information may include data generated by the vehicle's computing system, such as the vehicle's location, the type of transmission (flashing lights, displaying information on electronic display 152, honking the horn, etc.), the locations and / or characteristics of other road users (vehicles, pedestrians, cyclists, etc.) detected by the vehicle's perception system 172, ambient lighting conditions, etc.

[0050] By way of example, ambient lighting conditions may be determined in any number of different ways. For example, computing device 110 may receive feedback from the vehicle's light sensors, such as those used in some cases, such as electronic display 152, to control the state of the vehicle's headlights and adjust the brightness of interior electronic displays. If feedback from the light sensors is not directly available to computing device 110, this information may also be gleaned from the state of the vehicle's headlights and / or interior electronic displays. In other words, computing device 110 may be able to determine from this information whether the vehicle is "dark enough" to turn on its headlights or an interior electronic display at a particular brightness. Additionally, or alternatively, ambient lighting conditions may be determined from data generated by the vehicle's perception system. As previously mentioned, perception system 172 may include multiple different sensors, some of which may be used to determine ambient lighting conditions, such as a still camera or a video camera. For example, a "live" camera image of the vehicle's environment may be analyzed to determine ambient lighting conditions. This may include processing pixels to determine whether the area the camera is pointed at is a bright area. If a pixel is bright and the image has a short exposure time, this may indicate that the area is also bright. As another example, ambient lighting conditions may be determined in real time by using the camera's exposure value. As an example, when capturing an image, the perception system 172 camera may automatically recalibrate its exposure value given the ambient lighting conditions. In this regard, the exposure value may be considered a proxy for the brightness of the area currently viewable by the vehicle's camera. For example, the real-time exposure value may be used to determine the ambient lighting conditions. The longer the exposure value, the darker the scene, or rather, the lower the ambient lighting conditions. Similarly, the shorter the exposure value, the brighter the scene, or rather, the higher the ambient lighting conditions. Additionally, exposure values ​​for periods when the sun is not up (i.e., from dusk to dawn on any given day of the year) may be checked to identify those with short exposure times, which indicate brighter artificial lighting.

[0051] The messages and other message information, including sensor data, may then be processed by the server computing device 410 to generate and train a model. The model may be a machine learning model, such as a decision tree (such as a random forest decision tree), a deep neural network, a logistic regression, a neural network, etc. To train the model, the user's location, other message information (including sensor data generated by the perception systems 172 of the various vehicles that generated the messages), and map information may be used as training inputs, and the type of communication (from the message) may be used as a training output.

[0052] Thus, training may include receiving training data including various training inputs as well as training outputs or target outputs. The model can be trained with the training data using current values ​​of the model's parameters to generate a set of output values. These output values ​​may indicate a level of appropriateness of a type of transmission or other output data determined using the model. The target output and the set of output values ​​can be compared to each other to determine one or more difference values ​​indicating how far apart they are from each other. Based on the one or more difference values, the current values ​​of the model's parameters can be adjusted. Iterative training and adjustment can improve the accuracy of the model. Thus, the more training data used to train the model, the more accurate the model will be in determining whether and what type of transmission to automatically provide, or what type of transmission option to provide or enable, as described further below. Additionally, by using map information as training data, the model can be trained to incorporate how the environment, e.g., the type of road or area (e.g., residential or commercial) in which vehicles and / or pedestrians are located, affects a user's desire for transmission in determining what type of transmission or transmission option to provide or enable.

[0053] Additionally, by using time of day and / or ambient lighting conditions as training data, the model may be trained to distinguish between different times of day and lighting conditions for different types of communications output by the model. For example, ambient lighting conditions as well as time of day may be used as training inputs. Again, the more training data used to train the model, the more accurate the model may be in determining the timing and type of communications to enable and / or provide.

[0054] Additionally, through feedback and hand training, weights can be assigned (or generated) to various transmission options to reduce the likelihood of false positives or to indicate that the vehicle should generate a transmission at an inappropriate or inappropriate time. Such weights are likely to be highly dependent on environmental factors (e.g., map information and sensor data), and such inputs can influence the model's corresponding weighting factors. For example, when training a model, the model may weight more heavily against honking when pedestrians are within a short distance, such as one to two meters, from the vehicle, as this may be irritating to pedestrians.

[0055] Similarly, the model can be trained to distinguish between situations where visual communication is appropriate and when auditory communication is appropriate. For example, honking the horn (or playing a corresponding auditory communication via speaker 154) may be inappropriate (or more appropriate) in a crowded area, and using lights during the day may be inappropriate (or more appropriate). In this regard, user feedback regarding the effectiveness or usefulness of various communications can also be used to train the model. As an example, the user may provide feedback indicating whether a particular communication was inappropriate or inconvenient and why (e.g., whether a person was standing in front of the vehicle when the headlights flashed and could be painful to the person's eyes, whether the information displayed on the horn or electronic display was confusing to others in the vehicle's environment, etc.). Additionally, based on laws and regulations, auditory communication may or may not be more appropriate (e.g., it may be illegal to flash headlights or honk the horn in certain areas). Such examples of inappropriate, ineffective, inconvenient, or less useful communication may be generated and / or labeled (e.g., manually by a human operator) as inappropriate and / or used as training data. Thus, as described above, a model may be trained to output whether a communication is appropriate, and if so, the type of communication, or rather, whether the communication is auditory or visual. As an example, the model may identify a list of possible communication types and their corresponding levels of appropriateness. In some cases, users may provide positive and / or negative feedback about their experience. This information can also be used to help train the model to select the communication that the user deemed more appropriate and most helpful.

[0056] In some cases, depending on the amount of available training data, the model can be trained for a specific purpose. For example, a model may be trained for a particular user or type of user based on that user's or user's history of being picked up at a particular location. In this way, the model may enable the vehicle to provide advance notification to the user in situations where the vehicle needs to deviate from that user's typical pickup location. For example, if a user is normally picked up at one corner of a building but there is an obstruction (construction, parked vehicles, fallen tree debris, etc.) that forces the vehicle to move to another location, such as another corner of the building, the model may be trained to enable the vehicle to advance notification to the user via visual and / or auditory communication (such as honking the horn, flashing lights, etc., or by displaying information on the electronic display 152) to get the user's attention when the user is leaving the building. In this way, the vehicle can respond as needed. In some cases, the model may be trained to further notify the user via an application on the user's client computing device in conjunction with the visual and / or auditory communication.

[0057] The trained model, or rather the model and parameter values, can then be provided to one or more vehicles, such as vehicle 100, 100A, to enable the computing devices 110 of these vehicles to better communicate with people when the vehicles approach or wait at a pickup location (or drop off goods). The vehicle's computing device 110 can use the model to determine whether communication is appropriate and, if so, what type. This can be done, for example, based on the vehicle's environment and / or whether the user (or possible passenger) has a clear line of sight to the vehicle, or vice versa.

[0058] In one aspect, the model and parameter values ​​can be used to determine whether options such as those described above need to be surfaced to the application. For example, sensor data generated by the vehicle's perception system, local map information of the area surrounding the vehicle, and the vehicle's current location can be input into the model. The map information can include various relevant information, such as the distance to the nearest curb, staircase, entrance or exit, and / or whether the vehicle is in proximity to another object, such as a wall or tree, that may obstruct the vehicle's user's view. This determination can be made, for example, when the vehicle's computing device finds a location where the vehicle can stop and wait for the user, pulls into that location, and / or when the vehicle is already stopped (i.e., already parked). The model can then output a value indicating whether transmission is appropriate and the level of appropriateness of each type of transmission.

[0059] In one example, if the model output indicates that visual communication is more appropriate than auditory communication, the surfaced option may allow only visual communication. In other words, if the value indicating the appropriateness of visual communication is greater than the value indicating auditory communication, the surfaced option may allow only visual communication. For example, turning to Figure 7, option 620 for providing visual communication is not available, but option 610 for providing auditory communication is available.

[0060] Similarly, if the model output indicates that auditory communication is more appropriate than visual communication, the surfaced option may allow only visual communication. Again, in other words, if the value indicating the appropriateness of auditory communication is greater than the value indicating visual communication, the surfaced option may allow only visual communication. For example, turning to Figure 8, option 610 for providing auditory communication is unavailable, but option 620 for providing visual communication is available.

[0061] 9, which corresponds to map information 200 of FIG. 2, as an example, training data may indicate that when users exit building 220, e.g., via entrance and exit 286, they tend to stand near area 910 and request either an auditory or visual communication, e.g., via options 610 or 620. Thus, when the trained model is used, when users exit building 220, e.g., via entrance and exit 286, and are on a trajectory toward or near area 910 as tracked by their client computing device's GPS (and perhaps confirmed by pedestrian detection by vehicle perception system 172), the application may automatically surface an option to provide a communication (e.g., visual, auditory, or both), as in the examples of any of FIGS. 6, 7, and 8.

[0062] In another aspect, the model can be used to determine whether the vehicle should automatically issue an auditory communication rather than simply surfacing an option as described above. Again, this determination can be made, for example, when the vehicle's computing device finds a location where the vehicle will stop and wait for the user, pulls into that location, and / or when the vehicle is already stopped (i.e., already parked). For example, turning to FIG. 10 , training data indicates that when users exit building 220, e.g., via entrance and exit 282, they tend to stand near area 1010 and user option 610, causing their vehicle's computing device to honk the horn (or generate a corresponding auditory communication via speaker 154). Thus, when the trained model is used, vehicle's computing device 110 can automatically honk the horn (or generate a corresponding auditory communication via speaker 154) when users exit building 220 and are on a trajectory toward or near area 1010, as tracked by the GPS of their client computing device (and perhaps confirmed by pedestrian detection by the vehicle's perception system). In another example, training data may indicate that when a user exits building 220, for example, through entrances and exits 286, the user tends to stand near area 1020 and have the vehicle's computing device flash headlights 350, 352 using option 620. Thus, when the trained model is used, when a user exits building 220 through entrances and exits 286 and there are many pedestrians around if the user is on a trajectory toward or near area 1020 with many pedestrians around, the vehicle's computing device 110 may automatically flash headlights 350, 352 as tracked by the GPS of their client computing device (and perhaps confirmed by pedestrian detection by the vehicle's perception system).

[0063] In some cases, the vehicle's computing device can use information from the user's account information and / or other input from the user to determine the appropriate type of communication. For example, if the user's account information or other input indicates that the user has a disability (visual or hearing-related), the computing device may use this to "override" the model's output and / or as input to the mode. For example, a visually impaired person may benefit more from auditory communication. However, if there are many other people around, the system may prefer to provide instructions via the user's device rather than honking. Similarly, a hearing impaired person may benefit more from visual communication than auditory communication. Also, there may be higher thresholds for various parameters related to the distance the user needs to cover to reach the vehicle. For example, the vehicle's computing device should avoid instructing or encouraging a visually impaired person to cross roads or other pedestrian-unfriendly areas (heavy traffic) to reach the vehicle.

[0064] Additionally or alternatively, the output of the model can be used to determine and execute an initial action, and subsequent actions can be automatically executed depending on the initial action. Again, this determination can be performed, for example, when the vehicle's computing device finds a location where the vehicle will stop and wait for the user, pulls into that location, and / or when the vehicle is already stopped (i.e., already parked). For example, when the user exits the building 220 and approaches the area 1020, the vehicle's computing device 110 can automatically flash the headlights 350, 352. If there is no immediate change in the user's trajectory (e.g., toward the vehicle), an option such as option 610 can be surfaced via the user's client computing device to enable the user to cause the vehicle's computing device 110 to honk the horn (or generate a corresponding auditory communication via the speaker 154), as in the example of FIG. 7. As another example, when the user exits the building 220 and approaches the area 1020, the vehicle's computing device 110 can automatically flash the headlights 350, 352. If there is no immediate change in the user's trajectory (e.g., toward the vehicle), the vehicle's computing device 110 may automatically honk the vehicle horn (or generate a corresponding auditory communication via speaker 154). In some cases, in addition to automatically honking the horn, an option such as option 610 may also surface to allow the user to have the vehicle honk the vehicle horn (or generate a corresponding auditory communication via speaker 154), as shown in the example of FIG. 7. Alternatively, rather than surfacing an option, the vehicle may display a notification informing the user that the vehicle is honking the vehicle horn. At least initially, these subsequent actions can be selected randomly or by using human-tuned heuristics. In some cases, these heuristics may involve a response to specific audio cues or other information about the vehicle's environment (e.g., other message information). For example, if there is a lot of ambient noise, a loud audio communication may serve as an initial action.

[0065] The user's response to the subsequent action can be used to build a model of escalated transmission. The model of escalated transmission can be a machine learning model, such as a decision tree (e.g., a random forest decision tree), a deep neural network, a logistic regression, or a neural network. For example, for each situation in which a subsequent action is used, the results can be tracked. This information can then be analyzed, for example, by the server computing device 410, to train the model of escalated transmission, thereby identifying patterns that increase the likelihood that a user will respond to a vehicle transmission and enter the vehicle more quickly. For example, this analysis can include both the "time to board" + "time / consistency of trajectory changes" in response to the transmission. As an example, if a typical (or average) user takes N seconds to board when exiting the building 220, however, if the vehicle's computing device 110 provides an auditory transmission, this reduces to, for example, N / 2, resulting in a significant improvement in the vehicle's discoverability. The same is true for trajectory changes. If the user is generally moving away from the vehicle in the building 220 but eventually finds the vehicle in the average case, ideally the time between the vehicle providing the auditory communication and the user correcting their trajectory towards the vehicle could be significantly improved.

[0066] A model for escalated transfers can then be trained to determine what the next action should be to best facilitate the user reaching the vehicle based on the previous or initial action. As an example, the model can be trained using the user's orientation. For example, training inputs to the model can include the actual time it takes the user to reach and / or board the vehicle, which actions the user utilized over time, and the user's original orientation. These combinations can indicate whether an escalated transfer, such as a second or third transfer initiated by the user, shortened the boarding time by correcting the user's orientation when the user triggered the action. Thus, if a user exits a building heading north (when the vehicle is actually heading in the opposite direction, south in this case), the aforementioned option can be used to have the vehicle honk its horn, and then changing direction toward the vehicle. The model can be trained to have the vehicle honk its horn earlier when tracking the user exiting the building and heading north. Similarly, if the first action does not cause the user to change orientation, the model of escalated propagation can be used to determine a second propagation, a third propagation, etc., as needed, based on the user's response (e.g., a change of orientation, etc.) Again, the more training data used to train the model, the more accurate the model will be in determining how to escalate from the previous action.

[0067] As an example, the escalated communication model may be trained so that, for a user who has exited building 220 and is standing in area 1010, the vehicle first flashes its lights without a response from the user, after which the vehicle automatically honks its horn (or generates a corresponding auditory communication via speaker 154). If there is no immediate change in the user's trajectory (e.g., toward the vehicle), the vehicle's computing device 110 may automatically page a customer service representative, such as user 442 using computing device 440. The representative may be able to communicate with and direct the user to the vehicle by using sensor data generated by and received from the vehicle's perception system 172, the vehicle's location generated by and received from the vehicle's positioning system 170, and the user's location generated by and received from the user's client computing device.

[0068] As another example, the escalated transmission model may be trained such that for a user exiting building 220 and standing in area 1020, the vehicle should then automatically honk the vehicle horn three times while waiting each time to see if there is a change in the user's trajectory. As another example, the escalated transmission model may be trained such that for a user exiting building E at night, the vehicle's computing device 110 can always automatically call a customer service representative rather than surfacing an option.

[0069] Like the first model, the trained model of escalated transmission may then be provided to one or more vehicles, such as vehicles 100, 100A, to enable the computing devices 110 in those vehicles to better communicate with people.

[0070] In addition to using messages and other information to train models, the data can be analyzed to make pickups and drop-offs easier. For example, if a user typically is located in area 1010 for pickups and typically uses the option to activate the vehicle's horn when the vehicle is 1020, this can be used to cause the vehicle to stop near the location in area 1010.

[0071] FIG. 11 is an example flow diagram 1100 according to aspects of the present disclosure that may be executed by one or more processors of one or more computing devices, such as processor 120 of computing device 110, to facilitate communication from an autonomous vehicle to a user.

[0072] As shown in block 1110, while attempting to pick up a user by a vehicle, before the user enters the vehicle, the vehicle's current location and map information are input into a model to identify a type of communication action to communicate the vehicle's location to the user. This may include the model and / or the model of escalated communication described above. Thus, as described above, the model can output whether communication is appropriate and, if so, the type of communication, or rather, whether the communication is auditory or visual.

[0073] At block 1120, a first communication is enabled based on the type of communication action, which may include, for example, surfacing an option as described above and / or automatically generating an audio or visual communication as described above.

[0074] In block 1130, after enabling the first transmission, whether the user is moving toward the vehicle is determined from the received sensor data. In other words, whether the user responded to the first transmission may be determined. This sensor data may include sensor data generated by the vehicle's perception system 172 and / or sensor data from the user's client computing device. From this sensor data, the vehicle's computing device may determine, for example, whether the user is moving toward the vehicle, facing toward the vehicle, and / or whether the user has changed orientation to move toward the vehicle.

[0075] In block 1140, the second communication is enabled based on a determination of whether the user is moving toward the vehicle. As an example, the second communication may be enabled when the user is not moving toward the vehicle or changing their heading or orientation to move toward the vehicle in response to the activation of the first communication. Activation may include, for example, surfacing an option as described above and / or automatically generating an audio or visual communication as described above.

[0076] The features described herein may enable autonomous vehicles to improve passenger or user pickup and drop-off. For example, a user can have the vehicle visually and / or audibly communicate with the user, either on their own or by prompting them to use a surfaced option. This allows the user to more easily locate the vehicle. Additionally or alternatively, the vehicle may use the model to predetermine whether and how to communicate with the user, as well as how to escalate those communications over time.

[0077] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive but may be implemented in various combinations to achieve unique advantages. Because these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the foregoing description of embodiments should be considered as illustrative, and not limiting, of the subject matter defined by the claims. Additionally, the examples described herein, as well as the provision of phrases such as "such as," "including," and the like, should not be construed as limiting the subject matter of the claims to any particular example; rather, the example is intended to illustrate only one of many possible embodiments. Furthermore, the same reference numbers in different drawings may identify the same or similar elements.

Claims

1. 1. A computer-implemented method comprising: receiving, by one or more processors of a computing device, a request from a client computing device associated with a user to assist the user in reaching an autonomous vehicle; determining, by the one or more processors, a state within an environment of the autonomous vehicle; selecting, by the one or more processors, a type of communication from a plurality of types of communication that can be generated by the autonomous vehicle to assist the user in reaching the autonomous vehicle based on the determined condition; and initiating, by the one or more processors, generation of the selected type of transmission by the autonomous vehicle.

2. The computer-implemented method of claim 1 , wherein the selecting is performed using a trained model.

3. The computer-implemented method of claim 2 , wherein the model is trained to identify patterns that increase the likelihood that the user will reach the autonomous vehicle more quickly in response to a vehicle communication.

4. 4. The computer-implemented method of claim 3, wherein the pattern indicates one or more communications previously initiated by the user to minimize the time it takes the user to reach the autonomous vehicle.

5. The computer-implemented method of claim 1 , wherein the selected type of communication is generated by flashing headlights of the autonomous vehicle.

6. The computer-implemented method of claim 1 , wherein the determined conditions include ambient lighting conditions.

7. The computer-implemented method of claim 6 , wherein the ambient lighting conditions indicate whether the environment is dark enough for use.

8. The computer-implemented method of claim 6 , wherein the ambient lighting conditions are determined based on feedback from a light sensor of the autonomous vehicle.

9. 7. The computer-implemented method of claim 6, wherein the ambient lighting conditions are determined based on at least one of a state of the autonomous vehicle's headlights or a state of an internal electronic display of the autonomous vehicle.

10. The computer-implemented method of claim 6 , wherein the ambient lighting conditions are determined based on data generated by a perception system of the autonomous vehicle.

11. The computer-implemented method of claim 1 , wherein the selected type of communication is generated by honking the horn of the autonomous vehicle.

12. The computer-implemented method of claim 1 , wherein the selected type of communication is selected further based on time of day.

13. 10. The computer-implemented method of claim 1, wherein the selected type of communication is generated by displaying information on a display mounted externally to the autonomous vehicle.

14. The computer-implemented method of claim 1 , wherein the selected type of communication is produced by one or more speakers of the autonomous vehicle.

15. 2. The computer-implemented method of claim 1, further comprising: enabling the one or more processors to output one or more options on the client computing device of the user to enable the user to have the autonomous vehicle generate the selected type of transmission.

16. The computer-implemented method of claim 1 , wherein the selecting occurs in response to receiving the request.

17. 1. A computer-implemented method comprising: receiving, by one or more processors of a computing device, a request from a client computing device associated with a user to assist the user in reaching an autonomous vehicle; selecting, by the one or more processors, a type of communication from a plurality of types of communication capable of being generated by the autonomous vehicle to assist the user in reaching the autonomous vehicle; determining, by the one or more processors, whether the user is moving toward the autonomous vehicle based on received sensor data while attempting to pick up the user and before the user enters the autonomous vehicle; In response to determining, by the one or more processors, that the user is moving toward the autonomous vehicle, initiating generation of the selected type of communication by the autonomous vehicle; 20. A computer-implemented method comprising:

18. The computer-implemented method of claim 17 , wherein the received sensor data includes location information generated by the client computing device of the user.

19. 20. The computer-implemented method of claim 17, wherein the received sensor data comprises data generated by a perception system of the autonomous vehicle, the perception system comprising at least one sensor.

20. 1. A computer-implemented method comprising: receiving, by one or more processors of a computing device, a request from a client computing device associated with a user to assist the user in reaching an autonomous vehicle; selecting, by the one or more processors, a type of communication from a plurality of types of communication capable of being generated by the autonomous vehicle to assist the user in reaching the autonomous vehicle; determining, by the one or more processors, while attempting to pick up the user and before the user enters the autonomous vehicle based on received sensor data, whether the user is moving toward a particular area to await the arrival of the autonomous vehicle; upon determining by the one or more processors that the user is moving toward a particular area to wait for the autonomous vehicle, enabling output of one or more options on a client computing device of the user to enable the user to cause the autonomous vehicle to generate the communication; initiating, by the one or more processors, generation of the selected type of transmission by the autonomous vehicle; 20. A computer-implemented method comprising:

21. 21. The computer-implemented method of claim 20, wherein the received sensor data includes location information generated by the client computing device of the user.

22. 21. The computer-implemented method of claim 20, wherein the received sensor data comprises data generated by a perception system of the autonomous vehicle, the perception system comprising at least one sensor.

23. 1. A system comprising: one or more processors, wherein the one or more processors: receiving a request from a client computing device associated with a user to assist the user in reaching an autonomous vehicle; determining a state within an environment of the autonomous vehicle; selecting a type of communication from a plurality of types of communication that can be generated by the autonomous vehicle to assist the user in reaching the autonomous vehicle based on the determined condition; initiating generation of the selected type of transmission by the autonomous vehicle; and A system that is configured to:

24. 24. The system of claim 23, wherein the one or more processors are further configured to make the selection in response to receiving the request.

25. 1. A system comprising: one or more processors, wherein the one or more processors: receiving a request from a client computing device associated with a user to assist the user in reaching an autonomous vehicle; selecting a type of communication from a plurality of communication types capable of being generated by the autonomous vehicle to assist the user in reaching the autonomous vehicle; While attempting to pick up the user and before the user enters the autonomous vehicle, determining whether the user is moving toward the autonomous vehicle based on received sensor data; Initiating generation of the selected type of communication by the autonomous vehicle in response to determining that the user is moving toward the autonomous vehicle; A system that is configured to:

26. 1. A system comprising: one or more processors, wherein the one or more processors: receiving a request from a client computing device associated with a user to assist the user in reaching an autonomous vehicle; selecting a type of communication from a plurality of communication types capable of being generated by the autonomous vehicle to assist the user in reaching the autonomous vehicle; While attempting to pick up the user and before the user enters the autonomous vehicle, determining based on received sensor data whether the user is moving toward a specific area to await the arrival of the autonomous vehicle; Upon determining that the user is moving toward a particular area to wait for the autonomous vehicle, enabling output of one or more options on the user's client computing device to enable the user to cause the autonomous vehicle to generate the communication; initiating generation of the selected type of transmission by the autonomous vehicle; and A system that is configured to:

Images