Vehicle Door Interface Interaction

Abstract

Techniques for interacting with authorized and unauthorized users via a door interface component are discussed herein. A vehicle computing device can implement a door interface component and / or an authentication component to control the operation of a vehicle door or initiate communication for assistance. For example, the door interface component can include a button that provides different visual indicators and functionality based on whether the user is authorized to enter the autonomous vehicle (e.g., whether to open the door or provide a visual indicator for the user to select to open the door) or whether to request to move the vehicle, initiate a vehicle rental, or call for help if the user is not authorized to enter the autonomous vehicle.

Description

[Technical field]

[0001] The present disclosure relates to vehicle door interface interaction. [Background technology]

[0002] Related Applications

[0001] This application claims priority to U.S. patent application Ser. No. 17 / 507,720, filed Oct. 21, 2021, entitled "Vehicle Door Interface Interaction," the entire contents of which are incorporated herein by reference.

[0003]

[0002] When using ride-sharing services such as taxis, the driver may not only interact with the user for more complex interactions, but may also limit or grant input and output permissions to the user. In situations where a human driver is not present, for example in the case of an autonomous vehicle, pedestrians or potential passengers may not otherwise be able to interact with the driver for such more complex mutual interactions. [Brief description of the drawings]

[0004] The detailed description will be set forth with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the drawing in which that reference number first appears. Use of the same reference number in different drawings indicates similar or identical items or features. [Figure 1] FIG. 1 illustrates an example environment in which an example door interface component determines actions for an example vehicle. [Diagram 2]

[0005] FIG. 2 is a diagram illustrating an example of a door interface component that implements exemplary interaction techniques as described herein. [Figure 3A]

[0006] FIG. 3A illustrates an example of a visual indicator of an example door interface component for implementing an example interaction technique as described herein. [Figure 3B]

[0007] FIG. 3B illustrates an example configuration of example door interface components for implementing example interaction techniques as described herein. [Figure 3C] FIG. 3C illustrates an example configuration of example door interface components for implementing example interaction techniques as described herein. [Figure 3D] FIG. 3D illustrates an example configuration of example door interface components for implementing example interaction techniques as described herein. [Figure 4]

[0008] 11A-11C illustrate additional example configurations of exemplary door interface components for implementing exemplary interaction techniques as described herein. [Diagram 5]

[0009] FIG. 1 illustrates an example configuration of example door interface components for implementing example interaction techniques as described herein. [Figure 6]

[0010] FIG. 6 is a block diagram illustrating an example system for implementing the techniques described herein. [Figure 7]

[0011] FIG. 7 is a flow chart illustrating an example process for determining an interaction using an example door interface component. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005]

[0012] As discussed above, existing vehicles typically have handles, buttons, or other opening mechanisms to control operation of the doors for entry and exit by passengers or other authorized users while preventing or restricting unauthorized users from boarding or accessing the autonomous vehicle. However, in some instances, it may be desirable for the vehicle to interact with users (whether authorized or not) for more complex scenarios, such as allowing unauthorized users to initiate vehicle hiring, communicate an emergency, and / or request that the vehicle be removed from the path of another vehicle.

[0006]

[0013] The present application relates to techniques for interacting with authorized and unauthorized users by a vehicle door interface component. The techniques can include a vehicle computing device implementing a door interface component for responding differently to a user based on whether the user is authorized to interact with an autonomous vehicle. For example, the door interface component can be coupled to a door of an autonomous vehicle and can include a control, such as a button or a sensor, for interacting with a user. In various examples, after receiving an input associated with the control, the door interface component can operate the door (e.g., open or close the door) based on the user being associated with the autonomous vehicle (e.g., the user has been authenticated to use the autonomous vehicle, e.g., based on being verified as an authorized user, for a fee, etc.). In some examples, based on the user not being associated with the autonomous vehicle (e.g., the user is unauthenticated and / or has not initiated a request for a ride), the door interface component can output instructions to move the autonomous vehicle, authorize the user, and / or initiate communications with a remote operator or emergency personnel, just to name a few. Thus, control of the door interface components may be implemented differently depending, among other things, on whether a user has access to or is authorized to use the vehicle. By implementing the techniques described herein, interactions with the vehicle can be accurately and efficiently processed and identified, thereby improving the overall safety of the vehicle.

[0007]

[0014] In general, a door interface component implemented by a vehicle computing device may provide functionality to determine an interaction between the vehicle and a human in response to receiving input from a button, sensor, or other control associated with the door. In some examples, the vehicle may be operated as a ride service to transport authorized users to a destination, and the door interface component may determine a first set of operations (e.g., open one or more doors and / or direct the user to one of multiple doors that is safest for boarding) based on input received from the authorized user or from a device or token associated with the authorized user (e.g., the user's mobile device). In various examples, upon determining that the user is unauthorized, the door interface component may determine a second set of operations for interacting with the unauthorized user. By way of example and not limitation, the second set of operations may include moving the vehicle (e.g., to avoid blocking another vehicle, roadway, etc.), initiating communication (e.g., contacting a remote operator or an emergency service provider such as police, fire, medical), initiating an authorization sequence, etc. In this manner, the vehicle may be controlled in the environment based at least in part on input from the buttons of the door interface component, regardless of whether the input is associated with an authorized user (e.g., an authenticated user) or an unauthorized user (e.g., an unauthenticated user in proximity to the vehicle).

[0008]

[0015] By way of example and not limitation, a vehicle may be parked waiting to pick up a user (e.g., a passenger) who has rented the vehicle via an application on a mobile device (or other rental technology). The vehicle may detect that a user is nearby based at least in part on location services (e.g., based on GPS) that track the user's phone, signals (e.g., Bluetooth, Zigbee, WiFi, ultrasonic, etc.) transmitted between the user's phone and the vehicle, near field communication (NFC) when the user waves the phone near the door interface component, and / or biometric authentication (e.g., facial recognition, iris recognition, fingerprint recognition, etc.). If an authenticated passenger interacts with a button, sensor, or other control of the door interface component, the door may be opened, or if the door closest to the passenger is unsafe to enter due to an obstruction (or other reason), the door interface component may output a visual indicator (e.g., image, text, video, etc.) and / or an audio prompt to direct the passenger to another door on the other side of the vehicle.

[0009]

[0016] In some examples, a vehicle may be parked and a person not authorized to enter the vehicle may interact with the button. In such examples, the door interface component may provide functionality to present options to initiate movement of the vehicle (e.g., move the vehicle to another location, initiate a call to a remote operator to determine whether and / or where to move the vehicle, etc.), initiate an operation that allows the user to rent the vehicle (e.g., by initiating a call to a remote operator, requesting and / or receiving destination and / or payment information), and / or send a request for assistance (e.g., sending a contact to an emergency services provider). In this manner, the door interface component enables safer operation of the vehicle by providing different outputs or interactions depending on the authentication state of the user interacting with the button.

[0010]

[0017] In another example, an occupant may be inside the vehicle and a person outside the vehicle becomes aware that the occupant needs help (e.g., due to a medical condition or other reason). In such an example, the person can press a button on a door interface component to initiate an emergency communication that is sent to the occupant, an emergency service provider, and / or a remote computing device associated with the vehicle. Using the techniques described herein, the vehicle can provide assistance to the occupant within the vehicle based on receiving an input at a door interface component outside the vehicle.

[0011]

[0018] In some examples, the techniques described herein may include a vehicle computing device authenticating a user interacting with a control of a door interface component. The control may be a button having a first position and a second position, and the interaction may include a user pressing the button to cause a change between the first position and the second position. In some examples, the interaction with the control may include a sensor (e.g., a camera, an ultrasonic sensor, an infrared sensor, etc.) of the door interface component to detect the presence of a user. When the button is pressed or when a user is detected near the door by the sensor, the door interface component may send a request for the user's authentication status to the vehicle computing device. In some examples, the vehicle computing device may send a communication to the door interface component indicating that the user is associated with an authenticated state (e.g., authorized to enter the vehicle) or that the user is associated with an unauthenticated state (e.g., not authorized to enter the vehicle). In examples where a user is associated with an authenticated state, the door interface component may output instructions, visual indicators, and / or communications to operate the doors of the autonomous vehicle between a first position (e.g., a closed position) and a second position (e.g., an open position). In examples where a user is associated with an unauthenticated state, the door interface component may output different instructions, visual indicators, and / or communications to authorize the user, move the vehicle from its current location, and / or call emergency assistance, just to name a few.

[0012]

[0019] As mentioned, the door interface component may determine a first set of visual indicators to communicate options to the user if the user is authenticated (e.g., door open indicator, use of other door indicator, door lock / unlock indicator, etc.). For example, the user may be authorized to enter the vehicle, but the door associated with the input may be unsafe to enter (e.g., stationary or moving and within a threshold distance of another object that may intersect with the user or the door if the user attempts to use the door). To allow the user to safely enter the vehicle, the door interface component may output an indicator that the door is locked while also outputting an indicator for the user to use another door to enter the vehicle. Additional examples of visual door indicators are described throughout this disclosure in connection with FIGS. 1-7.

[0013]

[0020] The door interface component may determine a second set of visual indicators (e.g., a communication indicator, a moving vehicle indicator, a door lock / unlock indicator, etc.) to communicate options to the user if the user is unauthenticated. In such an example, one or more visual indicators may be presented in or adjacent to a button that enables the user to rent a vehicle (initiate the renting process in the vehicle and / or use a remote operator), request a moving location for the vehicle (e.g., navigate to another location in the environment), and / or request that a communication be sent to a remote operator and / or emergency personnel. Based on interaction with the control and the currently presented visual indicators, the door interface component may enable the control (e.g., a single button with two positions) to initiate multiple different types of behaviors or actions.

[0014]

[0021] In some examples, authenticating a user can include an authentication component that verifies a current authentication state of a user interacting with a control. Functionality associated with the authentication component can be included in the vehicle computing device and / or the door interface component 104. The authentication state can include an authenticated state or an unauthenticated state. The authenticated state can be based on the proximity of the user's mobile device to the vehicle, an application on the user's mobile device to the vehicle, a machine-readable code, or a biometric authentication, just to name a few. That is, the authentication component of the vehicle computing device can determine whether to approve a user to enter the vehicle as authenticated based on the distance between the user and the vehicle being within a threshold distance (e.g., a sensor that detects the user's mobile device within 3 meters of the vehicle). The authentication component can also approve a user to use the vehicle, or to interact with a machine-readable code (e.g., a Quick Response (QR) code, a barcode, a radio frequency identification (RFID) tag, etc.), or a biometric authentication (e.g., face recognition, iris recognition, fingerprint, etc.) based on the user using an application on a mobile device or other computing device. Additional examples of authentication components that determine authentication state are described throughout this disclosure, including in the following drawings.

[0015]

[0022] In some examples, the vehicle may include an autonomous or semi-autonomous vehicle having a vehicle computing device configured to receive sensor data from one or more sensors of the vehicle. The vehicle may detect objects using the one or more sensors while navigating within the environment. The objects may include static objects (e.g., ground surfaces, buildings, bridges, signs, etc.) and dynamic objects such as other vehicles (e.g., cars, trucks, motorcycles, mopeds, etc.), pedestrians, cyclists, etc. In some examples, the objects may be detected based on sensor data from sensors of the vehicle (e.g., cameras, motion detectors, lidar sensors, radar sensors, etc.). As yet another example, the objects may be detected based on sensor data received from a remote sensor, such as, for example, a sensor associated with another vehicle or a sensor located within the environment configured to share data with multiple vehicles. The sensor data representing the detected object may be used to determine the presence of a user adjacent to the door interface component.

[0016]

[0023] The techniques described herein can improve the functionality of a vehicle's computing device in many ways. For example, a door interface component of a vehicle computing device can receive input at a control, process the input, and cause different vehicle actions based on the authorization state of the operator of the control. In some examples, the door interface component improves the functionality and safety of the vehicle by allowing the vehicle to navigate to a safer location to pick up an authorized user, or by responding to an unauthorized user requesting that the vehicle move out of blocking a person or vehicle. In addition, the techniques described herein can improve occupant comfort and / or vehicle safety, such as indicating the safest door to enter from among multiple doors based on sensor data. In some examples, a single button can achieve multiple different functions, thereby reducing the number of buttons or other controls required. Utilizing the determinations made by the door interface component, the vehicle computing device can improve the accuracy and / or reduce the latency for the vehicle to respond to potential emergency situations in the environment.

[0017]

[0024] In various examples, the vehicle computing device can improve vehicle safety by predicting a vehicle trajectory based at least in part on instructions or communications output by the door interface component. For example, the planning component of the vehicle computing device can determine a vehicle trajectory that is most likely to avoid an object based on the door interface component sending a communication to the planning component indicating that a door is blocked or otherwise unsafe to enter.

[0018]

[0025] The techniques described herein can be implemented in many ways. Exemplary implementations are provided below with reference to the following drawings. Although described in the context of an autonomous vehicle, the methods, apparatus, and systems described herein can be applied to a variety of systems and are not limited to autonomous vehicles. In another example, the techniques can be utilized in an aviation or nautical context, or in any system that uses sensor data. Additionally, the techniques described herein can be used with real data (e.g., captured using sensors), simulated data (e.g., generated by a simulator), or any combination of the two.

[0019]

[0026] FIG. 1 illustrates an example environment 100 in which an example door interface component determines actions for an example vehicle. For example, a vehicle 102 can include a vehicle computing device (e.g., vehicle computing device 604 of FIG. 6) and / or a remote computing device (e.g., computing device 636 of FIG. 6) that implements the door interface component 104. Although described as separate systems, in some examples, the interaction techniques described herein may be implemented by other vehicle systems, components, and / or computing devices. For example, as described in further detail with respect to FIG. 6, the interaction techniques described herein may be implemented at least in part or in association with a localization component 620, a perception component 622, a prediction component 624, and / or a planning component 626.

[0020]

[0027] In some examples, the door interface component 104 can include a control 106 including a visual indicator 108 and a visual indicator 110, a camera 112, an audio interface 114 (e.g., a speaker and / or a microphone), infrared sensors 116(1) and 116(2), a light emitting diode (LED) 118, and a communication component 120 (e.g., an antenna, a transceiver, etc.).

[0021]

[0028] The vehicle 102 may be an unmanned vehicle, such as an autonomous vehicle configured to operate according to the Level 5 classification issued by the National Highway Traffic Safety Administration. The Level 5 classification describes a vehicle that can perform all safety-critical functions throughout the entire journey, with no driver (or passenger) expected to control the vehicle at any time. In such an example, the vehicle 102 may not include a driver and / or controls for manually driving the vehicle 102, such as a steering wheel, accelerator pedal, and / or brake pedal, since the vehicle 102 may be configured to control all functions from the beginning to the completion of the journey, including all parking functions. This is by way of example only, and the systems and methods described herein may be incorporated into any surface, air, or water vehicle, ranging from vehicles that must be manually controlled by a driver at all times to vehicles that are partially or fully autonomously controlled.

[0022]

[0029] The exemplary vehicle 102 can be any configuration of vehicle, such as, for example, a van, a sport utility vehicle, a crossover vehicle, a truck, a bus, an agricultural vehicle, and / or a construction vehicle. The vehicle 102 can be powered by one or more internal combustion engines, one or more electric motors, hydrogen power, any combination thereof, and / or any other suitable power source. The exemplary vehicle 102 has four wheels, but the systems and methods described herein can be incorporated into vehicles having a fewer or greater number of wheels, tires, and / or tracks. The exemplary vehicle 102 can have four-wheel steering and can generally operate with equal performance characteristics in all directions. For example, the vehicle 102 may be configured such that a first end of the vehicle 102 is the front end of the vehicle 102 and an opposite second end of the vehicle 102 is the rear end when traveling in a first direction, and such that the first end becomes the rear end of the vehicle 102 and the second end of the vehicle 102 is the front end of the vehicle 102 when traveling in the opposite direction. Stated differently, the vehicle 102 may be a bi-directional vehicle capable of traveling forward in either of the opposite directions. These example features may facilitate better maneuverability in tight spaces or crowded environments, such as, for example, parking lots and / or city streets.

[0023]

[0030] The example vehicle 102 may include a door 122 (e.g., an opening for occupants to enter and exit) on at least one side of the vehicle. An individual door may be comprised of any number of sections, panels, or portions that can be moved to create an opening for occupants. In some examples, the vehicle 102 may be configured with a door on each side of the vehicle. A door interface component 104 may be included in each door or in a panel of the vehicle adjacent to each door to provide the interaction techniques described herein.

[0024]

[0031] In some examples, the door interface component 104 can receive an input at the control 106 and determine an authentication state of a user associated with the input. For example, an authentication component (not shown) associated with the vehicle computing device or included as part of the door interface component 104 can determine an authentication state (e.g., authenticated or unauthenticated) based on receiving an indication of the input from the door interface component 104. In response to receiving the authentication state from the authentication component, the door interface component 104 can output different instructions to control the vehicle 102 (e.g., send instructions to another component of the vehicle computing device), communicate with the user, and / or communicate with a computing device remote from the vehicle 102 (e.g., a device of a remote operator capable of authorizing the user or a device associated with emergency services).

[0025]

[0032] In some examples, the door interface component 104 may be coupled to the door 122 of the vehicle 102. The control 106 may represent a button, such as a mechanical button, having a first position and a second position. In examples where the control 106 comprises a single button having two positions, the door interface component 104 may determine different types of interactions with the user based on the authentication status of the user. As discussed herein, the door interface component 104 may provide more complex functionality than is typical of a two-position button. In particular, the door interface component 104 may determine an action based at least in part on the visual indicators 108, the visual indicators 110, and / or the authentication status of the user. For example, an unauthorized user may receive a first set of visual indicators (e.g., the visual indicators may be green to represent an unlocked door) and an authorized user may receive a second set of visual indicators (e.g., the visual indicators may be red to represent a locked door).

[0026]

[0033] The visual indicators 108 and 110 can change appearance (e.g., color, size, animation, etc.) to correspond to different actions based on the authentication status. For example, the door interface component 104 can determine different configurations of the visual indicators 108 and 110 for presentation on a display device, such as a surface of the control 106, a liquid crystal display (LCD), or other display type. As shown in FIG. 1 , the control 106 includes the visual indicator 108 showing an arrow (various colors, sizes, light intensities, etc.) and a lock symbol to communicate an open, closed, or locked state of the door 122. For example, a green arrow, no lock, can indicate that an authenticated user can interact with (e.g., press) the control 106 to open the door. In some examples, a red arrow can be presented with a lock symbol to indicate that the door 122 is locked.

[0027]

[0034] In general, the door interface component 104 may determine a green lock symbol to associate with the visual indicator 108 and / or the visual indicator 110 to indicate that the user has been authenticated but is attempting to use a door that is less secure than another available door. A green arrow may indicate that the door 122 may be opened in response to the user pressing the control 106, interacting with an application, and / or being identified while approaching the door 122 (e.g., the door may open automatically without user input). A red lock and / or a red arrow may be associated with the visual indicator 108 and / or the visual indicator 110 to indicate that the door may not be open.

[0028]

[0035] In examples, if a user attempts to enter a door that is less safe to enter than another door, the visual indicator 108 may present a red arrow, a no entry symbol, "other door" or "use other door" text, etc. The door interface component 104 may determine the safest door for the occupant to enter based at least in part on sensor data associated with the vehicle 102. For example, the sensor data may indicate that an object (e.g., a parked car, a pedestrian, etc.) is blocking the door, that a malfunction of the door renders the door inoperable, and that another door is clear for entry. In some examples, the door interface component 104 may determine the safest door to enter based at least in part on receiving data from the user indicating that the user is not physically able to enter.

[0029]

[0036] In some examples, the door interface component 104 can present a visual indicator 110 having information including one or more words and / or one or more symbols (shown in FIG. 1 as “HELP” and a telephone symbol) to communicate that pressing the control 106 is the initiation of a communication. In examples where an authenticated user initiates an interaction with the vehicle 102 via the control 106 and / or via an application associated with the user's device, the control 106 can be used to receive assistance from a remote operator (e.g., the door interface component 104 can initiate a communication to a remote computing device such as a fleet service, an emergency service, etc.). In this manner, the authenticated user can receive emergency assistance from an emergency service or can receive assistance with the vehicle from the fleet service (e.g., requesting cleaning, relocation, or other reasons for the vehicle).

[0030]

[0037] The door interface component 104 may additionally or alternatively present a visual indicator 110 to communicate or interact with an unauthenticated user who has pressed a control 106 or who has been detected in the vicinity of the vehicle 102 using one or more sensors. In such an example, interaction with the control 106 may include a user pressing the control 106 or a sensor detecting a movement from the unauthenticated user indicating a selection of the control 106 (e.g., eye tracking, hand movements, verbal commands, etc. may be used to interact with the control 106). For example, an unauthenticated user may be detected in the vicinity of the vehicle 102 and a visual indicator interacted with by the unauthenticated user may be output for display, may cause the vehicle 102 to determine a trajectory to navigate to a new location, and / or may cause a vehicle computing device to request assistance with the vehicle 102.

[0031]

[0038] In various examples, the door interface component 104 may present the visual indicator 108 at a different time (e.g., before or after) than the presentation of the visual indicator 110, and in other examples, may present both the visual indicator 108 and the visual indicator 110 simultaneously. In some examples, the visual indicators 108 and 110 may comprise a single visual indicator that changes appearance based on a user's authentication state associated with the number of times the inputs and controls 106 are pressed or otherwise interacted with by the user. Additional examples for presenting the visual indicators 108 and 110 are described throughout this disclosure in connection with FIGS. 2-7.

[0032]

[0039] In some examples, the door interface component 104 can present “help” text and / or a phone symbol associated with the visual indicator 108 or the visual indicator 110. For example, a phone / help indication can be presented during entry and exit by an occupant while the vehicle 102 (with or without an occupant) navigates within the environment 100. In this manner, the door interface component 104 can receive requests from authenticated and / or unauthenticated users to provide assistance for themselves or for another person inside or outside the vehicle 102. In one non-limiting example, an unauthenticated user outside the vehicle 102 can request help for an occupant in the vehicle by interacting with the control 106 while the control 106 presents a phone / help indication. In such an example, the door interface component 104 can send a communication for emergency assistance directly from the vehicle 102 to an emergency services provider and / or send a communication to a remote operator to remotely operate the door 122 (e.g., open the door so the occupant can receive help).

[0033]

[0040] In some examples, an authenticated user can press control 106 while visual indicator 108 and / or visual indicator 110 present “help” text and / or a telephone symbol to begin the authentication process to enter vehicle 102 (e.g., to retrieve any belongings left behind, such as a mobile device that can be used to use an application to operate the door for re-entry).

[0034]

[0041] As shown in FIG. 1, the door interface component 104 can detect the presence of a user proximate to the vehicle using one or more sensors, including the camera 112, the audio interface 114, the infrared sensors 116(1) and 116(2), another sensor on the vehicle 102, or a sensor off the vehicle 102. For example, the camera 112 can capture an image of the user and determine an authentication status based on the image (e.g., comparing the image to images associated with authorized users). In some examples, a user who completes an authentication process (e.g., identity verification, payment verification, etc.) can be recognized by the vehicle 102 using the camera 112, and the visual indicator 108 can include a green arrow indicating that the door 122 is unlocked. In various examples, the camera 112 can capture images using one or more infrared sensors 116(1) and 116(2).

[0035]

[0042] In various examples, the LED 118 can change animation and color to communicate to the user the operation of the camera 112. For example, the LED 118 can output a blinking green light to indicate that an image is about to be captured and can output a continuous green light to indicate that the camera 112 is capturing an image. In some examples, the functionality associated with the LED 118 can be included in the display of the control 106 or other area of ​​the door interface component 104.

[0036]

[0043] In general, the audio interface 114 is configured to output and / or receive audio signals to interact with the user (e.g., using a speaker to present information regarding the functionality of the door interface component 104, receiving audio via a microphone, etc.). The audio interface 114 can, for example, indicate a period for receiving additional input and can associate the additional input with an action based on the time period and the authentication status of the user. For example, the audio interface 114 can provide audio to communicate when to press the control 106 to initiate an action to open the door, to rent the vehicle 102, to request emergency assistance, or to communicate with an occupant within the vehicle 102, although other actions are contemplated. In some examples, the audio interface 114 can include an ultrasonic sensor to generate a signal usable to enable the user to access the vehicle 102 (e.g., an ultrasonic signal to determine the distance to the user to enable the door to automatically operate based on the distance between the user and the vehicle).

[0037]

[0044] In various examples, the audio from audio interface 114 can be accompanied by various visual indicators to communicate with the user. For example, audio interface 114 can indicate which door to use, when to press control 106 to activate a particular function, how to pay to rent a vehicle, how to initiate communication (including, for example, outputting a ringing tone), warning signals if a door is operating, etc.

[0038]

[0045] As described above, the authentication component can determine whether a user in proximity to the door interface component 104 is associated with an authenticated or unauthenticated state. In some examples, the authentication component can determine an authentication state of a user interacting with the door interface component 104 via the camera 112, another sensor, the control 106, and / or the communication component 120 (e.g., an antenna associated with a proximity technology such as near field communication (NFC)). For example, determining the authentication state can be based on the vehicle 102 detecting the proximity of the user's mobile device (e.g., Bluetooth, ultra-wide band, NFC, WiFi, ultrasonic, etc.). In various examples, the authentication component can receive and / or access credentials from another entity, such as an application used by the user to rent the vehicle 102, and verify the authentication state based at least in part on the credentials.

[0039]

[0046] In some examples, authentication of a user by the door interface component 104 can be based at least in part on a machine-readable code (e.g., a quick response (QR) code, a bar code, or a radio frequency identification (RFID) tag). For example, authentication can be initiated by the user accessing a QR code associated with the vehicle (e.g., displayed on the exterior of the vehicle 102, in the control 106, in an application, etc.). The door interface component 104 can additionally or alternatively initiate authentication based on scanning a bar code or detecting an RFID tag (e.g., which may be presented by the user's device). The machine-readable code can also be scanned by the user to initiate renting of the vehicle 102 and request that the vehicle be moved, among other potential actions.

[0040]

[0047] In various examples, the authentication state (e.g., an association between a user and the vehicle 102) may be based at least in part on a biometric associated with the user. For example, the authentication component may identify a user based on facial recognition, iris recognition, fingerprint recognition, etc., and determine whether the identified user is authorized to enter the vehicle (e.g., authenticated while renting the vehicle 102).

[0041]

[0048] In various examples, the door interface component 104 can output instructions to initiate, generate, or otherwise determine a communication for transmission over a network to a device associated with an emergency services provider (e.g., police, fire, ambulance, etc.). For example, if an unauthenticated user interacts with the control 106 and the visual indicator 110 presents an image representing a request for help (e.g., help text, a phone call, a transmission wave, etc.), the door interface component 104 can send a communication indicating a request for help to an emergency services provider.

[0042]

[0049] In various examples, a vehicle computing device associated with the door interface component 104 may be configured to receive sensor data representative of objects of the environment 100, such as via a perception component (e.g., perception component 622). In some examples, the sensors may include sensors onboard the vehicle 102 and may include, but are not limited to, ultrasonic sensors, radar sensors, light detection and ranging (lidar) sensors, cameras, microphones, inertial sensors (e.g., inertial measurement units, accelerometers, gyros, etc.), global positioning system (GPS) sensors, etc. In some examples, the sensors may include one or more remote sensors, such as sensors onboard another autonomous vehicle and / or sensors attached to the environment 100. In various examples, the vehicle 102 may be configured to transmit and / or receive data from other autonomous vehicles. The data may include sensor data and / or state data, such as sensor data associated with the environment 100.

[0043]

[0050] In various examples, the door interface component 104 can determine a function to associate with the control 106 based at least in part on the sensor data. For example, the sensor data can be used to detect the presence, distance, and / or identity of one or more users around the vehicle 102. The door interface component 104 can output instructions to open the door, enable renting of the vehicle, or communicate with another computing device associated with the user and / or a remote operator. In one example, the sensor data can be used to enable a user to enter the vehicle 102 in cases where the user does not have a mobile device (or has left the mobile device in the vehicle). As described herein, the mobile device can include a phone, a wearable device (watch, glasses, etc.), a key, a card having a machine-readable code, etc.

[0044]

[0051] In general, the door interface component 104 can enable users inside or outside the vehicle to communicate with the vehicle 102 for a variety of reasons. For example, if a mobile device is not available or an application is not accessible, the door interface component 104 can enable a user to enter or exit the vehicle, book a ride (e.g., rent a vehicle), and / or have the vehicle change location within the environment. The door interface component 104 can additionally or alternatively enable communication between a user within the vehicle and a user outside the vehicle (e.g., an inside occupant needs help and a non-occupant outside the vehicle can call for help, an inside occupant can authorize another user to enter the vehicle, etc.).

[0045]

[0052] A user at an airport, hotel, or other location may, for example, rent a vehicle 102 using the door interface component 104 without requiring the user to register through an application. In one particular example, the user may receive a machine-readable code (e.g., a QR code, a barcode, etc.) from a device associated with the airport, hotel, or other entity, and the vehicle 102 may authenticate the user by capturing an image of the machine-readable code. For example, the user may obtain a travel voucher (e.g., from a hotel concierge) having a machine-readable code that includes destination and / or payment information, and present the voucher to the vehicle 102 to rent the vehicle. The vehicle 102 may scan the voucher using a camera or scanner to authenticate the user.

[0046]

[0053] In various examples, a key card, such as a hotel key card, can store credentials for accessing the vehicle 102, and a user can scan the key card at a communication component 120 (e.g., an NFC antenna) of the door interface component 104 to interact with the vehicle 102.

[0047]

[0054] In some examples, enabling a user to rent a vehicle 102 may include receiving destination and / or payment information from the user via voice commands. For example, the user may be prompted to verbally provide destination and payment information (e.g., using the audio interface 114) to rent the vehicle 102.

[0048]

[0055] 2 is a diagram illustrating an example door interface component 200 implementing example interaction techniques as described herein. For example, the door interface component 104 can configure one or more visual indicators associated with the control 106 to operate the door 122, initiate communication, etc. In general, the control 106 can represent a single button that outputs different functions based on the configuration of the visual indicator 202 (e.g., color, presentation time, brightness, animation, etc.) and the authentication state of the user 204.

[0049]

[0056] 2, the visual indicators 202 represent arrows (e.g., green arrows facing each other) to indicate to the user 204 that the door 122 can be opened in response to the door interface component 104 determining that the user 204 has touched, hovered, or otherwise interacted with the control 106. In some examples, the door interface component 104 may output the visual indicator 202 based on the user 204 having previously been authorized to interact with the door interface component 104. For example, the user 204 may interact with a sensor of the door interface component 104 (e.g., the camera 112 or other sensor), an application associated with the door interface component 104, and / or a computing device associated with the door interface component 104 to initiate authentication. The user 204 may be authenticated to operate the door 122 as an authenticated user based on the door interface component 104 verifying an authentication status of the user 204.

[0050]

[0057] The location of the visual indicator 202 may vary relative to the control 106. For example, the visual indicator 202 may appear in a different area of ​​the control 106 or in an area adjacent to the control 106. As shown in FIG. 2, the visual indicator 202 occupies a central area of ​​the control 106, but in other examples, the visual indicator 202 may appear in an upper area or a lower area of ​​the control 106. The example of FIG. 2 shows the visual indicator 202 as an arrow, but other symbols, words, and other indicia are also possible. For example, the visual indicator 202 may represent other text or visual indicators for using another door other than the door 122.

[0051]

[0058] The visual indicator 202 may include a colored arrow to communicate that the door is unlocked (e.g., green) or locked (e.g., red). The door interface component 104 may present the visual indicator 202 along with an audio indicator to communicate whether the door 122 is locked, unlocked, or functioning properly (e.g., can the opening be fully opened, etc.). In some examples, the visual indicator 202 may include a lock or other symbol instead of or in addition to an arrow. In various examples, the lock represented by the visual indicator 202 may change color (e.g., green if the door is unlocked or red if the door is locked) and / or intensity (e.g., from lighter to darker, from darker to light, etc.) for presentation to the user 204. As described herein, the visual indicator 202 may include one or more of a symbol (e.g., phone, lock, etc.), text (e.g., "help"), a transmission wave, etc. to communicate a potential function to be used by interacting with the control 106.

[0052]

[0059] In some examples, visual indicator 202 may include at least the functionality described in connection with visual indicators 108 and 110 of Figure 1. In some examples, door interface component 104 may include Braille for implementing the interaction techniques described herein (e.g., Braille on control 106 may enable a user to move the vehicle, call for help, or rent a vehicle).

[0053]

[0060] 2 further illustrates a sensor 206 in proximity to the LED 118, the camera 112, and the audio interface 114. In various examples, the sensor 206 may represent an infrared sensor that may be used during image capture or detection of the user 204, and in at least some examples, to determine the location of the contact point (e.g., fingertip) of the user 204 when interacting with the control 106.

[0054]

[0061] By way of example and not limitation, a user 204 may interact with the door interface component 104 to move the vehicle 102 (e.g., navigate from a current location to another location). For example, a user 204 (either an authenticated user or an unauthenticated user) may interact with the controls 106 to cause the vehicle 102 to determine a trajectory to move the vehicle to another location within an environment. In some examples, the user 204 may request a move by initiating a communication to a remote operator based on one or more interactions with a sensor, a camera, an audio interface (audio interface 114), a communication component (e.g., communication component 120, communication connection 610), and / or a control (e.g., control 106) of the door interface component 104. The remote operator may ascertain whether the vehicle should move in response to a request from the user 204 (e.g., the remote operator may receive data from the door interface component 104 and / or sensor data from one or more sensors of the vehicle 102 to "see" the environment of the vehicle 102 via one or more user interfaces of the remote computing device). In examples where the vehicle 102 can safely move, the remote operator may send a signal to the vehicle 102 verifying the relocation request, and the vehicle 102 may determine a trajectory to move the vehicle to another location. In examples where the vehicle 102 cannot or should not safely move, the remote operator may send a signal indicating that the vehicle should remain in place.

[0055]

[0062] In some examples, an unauthenticated user (or a group of unauthenticated users) may interact with the door interface component 104, and the remote computing device may receive communications from the vehicle 102 to analyze the environment and determine if the vehicle 102 should move. For example, a machine learning algorithm (or a human operator) associated with the remote computing device may distinguish between a malicious request to move the vehicle (e.g., the vehicle is not blocking an area when an unauthenticated user interacts with the vehicle 102) and a valid request to move the vehicle (e.g., moving the vehicle 102 improves safety for the occupant and / or occupants as they are picked up, limits the possibility of potential interaction with an obstacle, etc.). In this manner, the vehicle 102 may be able to move in scenarios where the vehicle 102 is blocking a road, driveway, or person, but may also remain in place to pick up occupants if an unauthorized user has requested that the vehicle 102 be moved but the occupants are within a threshold distance to the vehicle door and / or if the occupants are within the vehicle 102 and no relocation is required.

[0056]

[0063] 3A illustrates an example visual indicator 302 of an example door interface component (door interface component 104) for implementing example interaction techniques as described herein. In some examples, the visual indicator 302 can be associated with a control 106.

[0057]

[0064] In some examples, the visual indicators 302 may include one or more of a left arrow, a right arrow, an up arrow, a down arrow, a circular or partially circular arrow, a telephone symbol, a lock symbol, a communication symbol (e.g., a waveform), a "help" text, or a "use other door" text, just to name a few. In some examples, the visual indicators 302 may include a colored arrow to communicate that the door is unlocked (e.g., green) or locked (e.g., red). The door interface component 104 may present the visual indicators 302 in various colors, sizes, shapes, and brightness over a period of time to attract the user's attention (e.g., a green arrow and a green lock may be presented if the door is unlocked and a red arrow and a red lock may be presented if the door is locked). The visual indicators 302 may include symbols, text, etc. to communicate a function associated with the door 122 or the vehicle 102.

[0058]

[0065] The door interface component 104 may output the visual indicator 302 "use other door" if, for example, the user is proximate to a door that is not safe for entry (e.g., sensor data detects an object moving near the door, the door is not operating properly (broken down or not fully open)). In this way, the door interface component 104 may present a reason for not allowing operation of the door while communicating to the user that another door is available for entry to the vehicle 102. Of course, other text or visual indicators may be used that make it clear to the user that they should access another door other than the proximate door.

[0059]

[0066] 3B, 3C, and 3D illustrate example configurations of an example door interface component 104 for implementing example interaction techniques as described herein. For example, the door interface component 104 can output text, images, video, and / or audio to interact with authenticated and unauthenticated users.

[0060]

[0067] In general, the door interface component 104 can configure the control 106 to perform different functions (e.g., present visual indicators, etc.) based on the authentication state of a user who may interact or has already interacted with the control 106. For example, the door interface component 104 can determine that a user (e.g., user 204) is adjacent (within a few meters) to the door 122 based on sensor data associated with sensors of the door interface component 104 and / or sensors associated with the vehicle 102 (e.g., sensor system 606 of FIG. 6). In some examples, the door interface component 104 can determine which of one or more visual indicators 302 to output for display on the control 106 based on whether the user is associated with an authenticated or unauthenticated state. In some examples, the visual indicator 302 can include at least the functionality described in connection with visual indicators 108 and 110 of FIG. 1 and visual indicator 202 of FIG. 2.

[0061]

[0068] As shown in Figure 3B, the door interface component 104 may output a visual indicator 302 for display within a region 304 of the control 106. As shown, the region 304 may extend to substantially each edge of the control 106, and the visual indicator 302 may be presented anywhere within the region 304. Figure 3C shows an example in which the control 106 includes a region 306 and a region 308, each of which may be associated with a visual indicator 302.

[0062]

[0069] As shown in FIG. 3D, the control 106 can be associated with a first button 310 and a second button 312 that can be used to interact with a user. For example, the first button 310 can initiate a function associated with a visual indicator (e.g., an arrow to open the door), and the second button 312 can be associated with another visual indicator to generate a call for help (e.g., to an emergency service provider or a remote operator to authorize the user). The door interface component 104 can receive an input from the first button 310 to open or close the door based on the visual indicator 302, which can be a red arrow, a green arrow, a green lock, a red lock, etc., and / or can receive another input from the second button 312 to initiate a communication (e.g., a message over a network to a device and / or application). In an example where the door interface component 104 is sending a communication to an emergency service provider or other remote operator and is not yet connected, a flashing waveform symbol can be shown along with a telephone symbol. Once the door interface component 104 receives an indication that communications are connected to the device, the waveform may remain visible for the duration of the communications session.

[0063]

[0070] In some examples, area 304, area 306, and / or area 308 can be associated with a liquid crystal display or other display technology that can be used to output visual indicator 302. Area 304, area 306, and / or area 308 can include functionality for receiving input from a user. For example, one or more touch sensors, mechanical buttons, etc. can be included in an area of ​​control 106 to enable door interface component 104 to receive one or more inputs from a user. Additionally or alternatively, a user can provide an input by contacting control 106 with sufficient force to change control 106 from a first position to a second position.

[0064]

[0071] 4 illustrates an additional example configuration 400 of an example door interface component for implementing the example interaction techniques as described herein. For example, the door interface component 402 can be configured to implement at least the functionality described in connection with the door interface component 104 of FIG.

[0065]

[0072] 4 illustrates example visual indicators, such as an arrow, a lock, and the text "Other Doors" and "Help," that may appear on the door interface component 402 based on different scenarios. In some examples, the control 404 may include one or more of the visual indicators in FIG. 4 to enable potential user interaction. The visual indicators associated with the control 404 may also include one or more of the visual indicators 302.

[0066]

[0073] Figure 4 illustrates a door interface component 402 that includes a camera 112, an LED 118, a communication component 120, a control 404, a sensor 406, and some of the visual indicators shown in Figure 4. In various examples, the control 404 may include functionality associated with the control 106 in Figure 1. The sensor 406 may represent an infrared sensor, an ultrasonic sensor, a proximity sensor, or other sensor type.

[0067]

[0074] As shown in FIG. 4, the control 404 can include a lock, a left arrow, a right arrow, and “help” text to give the user the option to operate a door associated with the door interface component 402 (e.g., door 122) or request help (e.g., emergency help or help from a remote operator who authorizes the user to interact with the door interface component 402). In some examples, the arrows and lock shown in FIG. 4 can appear green if the user is authenticated or red (or yellow) if the user is not authenticated. The control 404 can be configured, for example, as a single button having a first position and a second position. In such examples, the user can initiate a door operation or help request by touching an area of ​​the control 404 that corresponds to a respective visual indicator. For example, the control 404 can have different interactive zones to detect if the user touches an area near the lock and arrows or another area that includes the “help” text. Alternatively, the control may consist of two buttons, one for operating the door and one for initiating communication (e.g., one button has a raised edge relative to the edge of the other button.) Note that the location of the visual indicators, camera 112, LEDs 118, controls 404, sensors 406, etc. may vary in size, appearance, location, etc. while maintaining the interaction techniques described herein.

[0068]

[0075] 5 is a diagram illustrating an example configuration of an example door interface component for implementing the example interaction techniques as described herein. For example, the example door interface component may represent the door interface component 104 of FIG. 1 or the door interface component 402 of FIG. 4.

[0069]

[0076] At 502, Figure 5 illustrates visual indicators 504 and 506 in an example scenario in which an unauthorized individual approaches the door interface component 104. The door interface component 104 may present visual indicators 504 and 506 in response to the door interface component 104 determining that the user is within a threshold distance of the control 106. For example, when a user approaches a door associated with the door interface component 104, the visual indicator 504 may represent a green lock indicating that the door is locked. The door interface component 104 may additionally or instead present visual indicator 506 representing a telephone symbol and "help" text telling the user that the control may be used to initiate communication.

[0070]

[0077] 5 shows an example of causing the door interface component 104 to present the visual indicator 504 as a red lock (shown shaded) and / or the visual indicator 506 as a telephone symbol with "help" text that indicates to the user that access to the door is denied when the user presses the control 106 at 508. In some examples, the visual indicator 504 and / or the visual indicator 506 may then be displayed on the display device of the control 106 for a period of time (e.g., 5 seconds) during which the visual indicator 504 may revert to showing a green lock.

[0071]

[0078] At 510, FIG. 5 depicts a configuration of the door interface component 104 in response to a user pressing the control 106 during presentation of the red lock and / or phone / help indicators at 508. In FIG. 5C, the visual indicator 504 (e.g., red lock) is removed and the visual indicator 506 (phone and help text) changes from a first color (e.g., white) to a second color (e.g., blue), optionally flashing, to indicate that the door interface component 104 has generated a communication to authenticate the user. In some examples, the LED 118 may flash to coincide with the flashing of the visual indicator 506. In such examples, a pre-recorded message may instruct the user to re-press the control 106 to initiate a call for customer service. In examples where the user does not press the control 106 for a predetermined time, the visual indicators 504 and 506 may revert to the state described in connection with 502.

[0072]

[0079] At 512, FIG. 5 illustrates an example of having the door interface component 104 present the visual indicator 506 as a telephone symbol with “help” text having a higher intensity (e.g., greater brightness) and / or lack of blinking for 510 when the user presses the control 106. In this manner, the visual indicator 506 at 512 can communicate to the user that an active call has been established (e.g., the door interface component 104 has established a communication session in response to sending a communication to the computing device). In some examples, the LED 118 can be illuminated while the communication session is active. The audio interface 114 can exchange audio between the user and an operator of the computing device as part of the communication session. Additionally, the camera 112 can capture data representing images and / or video for transmission to the computing device as part of the communication session. The data can be used by the computing device to verify the identity of the user. In some examples, data from the camera 112 can be used by the vehicle computing device to verify the identity of the user, independent of transmitting data to a remote operator.

[0073]

[0080] The computing device operator may represent a human in a call center supporting a fleet of vehicles for hire. The operator may instruct the user to access an application, website, QR code, etc. for authentication and / or may authenticate the user (e.g., when the mobile device is not available to the user) and, optionally, communicate with the user's mobile device. The operator's computing device may implement a variety of authentication techniques including verifying payment, verifying user identity based on an image from the camera 112, or verifying user identity based on information provided by the user prior to and / or during the communication session.

[0074]

[0081] At 514, Figure 5 depicts a configuration of the door interface component 104 to respond to the authenticated user during the communication at 512. For example, the door interface component 104 may present the visual indicator 504 as a green arrow that, if selected by the user, causes a door associated with the door interface component 104 to open. After the communication session ends, the door interface component 104 may change the visual indicator 506 to reduce brightness and / or no longer animate the telephone symbol or "help" text.

[0075]

[0082] In various examples, audio may also be output by the door interface component 104 associated with any of the examples described in connection with 502, 508, 510, 512, and / or 514.

[0076]

[0083] 6 is a block diagram of an example system 600 for implementing the techniques described herein. In at least one example, the system 600 may include a vehicle, such as a vehicle 602.

[0077]

[0084] The vehicle 602 may have a vehicle computing device 604, one or more sensor systems 606, one or more emitters 608, one or more communication connections 610, at least one direct connection 612, and one or more drive systems 614.

[0078]

[0085] The vehicle computing device 604 may have one or more processors 616 and memory 618 communicatively coupled to the one or more processors 616. In the illustrated example, the vehicle 602 is an autonomous vehicle, however, the vehicle 602 may be any other type of vehicle, such as a semi-autonomous vehicle, or any other system having at least an image capture device (e.g., a camera-enabled smartphone). In some examples, the autonomous vehicle 602 is configured to operate in accordance with a Level 5 classification issued by the National Highway Traffic Safety Administration. It may be an autonomous vehicle, which describes a vehicle that can perform all safety-critical functions throughout the journey without a driver (or passenger) being expected to control the vehicle at any time, although in other examples, the autonomous vehicle 602 may be a fully or partially autonomous vehicle having any other level or classification.

[0079]

[0086] In various examples, the vehicle computing device 604 may store sensor data associated with the actual position of the object at the end of the set of estimated states (e.g., the end of a time period) and may use this data as training data to train one or more models. In some examples, the vehicle computing device 604 may provide the data to a remote computing device (i.e., a computing device separate from the vehicle computing device, such as computing device 636) for data analysis. In such examples, the remote computing device may analyze the sensor data to determine the actual position, speed, heading, etc. of the object at the end of the set of estimated states. Additional details of training machine learning models based on stored sensor data by minimizing the difference between the actual position and the predicted position and / or predicted trajectory are described in U.S. Patent Application No. 16 / 282,201, filed March 12, 2019, entitled "Motion Prediction Based on Appearance," which is incorporated herein by reference.

[0080]

[0087] In the depicted example, memory 618 of vehicle computing device 604 stores a localization component 620, a perception component 622, a prediction component 624, a planning component 626, one or more system controllers 628, one or more maps 630, a door interface component 632, and an authentication component 634. Although shown in FIG. 6 as residing in memory 618 for illustrative purposes, the localization component 620, the perception component 622, the prediction component 624, the planning component 626, the one or more system controllers 628, the one or more maps 630, the door interface component 632, and / or the authentication component 634 may additionally or alternatively be accessible to the vehicle 602 (e.g., may be stored in or otherwise accessible in memory remote from the vehicle 602, such as memory 640 of a remote computing device 636).

[0081]

[0088] In at least one example, the localization component 620 may include functionality for receiving data from the sensor system 606 to determine a position and / or orientation (e.g., one or more of an x ​​position, a y position, a z position, a roll, a pitch, or a yaw) of the vehicle 602. For example, the localization component 620 may include and / or request / receive a map of the environment, such as from the map 630 and / or the map component 646, and may continually determine the position and / or orientation of the autonomous vehicle within the map. In some examples, the localization component 620 may receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc., and accurately determine the position of the autonomous vehicle using simultaneous localization and mapping (SLAM), calibration, localization and mapping, simultaneously (CLAMS), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc. In some examples, the localization component 620 may determine an initial position of the autonomous vehicle to provide data to various components of the vehicle 602 to determine the relevance of objects to the vehicle 602, as described herein.

[0082]

[0089] In some examples, the perception component 622 may include functionality to perform object detection, segmentation, and / or classification. In some examples, the perception component 622 may provide processed sensor data indicating the presence of objects (e.g., entities) proximate to the vehicle 602 and / or the classification of the objects as object types (e.g., cars, pedestrians, cyclists, animals, buildings, trees, road surfaces, curbs, sidewalks, unknowns, etc.). In some examples, the perception component 622 may provide processed sensor data indicating the presence of stationary entities proximate to the vehicle 602 and / or the classification of the stationary entities as types (e.g., buildings, trees, road surfaces, curbs, sidewalks, unknowns, etc.). In additional or alternative examples, the perception component 622 may provide processed sensor data indicating one or more characteristics associated with a detected object (e.g., a tracked object) and / or an environment in which the object is located. In some examples, characteristics associated with an object may include, but are not limited to, x position (global and / or local position), y position (global and / or local position), z position (global and / or local position), orientation (e.g., roll, pitch, yaw), object type (e.g., classification), object velocity, object acceleration, object range (size), etc. Characteristics associated with an environment may include, but are not limited to, the presence of other objects in the environment, the state of other objects in the environment, time of day, day of the week, season, weather conditions, darkness / lightness indicators, etc.

[0083]

[0090] The prediction component 624 can generate one or more probability maps that represent predicted probabilities of likely locations of one or more objects in the environment. For example, the prediction component 624 can generate one or more probability maps for vehicles, pedestrians, animals, etc. within a threshold distance from the vehicle 602. In some examples, the prediction component 624 can measure the trajectories of the objects and generate discretized predicted probability maps, heat maps, probability distributions, discretized probability distributions, and / or trajectories for the objects based on the observed and predicted behavior. In some examples, the one or more probability maps can represent the intent of one or more objects in the environment.

[0084]

[0091] In some examples, the prediction component 624 may generate predicted trajectories for objects (e.g., objects) in the environment and / or generate predicted candidate trajectories for the vehicle 602. For example, the prediction component 624 may generate one or more predicted trajectories for objects within a threshold distance from the vehicle 602. In some examples, the prediction component 624 may measure a trace of the object and generate a trajectory for the object based on the observed and predicted behavior.

[0085]

[0092] In general, the planning component 626 may determine a path that the vehicle 602 will follow to travel through the environment. For example, the planning component 626 may determine various paths and trajectories as well as various levels of detail. For example, the planning component 626 may determine a path to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the path may include a sequence of waypoints for traveling between the two locations. As non-limiting examples, the waypoints include roads, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 626 may generate instructions for guiding the autonomous vehicle along at least a portion of the path from the first location to the second location. In at least one example, the planning component 626 may determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instructions may be a candidate trajectory or a portion of a trajectory. In some examples, the multiple trajectories may be generated substantially simultaneously (e.g., within technical tolerances) according to receding horizon techniques. A single path of the receding horizon data having the highest confidence level of the multiple paths may be selected to maneuver the vehicle. In various examples, the planning component 626 may select a trajectory for the vehicle 602 based at least in part on receiving data representative of an output of the door interaction component 632 (e.g., data indicating that the vehicle 602 should be moved).

[0086]

[0093] In other examples, the planning component 626 can alternatively or additionally use data from the localization component 620, the perception component 622, and / or the prediction component 624 to determine a path for the vehicle 602 to follow as it travels through the environment. For example, the planning component 626 can receive data from the localization component 620, the perception component 622, and / or the prediction component 624 regarding objects associated with the environment. Using this data, the planning component 626 can determine a path to travel from a first location (e.g., a current location) to a second location (e.g., a target location) to avoid objects in the environment. In at least some examples, such a planning component 626 can determine that such a collision-free path does not exist and then provide a path that leads the vehicle 602 to a safe stop that avoids all collisions and / or otherwise reduces damage. Additionally or alternatively, the planning component 626 can determine a path for the vehicle 602 to follow based at least in part on data received from the door interface component 632.

[0087]

[0094] In at least one example, vehicle computing device 604 may have one or more system controllers 628, which may be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems of vehicle 602. System controller 628 may communicate with and / or control corresponding systems of drive system 614 and / or other components of vehicle 602.

[0088]

[0095] The memory 618 may further include one or more maps 630 that may be used by the vehicle 602 to navigate within the environment. For purposes of this description, a map may be any number of data structures modeled in two, three, or N dimensions that may provide information about the environment, such as topology (such as intersections), streets, mountain ranges, roads, terrain, general environment, etc. In some examples, the map may include, but is not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., lidar information, radar information, etc.), spatial information (e.g., image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and / or intensities), reflectivity information (e.g., specularity information, retroreflectivity information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional mesh of the environment. In some examples, the vehicle 602 may be controlled based at least in part on the map 630. That is, map 630 may be used in conjunction with localization component 620, perception component 622, prediction component 624, and / or planning component 626 to determine the position of vehicle 602, detect objects within the environment, generate routes, and determine actions and / or trajectories for navigating the environment.

[0089]

[0096] In some examples, one or more maps 630 may be stored on a remote computing device (such as computing device 636) accessible via network 642. In some examples, multiple maps 630 may be stored based on, for example, characteristics (e.g., type of entity, time of day, day of the week, season of the year, etc.). Storing multiple maps 630 may have similar memory requirements but may increase the speed at which data in the maps may be accessed.

[0090]

[0097] As shown in FIG. 6 , the vehicle computing device 604 may have a door interface component 632. The model component 632 may be configured to perform the functions of the door interface component 104 and the door interface component 402, including controlling door operation, initiating communications, etc. In various examples, the door interface component 632 may receive one or more characteristics associated with a detected object from the perception component 622 and / or the sensor system 606. In some examples, the door interface component 632 may receive environmental characteristics (e.g., environmental factors, etc.) and / or weather characteristics (e.g., weather factors such as snow, rain, ice, etc.) from the perception component 622 and / or the sensor system 606. Although shown separately in FIG. 6 , the door interface component 632 may be part of the prediction component 624, the planning component 626, or other components of the vehicle 602.

[0091]

[0098] In various examples, the model component 632 may send the determinations from the prediction component 624 and / or the planning component 626, which may be used by the prediction component 624 and / or the planning component 626 to generate one or more predicted trajectories (e.g., heading, speed, etc.) of the object and / or one or more predicted trajectories (e.g., heading, speed, etc.) of the object. In some examples, the planning component 626 may determine one or more actions (e.g., baseline actions and / or sub-actions) for the vehicle 602, such as candidate vehicle trajectories. In some examples, the model component 632 may be configured to determine a visual indicator (e.g., visual indicators 108, 110, 202, and / or 302) based at least in part on the one or more actions for the vehicle 602.

[0092]

[0099] The authentication component 634 may provide functionality to authenticate a user to enter or otherwise associate with the vehicle 602 by determining an authenticated or unauthenticated status. The authentication component 634 may determine the authentication status of a user based at least in part on receiving data from a mobile device application associated with the user, a machine-readable code (e.g., a Quick Response (QR) code, a barcode, a radio frequency identification (RFID) tag, etc.), or a biometric (e.g., facial recognition, iris recognition, fingerprint, etc.).

[0093]

[0100] The authentication component 634 may determine the authentication status based at least in part on data received from a camera (e.g., camera 112), a sensor, and / or a proximity technology that detects the proximity of the user's mobile device (e.g., using Bluetooth, ultra-wide band, NFC, WiFi, ultrasonic, etc.). In various examples, the authentication component 634 may receive and / or access credentials from another entity, such as an application used by the user to rent the vehicle 602, and verify the authentication status based at least in part on the credentials.

[0094]

[0101] As can be appreciated, the components discussed herein (e.g., the localization component 620, the perception component 622, the prediction component 624, the planning component 626, the one or more system controllers 628, the one or more maps 630, the door interface component 632, and the authentication component 634) are described separately for illustrative purposes. However, the operations performed by the various components may be combined or implemented in any other component.

[0095]

[0102] Although examples are shown in which the techniques described herein are implemented by a planning component and / or a model component of the vehicle, in some examples, some or all of the techniques described herein may be implemented by another system of the vehicle, such as a secondary safety system. In general, such an architecture may include a first computing device for controlling the vehicle 602 and a secondary safety system operating on the vehicle 602 to verify the operation of the primary system and to control the vehicle 602 to avoid a collision.

[0096]

[0103] In some examples, some or all aspects of the components discussed herein may include any models, techniques, and / or machine learning techniques. For example, in some examples, the components in memory 618 (and memory 640, described below) may be implemented as neural networks.

[0097]

[0104] As described herein, an exemplary neural network is a technology that passes input data through successively connected layers to generate an output. Each layer in a neural network may also include another neural network or may include any number of layers (convolutional or not). As may be understood in the context of the present disclosure, a neural network may utilize machine learning, which may refer to a broad class of technology in which an output is generated based on learned parameters.

[0098]

[0105] Although described in the context of neural networks, any type of machine learning may be used consistent with the present disclosure. For example, machine learning techniques may include regression techniques (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based techniques (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), Elastic Net, least-angle regression (LARS)), decision tree techniques (e.g., classification and regression tree (CART), iterative dichotomiser 3 (ID3), chi-squared automated interaction detection (CHAID), decision strains, conditional decision trees), Bayesian techniques (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, average one-dependence Bayes, etc.), and may be used in conjunction with machine learning techniques (e.g., ensemble regression, linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), and other techniques (e.g., ensemble regression, linear regression, logistic regression, stepwise regression, logistic ... estimators), Bayesian confidence networks (BNNs), Bayesian networks), clustering techniques (e.g., k-means, k-medians, expectation maximization (EM), hierarchical clustering), association rule learning techniques (e.g., perceptrons, backpropagation, Hopfield networks, RBFNs (Radial Basis FunctionNetwork), deep learning techniques (e.g., Deep Boltzmann Machines (DBM), Deep Confidence Networks (DBN), Convolutional Neural Networks (CNN), Hierarchical Autoencoders), dimensionality reduction techniques (e.g., Principal Component Analysis (PCA), Principal Component Regression (PCR), Partial Least Squares Regression (PLSR), Sammon Mapping, Multidimensional Scaling (MDS), Projection Pursuit, Linear Discriminant Analysis (LDA), Mixture Discriminant Analysis (MDA), Quadratic Discriminant Analysis (QDA), Flexible Discriminant Analysis (FDA)), ensemble techniques (e.g., Boosting, Bootstrap Aggregation (Bagging), AdaBoost, Hierarchical Generalization (Blending), Gradient Boosting Machines (GBM), Gradient Boosted Regression Trees (GBRT), Random Forests), Support Vector Machines (SVM), supervised learning, unsupervised learning, semi-supervised learning, and the like. Additional examples of architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, PointNet, and the like.

[0099]

[0106] In at least one example, the sensor system 606 may include lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), etc. The sensor system 606 may include multiple instances of each of these or other types of sensors. For example, the lidar sensors may include lidar sensors located on each of the corners, front, back, sides, and / or top of the vehicle 602. As another example, the camera sensors may include multiple cameras located at various locations about the exterior and / or interior of the vehicle 602. The sensor system 606 may provide input to the vehicle computing device 604. Additionally or alternatively, the sensor system 606 may transmit sensor data via one or more networks 642 to one or more computing devices 636 after a predetermined period of time, in quasi-real time, at a particular frequency, etc.

[0100]

[0107] The vehicle 602 may also include one or more emitters 608 that emit light and / or sound. The emitters 608 may include interior audio and visual emitters for communicating with occupants of the vehicle 602. By way of example and not limitation, the interior emitters may include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. The emitters 608 may also include exterior emitters. By way of example and not limitation, the exterior emitters may include lights for indicating driving direction or other indicators of vehicle actions (e.g., indicator lights, signs, light arrays, etc.), as well as one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for acoustically communicating with pedestrians or one or more other nearby vehicles, one or more of which may include acoustic beam steering technology.

[0101]

[0108] Vehicle 602 may also include one or more communication connections 610 that enable communication between vehicle 602 and one or more other local or remote computing devices. For example, communication connections 610 may facilitate communication with other local computing devices on vehicle 602 and / or drive system 614. Communication connections 610 may also enable the vehicle to communicate with other nearby computing devices (e.g., remote computing device 636, other nearby vehicles, etc.) and / or one or more remote sensor systems 644 to receive sensor data. Communication connections 610 also enable vehicle 602 to communicate with remote teleoperation computing devices or other remote services.

[0102]

[0109] The communication connection 610 may include physical and / or logical interfaces for connecting the vehicle computing device 604 to another computing device or network, such as the network 642. For example, the communication connection 610 may enable Wi-Fi based communications, such as via frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communications protocol that allows each computing device to interface with other computing devices.

[0103]

[0110] In at least one example, the vehicle 602 may include one or more drive systems 614. In some examples, the vehicle 602 may have a single drive system 614. In at least one example, when the vehicle 602 has multiple drive systems 614, the individual drive systems 614 may be located at opposite ends of the vehicle 602 (e.g., front and rear, etc.). In at least one example, the drive system 614 may include one or more sensor systems that detect conditions surrounding the drive system 614 and / or the vehicle 602. By way of example and not limitation, the sensor systems may include one or more wheel encoders (e.g., rotary encoders) that sense the rotation of the wheels of the drive module, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) that measure the orientation and acceleration of the drive module, cameras or other imaging sensors, ultrasonic sensors that acoustically detect objects in the vicinity of the drive module, lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be intrinsic to the drive system 614. In some cases, a sensor system on drive system 614 may overlap or complement a corresponding system on vehicle 602 (eg, sensor system 606).

[0104]

[0111] The drive system 614 may include many of the vehicle systems, including a high voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery to alternating current for use by other vehicle systems, a steering system (which may be electric) including a steering motor and a steering rack, a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights for illuminating the exterior surroundings of the vehicle), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC / DC converters, high voltage junctions, high voltage cables, charging systems, charging ports, etc.). Additionally, the drive system 614 may include a drive module controller for controlling the operation of various vehicle systems, which may receive and preprocess data from the sensor systems. In some examples, the drive module controller may include one or more processors and memory communicatively coupled to the one or more processors. The memory may store one or more modules to perform various functions of drive system 614. In addition, drive system 614 may also include one or more communication connections that enable communication by each drive system with one or more other local or remote computing devices.

[0105]

[0112] In at least one example, direct connection 612 may provide a physical interface for coupling one or more drive systems 614 with the body of vehicle 602. For example, direct connection 612 may enable the transfer of energy, fluid, air, data, etc. between drive system 614 and the vehicle. In some examples, direct connection 612 may also releasably secure drive system 614 to the body of vehicle 602.

[0106]

[0113] In at least one example, the localization component 620, the perception component 622, the prediction component 624, the planning component 626, the one or more system controllers 628, the one or more maps 630, the door interface component 632, and the authentication component 634 may process the sensor data as described above and transmit their respective outputs to the computing device 636 via one or more networks 642. In at least one example, the localization component 620, the perception component 622, the prediction component 624, the planning component 626, the one or more system controllers 628, the one or more maps 630, the door interface component 632, and the authentication component 634 may transmit their respective outputs to the remote computing device 636 at a particular frequency, after a predetermined period of time, in near real-time, etc.

[0107]

[0114] In some examples, the vehicle 602 may transmit sensor data to the computing device 636 over the network 642. In some examples, the vehicle 602 may receive sensor data from the computing device 636 and / or a remote sensor system 644 over the network 642. The sensor data may include raw sensor data and / or processed sensor data and / or representations of the sensor data. In some examples, the sensor data (raw or processed) may be transmitted and / or received as one or more log files.

[0108]

[0115] The computing device 636 may include a processor 638 and a memory 640 that stores a map component 646, a sensor data processing component 648, and a training component 650. In some examples, the map component 646 may include functionality to generate maps of various resolutions. In such examples, the map component 646 may transmit one or more maps to the vehicle computing device 604 for navigation purposes. In various examples, the sensor data processing component 648 may be configured to receive data from one or more remote sensors, such as the sensor system 606 and / or the remote sensor system 644. In some examples, the sensor data processing component 648 may be configured to process the data, such as for use by the door interface component 632 and / or the authentication component 634, and transmit the processed sensor data to the vehicle computing device 604. In some examples, the sensor data processing component 648 may be configured to transmit the raw sensor data to the vehicle computing device 604.

[0109]

[0116] In some examples, the training component 650 can include functionality for training a machine learning model and output the evaluated trajectory. For example, the training component 650 can receive sensor data representing an object moving through an environment for a time period of 0.1 milliseconds, 1 second, 3 seconds, 5 seconds, 7 seconds, etc. At least a portion of the sensor data can be used as an input for training the machine learning model.

[0110]

[0117] In some examples, the training component 650 may be executed by the processor 638 to train a machine learning model based on training data. The training data may include a wide variety of data, such as sensor data, audio data, image data, map data, inertial data, vehicle state data, historical data (log data), or combinations thereof, associated with a value (e.g., a desired classification, inference, prediction, etc.). Such values ​​may be generally referred to as "ground truth." To illustrate, the training data may be used to determine a risk associated with an evaluated trajectory and thus may include data representing an environment captured by an autonomous vehicle and associated with one or more classifications or decisions. In some examples, such classifications may be based on user input (e.g., a user input indicating that the data represents a particular risk) or may be based on the output of another machine learning model. In some examples, such labeled classifications (or, more generally, labeled outputs associated with the training data) may be referred to as ground truth.

[0111]

[0118] In some examples, the training component 650 can include functionality to train a machine learning model to output a classification value. For example, the training component 650 can receive data representing labeled crash data (e.g., publicly available data, sensor data, and / or a combination thereof). At least a portion of the data can be used as input to train the machine learning model. Thus, by providing data of a vehicle traveling through an environment as described herein, the training component 650 can be trained to output potential intersection points associated with an object.

[0112]

[0119] In some examples, the training component 650 can include training data generated by a simulator. For example, the simulated training data can represent instances in which a vehicle crashes into or near-crashes into objects in the environment to provide additional training examples.

[0113]

[0120] The processor 616 of the vehicle 602 and the processor 638 of the computing device 636 may be any suitable processor capable of executing instructions and processing data to perform operations as described herein. By way of example and not limitation, the processors 616 and 638 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data and converts the electronic data into other electronic data that may be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered to be processors so long as they are configured to execute encoded instructions.

[0114]

[0121] Memory 618 and memory 640 are examples of non-transitory computer-readable media. Memory 618 and 640 may store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods and functions attributed to the various systems described herein. In various implementations, memory may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmatic, and physical components, of which those shown in the accompanying drawings are merely examples in connection with the description herein.

[0115]

[0122] 6 is illustrated as a distributed system, in alternative examples, components of the vehicle 602 may be associated with the computing device 636 and / or components of the computing device 636 may be associated with the vehicle 602. That is, the vehicle 602 may perform one or more of the functions associated with the computing device 636, and vice versa.

[0116]

[0123] 7 is a flow chart illustrating an example process 700 for determining an interaction using an example door interface component. For example, some or all of the process 700 can be performed by the vehicle computing device 604 or the computing device 636, as described herein.

[0117]

[0124] At operation 702, the process may include receiving an input associated with the door interface component. In some examples, operation 702 may include a vehicle computing device implementing the door interface component 104 to receive an input from a control 106 (e.g., a button on the control 106). In various examples, the user may include an occupant who has rented the vehicle 602 for a fee, a potential occupant who wants to rent the vehicle 602, a pedestrian, a person who wants to communicate with the vehicle 602, or a person who wants to communicate with an occupant in the vehicle 602, to name a few. In various examples, the user may be approved to enter the vehicle or may not be approved to enter the vehicle (based on being unauthenticated). Operation 702, in some examples, may include the door interface component 104 determining that a button has changed from a first position to a second position (e.g., the button has been pressed by the user).

[0118]

[0125] Operation 704 may include determining whether the user has been authenticated. In some examples, operation 704 may include determining an authentication state associated with the user. For example, the vehicle computing device may implement an authentication component configured to access data indicative of an authentication state of the user and / or the vehicle (e.g., credentials associated with a login, a rented status from an application, etc.). In various examples, determining the authentication state may be based at least in part on the proximity of the user's mobile device to the vehicle, an application on the user's mobile device to the vehicle, a machine-readable code, and / or a biometric associated with the user.

[0119]

[0126] In some examples, the authentication state may include an authenticated state where the user is associated with a vehicle (e.g., authorized to ride in the vehicle) or an unauthenticated state where the user is not associated with the vehicle (e.g., the user has not performed an operation to rent a vehicle). The authentication component may determine the authentication state based on accessing data from a memory, database, or other storage device that stores data associated with a ride service associated with a collection of autonomous vehicles, such as vehicles 602.

[0120]

[0127] Operation 704 may be followed by operation 706 if the user is authenticated (e.g., "yes" at operation 704). Operation 704 may proceed to operation 708 if the user is not authenticated (e.g., "no" at operation 704).

[0121]

[0128] At operation 706, the process may include outputting instructions to operate a door of the autonomous vehicle between the first location and the second location based at least in part on the input and the authentication state being in an authenticated state. In some examples, operation 706 may include the door interface component 104 changing the door 122 of the vehicle 602 between a closed state associated with the first location and an open state associated with the second location. Operation 706 may also include outputting a first visual indicator based at least in part on the authentication state being in an authenticated state. In such examples, the first visual indicator may include instructions to operate a button to open a door (e.g., the door 122) or instructions to use a particular door of the vehicle that is safest to open.

[0122]

[0129] At operation 708, the process may include generating a signal for output by the door interface component or for transmission to a remote operation system based at least in part on the input and the authentication status being in an unauthenticated state. In some examples, operation 708 may include transmitting a communication over a network to have the user authenticated by a remote operator. In some examples, the remote operation system may be associated with an emergency services provider, and the communication may request assistance with an emergency scenario near the vehicle. Operation 708 may also include outputting a second visual indicator (and / or audio) based at least in part on the authentication status being in an unauthenticated state. In such examples, the second visual indicator may include an instruction to initiate renting of the vehicle (e.g., at the vehicle or from a remote operator associated with the ride service) or an instruction to initiate generation of a communication for emergency assistance to an emergency services provider (e.g., the control 106 may include a visual indicator 110, which may be a telephone symbol and a transmission wave symbol).

[0123]

[0130] FIG. 7 illustrates exemplary processes according to examples of the present disclosure. These processes are illustrated as logical flow graphs, with each operation representing a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement specific abstract data types. The order in which the operations are described is not intended to be limiting, and any number of the described operations can be omitted and / or combined in any order and / or in parallel to implement the process. For example, the process can include performing either operation 706 or operation 708.

[0124]

[0131] The methods described herein represent sequences of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, perform the described operations. In general, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be limiting, and any number of the described operations can be combined in any order and / or in parallel to implement a process. In some examples, one or more operations of a method can be omitted entirely. For example, an operation can include determining a first action and a second action by a vehicle on a selected trajectory without determining a cost of each of the one or more actions by the vehicle. Additionally, the methods described herein can be combined, in whole or in part, with each other or with other methods.

[0125]

[0132] Various techniques described herein may be implemented in the context of computer-executable instructions or software, such as program modules, stored in computer-readable storage and executed by processors of one or more computing devices such as those illustrated. Generally, program modules include routines, programs, objects, components, data structures, etc., and define operational logic for performing particular tasks or implement particular abstract data types.

[0126]

[0133] Other architectures may be used to implement the described functionality and are intended to be within the scope of the present disclosure. Additionally, although specific distributions of responsibilities have been defined above for purposes of illustration, the various functions and responsibilities may be distributed and divided in different manners, depending on the circumstances.

[0127]

[0134] Similarly, software may be stored and distributed in a variety of ways and using different means, and the particular software storage and execution configurations described above may be varied in many different ways. Thus, software implementing the techniques described above is not limited to the specifically described forms of memory, but may be distributed on various types of computer-readable media.

[0128] Illustrative clauses

[0135] Any of the example clauses in this section may be used with any other example clause and / or any other example or embodiment described herein.

[0129]

[0136] A: A system comprising a door interface component coupled to a door of an autonomous vehicle, one or more processors, and one or more non-transitory computer-readable media storing instructions executable by the one or more processors, which instructions, when executed, cause the system to perform operations including receiving an input associated with the door interface component, determining an authentication state associated with a user, and based at least in part on the input and the authentication state, at least one of: outputting instructions to operate the door of the autonomous vehicle between a first position and a second position based at least in part on the authentication state being an authenticated state, or transmitting a signal for output by the door interface component or a signal for transmission to a remote operation system based at least in part on the authentication state being an unauthenticated state.

[0130]

[0137] B: The system of paragraph A, wherein the operations further include outputting at least one of a first visual indicator based at least in part on the authentication status being an authenticated state, or a second visual indicator based at least in part on the authentication status being an unauthenticated state, based at least in part on the input.

[0131]

[0138] C: The system of either paragraph A or B, wherein the authentication state is an unauthenticated state, the input is a first input, and the operation further includes receiving a second input from the remote operation system to place the authentication state associated with the user in an authenticated state.

[0132]

[0139] D: The system of any one of paragraphs AC, wherein the instructions are first instructions and the operation further includes outputting second instructions for causing the autonomous vehicle to navigate to an alternative location within an environment based at least in part on the authentication status being an unauthenticated state.

[0133]

[0140] E: The system of any one of paragraphs AD, wherein the door is a first door, and the action further includes determining that the first door is not secure for entry, and outputting a visual indicator to use a second door based at least in part on the authentication state being authenticated and the first door being unsecure for entry.

[0134]

[0141] F: A method comprising receiving an input signal from a vehicle door interface component; determining an authentication state of the vehicle, the authentication state indicating whether a user is authorized to enter the vehicle; and causing an action based at least in part on the input signal and the authentication state, the action including a first action based at least in part on the authentication state being an authenticated state or a second action based at least in part on the authentication state being an unauthenticated state, the second action including outputting one or more options for controlling the vehicle to a user or sending an additional signal to a remote computing device.

[0135]

[0142] G: The method of paragraph F, wherein the door interface component includes a button, and the first action includes opening a door of the vehicle or outputting a visual indicator indicating whether it is safe to open the door, and the second action includes outputting one or more options on a display device for presentation to the user, the options including a request for the vehicle to be moved to another location, a request to initiate communications for emergency assistance, or a request to rent the vehicle.

[0136]

[0143] H: The method of any of paragraphs F or G, further comprising outputting at least one of a first visual indicator based at least in part on the authentication status being an authenticated state, or a second visual indicator based at least in part on the authentication status being an unauthenticated state, based at least in part on the input signal.

[0137]

[0144] I: The method of any one of paragraphs FH, wherein the authentication status is an unauthenticated state and the input signal is a first signal, and further comprising receiving a second signal that places the authentication status of the vehicle in the authenticated state.

[0138]

[0145] J. The method of any one of paragraphs FI, further comprising outputting instructions to navigate the vehicle to an alternative location within an environment based at least in part on the authentication status being the unauthenticated state.

[0139]

[0146] K. The method of any one of paragraphs FJ, further comprising: determining that a first door is not secure for entry; and outputting a visual indicator to use a second door based at least in part on the authentication state being the authenticated state and the first door being not secure for entry.

[0140]

[0147] L: The method of any one of paragraphs FK, wherein the authenticated state is based at least in part on at least one of the following: proximity of the user's mobile device to the vehicle, an application on the user's mobile device to the vehicle, a machine-readable code, or biometric authentication.

[0141]

[0148] M: The method of any one of paragraphs FL, wherein determining the authentication status of the vehicle occurs prior to receiving the input signal.

[0142]

[0149] N: The method of any one of paragraphs FM, wherein the second action includes one of moving the vehicle from a first location to a second location within the environment, initiating a first communication to authenticate the user, or initiating a second communication to request emergency assistance from an emergency services provider.

[0143]

[0150] O: The method of paragraph N, wherein the second action is performed by a vehicle computing device of the vehicle.

[0144]

[0151] P: The method of any of paragraphs N or O, further comprising: outputting a set of commands to a display device of the vehicle or a mobile device of a user based at least in part on the input signal and the authentication status being the unauthenticated state; and receiving via the display device or the mobile device a selection of a command from the set of commands, wherein causing the second action is based at least in part on the command.

[0145]

[0152] Q: One or more non-transitory computer-readable media storing instructions executable by one or more processors that, when executed, cause the one or more processors to perform operations: receiving an input signal from a control of a door interface component of a vehicle; determining an authentication state of the vehicle, the authentication state indicating whether a user is authorized to enter the vehicle; and causing an action based at least in part on the input signal and the authentication state, the action including a first action based at least in part on the authentication state being an authenticated state or a second action based at least in part on the authentication state being an unauthenticated state, the second action including outputting one or more options for controlling the vehicle to the user or sending an additional signal to a remote computing device.

[0146]

[0153] R: The one or more non-transitory computer-readable media of paragraph Q, further comprising: outputting, based at least in part on the input signal, at least one of a first visual indicator based at least in part on the authentication status being the authenticated state, or a second visual indicator based at least in part on the authentication status being the unauthenticated state.

[0147]

[0154] S: One or more non-transitory computer-readable media of any of paragraphs Q or R, wherein the door interface component includes a button and the first action includes opening a door of the vehicle or outputting a visual indicator indicating whether it is safe to open the door.

[0148]

[0155] T: One or more non-transitory computer-readable media described in any one of paragraphs QS, wherein the second action includes one of moving the vehicle from a first location to a second location within the environment, initiating a first communication to authenticate a user, or initiating a second communication to request emergency assistance from an emergency services provider.

[0149]

[0156] While the above example clauses are described with respect to one particular implementation, it should be understood in the context of this specification that the contents of the example clauses can also be implemented via methods, devices, systems, computer-readable media, and / or other implementations. Additionally, any of the example clauses AT can be implemented alone or in combination with one or more other example clauses AT.

[0150] Conclusion

[0157] While one or more examples of the technology described herein have been described, various modifications, additions, permutations, and equivalents thereof are included within the scope of the technology described herein.

[0151]

[0158] In the examples, reference is made to the accompanying drawings, which form a part thereof, showing specific examples of the claimed subject matter. It should be understood that other examples can be used and modifications or substitutions, such as structural changes, can be made. Such examples, modifications or variations do not necessarily depart from the intended scope of the claimed subject matter. While steps herein may be presented in a certain order, in some cases the order can be changed such that certain inputs are provided at different times or in a different order without changing the functionality of the described systems and methods. The disclosed procedures can also be performed in different orders. Additionally, the various calculations herein need not be performed in the order disclosed, and other examples using alternative orders of calculations could be readily implemented. In addition to reordering, calculations could also be decomposed into sub-calculations with the same results.

Claims

1. 1. A system comprising: a door interface component coupled to a door of the autonomous vehicle; one or more processors; one or more non-transitory computer-readable media storing instructions executable by the one or more processors, the instructions, when executed, causing the system to perform operations, such as: receiving an input signal from the door interface component of the autonomous vehicle; determining an authorization status of the autonomous vehicle, the authorization status indicating whether a user is authorized to enter the autonomous vehicle; and and causing an action based at least in part on the input signal and the authentication status, the action comprising: a first action based at least in part on the authentication status being authenticated; or a second action based at least in part on the authentication status being an unauthenticated state, the second action including outputting one or more options to the user for controlling the autonomous vehicle or sending an additional signal to a remote computing device.

2. The action is based at least in part on the input signal: a first visual indicator based at least in part on the authentication status being the authenticated state; or The system of claim 1 , further comprising outputting at least one second visual indicator based at least in part on the authentication status being the unauthorized status.

3. The authentication state is the unauthenticated state, and the input signal is a first input signal, and the operation is The system of claim 1 or 2, further comprising receiving a second input signal from a remote operation system to place the authentication status associated with the user in the authenticated state.

4. The instruction is a first instruction, and the action is 3. The system of claim 1, further comprising: outputting a second instruction to cause the autonomous vehicle to navigate to an alternative location within an environment based at least in part on the authentication status being the unauthenticated state.

5. the door is a first door, and the operation comprises: determining that the first door is unsafe for entry; 3. The system of claim 1, further comprising: outputting a visual indicator to use a second door based at least in part on the authentication status being the authenticated state and the first door being unsecure for entry.

6. the door interface component includes a button; the first action includes opening a door of the autonomous vehicle or outputting a visual indicator indicating whether it is safe to open the door; and 3. The system of claim 1 or 2, wherein the second action includes outputting on a display device for presentation to the user one or more options including a request for the autonomous vehicle to move to another location, a request to initiate communications for emergency assistance, or a request to rent the autonomous vehicle.

7. 3. The system of claim 1 or 2, wherein the authenticated state is based at least in part on at least one of the following: proximity of a user's mobile device to the autonomous vehicle, an application on the user's mobile device to the autonomous vehicle, a machine-readable code, or biometric authentication.

8. The system of claim 1 or 2, wherein determining the authentication state of the autonomous vehicle occurs before receiving the input signal.

9. The second action is causing the autonomous vehicle to move from a first location to a second location within an environment; initiating a first communication to authenticate the user; or 3. The system of claim 1 or 2, further comprising one of initiating a second communication to request emergency assistance from an emergency services provider.

10. The system of claim 9 , wherein the second action is performed by a vehicle computing device of the autonomous vehicle.

11. outputting a set of commands to a display device of the autonomous vehicle or a mobile device of the user based at least in part on the input signal and the authentication status being the unauthenticated state; receiving a selection of a command from the set of commands via the display device or the mobile device; The system of claim 9 , wherein causing the second action is based at least in part on the command.

12. 1. A method comprising: receiving an input signal from a vehicle door interface component; determining an authorization status of the vehicle, the authorization status indicating whether a user is authorized to enter the vehicle; causing an action based at least in part on the input signal and the authentication status, the action comprising: a first action based at least in part on the authentication status being authenticated; or a second action based at least in part on the authentication status being an unauthenticated state, the second action including outputting one or more options to the user for controlling the vehicle or sending an additional signal to a remote computing device.

13. based at least in part on the input signal; a first visual indicator based at least in part on the authentication status being the authenticated state; or a second visual indicator based at least in part on the authentication status being the unauthorized state; and The method of claim 12 , further comprising outputting at least one of:

14. the authentication state is the unauthenticated state, and the input signal is a first input signal; 14. The method of claim 12 or 13, further comprising receiving a second input from a remote operation system to place the authentication status associated with the user in the authenticated state.

15. A computer program comprising coded instructions which, when executed on a computer, implements the method according to claim 12 or 13.

Images