Vehicle door interface interaction

Abstract

Techniques for interacting with authorized and unauthorized users via a door interface component are discussed herein. A vehicle computing device can implement a door interface component and / or an authentication component to control the operation of a vehicle door or initiate communication for assistance. For example, the door interface component can include a button that provides different visual indicators and functionality based on whether the user is authorized to enter the autonomous vehicle (e.g., whether to open the door or provide a visual indicator for the user to select to open the door) or whether to request to move the vehicle, initiate a vehicle rental, or call for help if the user is not authorized to enter the autonomous vehicle.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to vehicle door interface interaction.

BACKGROUND ART

[0002] Related Applications

[0001] This application claims priority to U.S. Patent Application No. 17 / 507,720, filed on October 21, 2021, entitled "Vehicle Door Interface Interaction", the entire contents of which are incorporated herein by reference.

[0003]

[0002] When using a ride sharing service such as a taxi, a driver may not only interact with a user for more complex interactions, but also restrict or grant input / output permissions to the user. In situations where a human driver is not present, for example, in the case of an autonomous vehicle, a pedestrian or potential occupant may otherwise be unable to interact with the driver for such more complex mutual interactions.

BRIEF DESCRIPTION OF THE DRAWINGS

[0004]

[0003] The detailed description is described with reference to the accompanying drawings. In the drawings, the leftmost digit of a reference number identifies the drawing in which that reference number first appears. The use of identical reference numbers in different drawings indicates similar or identical items or features. [Figure 1]

[0004] Figure 1 is a diagram illustrating an example environment in which an exemplary door interface component determines actions of an exemplary vehicle. [Figure 2]

[0005] Figure 2 is a diagram illustrating an example of a door interface component that implements the exemplary interaction techniques as described herein. [Figure 3A]

[0006] Figure 3A shows an example of a visual indicator for an exemplary door interface component for implementing exemplary interaction technology as described herein. [Figure 3B]

[0007] Figure 3B shows an example configuration of an exemplary door interface component for implementing exemplary interaction technology as described herein. [Figure 3C]

[0007] Figure 3C shows an example configuration of an exemplary door interface component for implementing exemplary interaction technology as described herein. [Figure 3D]

[0007] Figure 3D shows an example configuration of an exemplary door interface component for implementing exemplary interaction technology as described herein. [Figure 4]

[0008] This figure shows an example of an additional configuration of an exemplary door interface component for implementing exemplary interaction techniques as described herein. [Figure 5]

[0009] This figure shows an example configuration of an exemplary door interface component for implementing exemplary interaction technology as described herein. [Figure 6]

[0010] Figure 6 is a block diagram showing an exemplary system for implementing the technology described herein. [Figure 7]

[0011] Figure 7 is a flowchart illustrating an exemplary process for determining interaction using exemplary door interface components. [Modes for carrying out the invention]

[0005]

[0012] As mentioned above, existing vehicles typically have handles, buttons, or other opening mechanisms to control the operation of doors for occupants or other authorized users to enter and exit, while preventing or restricting unauthorized users from boarding or accessing the autonomous vehicle. However, in some cases, it may be desirable for the vehicle to interact with the user (whether authorized or not) for more complex scenarios, such as allowing an unauthorized user to initiate a vehicle rental, communicate an emergency, and / or request that the vehicle move off the route of another vehicle.

[0006]

[0013] This application relates to a technology for interacting with authorized and unauthorized users by a vehicle door interface component. The technology may include a vehicle computing device that implements a door interface component for responding differently to a user based on whether the user is authorized to interact with an autonomous vehicle. For example, the door interface component may be coupled to the door of an autonomous vehicle and may include controls such as buttons or sensors for interacting with the user. In various examples, after receiving an input associated with the control, the door interface component may operate the door (e.g., open or close the door) based on the user being associated with the autonomous vehicle (e.g., the user is authenticated to use the autonomous vehicle, for example, based on the user being verified as an authorized user, for example, for a fee). In some examples, based on the user not being associated with the autonomous vehicle (e.g., the user is unauthenticated and / or has not initiated a request to board), the door interface component may output instructions to move the autonomous vehicle, authorize the user, and / or initiate communication with a remote operator or emergency personnel, to name just a few. Therefore, the control of the door interface components may be implemented differently depending, among other things, on whether the user has access to or is authorized to use the vehicle. By implementing the technologies described herein, it is possible to accurately and efficiently process and identify interactions with the vehicle, thereby improving the overall safety of the vehicle.

[0007]

[0014] Generally, door interface components implemented by vehicle computing devices may provide the capability to determine vehicle-to-human interaction in response to receiving input from buttons, sensors, or other controls associated with the door. In some examples, a vehicle may operate as a ride service to transport authorized users to their destinations, and the door interface component may determine a first set of actions (e.g., opening one or more doors and / or guiding the user to the safest door for boarding) based on input received from the authorized user or from a device or token associated with the authorized user (e.g., the user's mobile device). In various examples, if the door interface component determines that the user is not authorized, it may determine a second set of actions for interacting with the unauthorized user. Examples, but not limited to, the second set of actions may include moving the vehicle (e.g., avoiding blocking another vehicle, roadway, etc.), initiating communication (e.g., contacting a remote operator or an emergency service provider such as police, fire, or medical), or initiating an authorization sequence. In this way, the vehicle can be controlled in the environment at least in part based on input from buttons on the door interface components, regardless of whether the input is associated with an authorized user (e.g., an authenticated user) or an unauthorized user (e.g., an unauthenticated user approaching the vehicle).

[0008]

[0015] For example, and not an limitation, a vehicle may be parked while waiting to pick up a user (e.g., an occupant) who has rented the vehicle via an application on a mobile device (or other rental technology). The vehicle may be able to detect the presence of a user at least in part on location services (e.g., GPS-based) that track the user's phone, signals transmitted between the user's phone and the vehicle (e.g., Bluetooth®, Zigbee®, WiFi®, ultrasound, etc.), near-field communication (NFC) when the user shakes their phone near a door interface component, and / or biometric authentication (e.g., facial recognition, iris recognition, fingerprint recognition, etc.). If an authenticated occupant interacts with a button, sensor, or other control on the door interface component, the door may be opened, or if the door closest to the occupant is not safe to enter due to an obstacle (or other reason), the door interface component may output a visual indicator (e.g., image, text, video, etc.) and / or an audible prompt to guide the occupant to another door on the opposite side of the vehicle.

[0009]

[0016] In some cases, a vehicle may be parked, and a person not authorized to enter the vehicle may interact with a button. In such cases, the door interface component can provide the functionality to present options for initiating the movement of the vehicle (e.g., moving the vehicle to another location, initiating a call to a remote operator to decide whether to move the vehicle and / or where to move it), initiating an operation that allows the user to rent the vehicle (e.g., by initiating a call to a remote operator, by requesting and / or receiving destination and / or payment information), and / or sending a request for assistance (e.g., sending a contact to an emergency service provider). In this way, the door interface component enables safer operation of the vehicle by providing different outputs or interactions depending on the authentication status of the user interacting with the button.

[0010]

[0017] In another example, the occupant may be inside the vehicle, and a person outside the vehicle becomes aware that the occupant needs assistance (e.g., due to illness or other reasons). In such an example, the person can press a button on the door interface component to initiate an emergency communication to be sent to the occupant, an emergency service provider, and / or a remote computing device associated with the vehicle. Using the techniques described herein, a vehicle can provide assistance to an occupant inside the vehicle based on receiving input at a door interface component on the exterior of the vehicle.

[0011]

[0018] In some examples, the techniques described herein may include a vehicle computing device that authenticates a user interacting with the control of a door interface component. The control may be a button having a first position and a second position, and the interaction may include the user pressing the button to cause a change between the first and second positions. In some examples, the interaction with the control may include a sensor on the door interface component (e.g., a camera, ultrasonic sensor, infrared sensor, etc.) for detecting the presence of the user. When a button is pressed, or when a user is detected near the door by a sensor, the door interface component may send a request for user authentication status to the vehicle computing device. In some examples, the vehicle computing device may send a communication to the door interface component indicating that the user is associated with an authenticated state (e.g., authorized to enter the vehicle) or an unauthenticated state (e.g., not authorized to enter the vehicle). In an example where the user is authenticated, the door interface component can output instructions, visual indicators, and / or communications to operate the autonomous vehicle's doors between a first position (e.g., closed position) and a second position (e.g., open position). In an example where the user is not authenticated, the door interface component can output different commands, visual indicators, and / or communications to authenticate the user, move the vehicle from its current position, and / or call emergency assistance, to name just a few examples.

[0012]

[0019] As mentioned, a door interface component may determine a first set of visual indicators to communicate options to the user when the user is authenticated (e.g., a door open indicator, the use of other door indicators, a door lock / unlock indicator, etc.). For example, a user may be authorized to enter a vehicle, but the door associated with the input may not be safe to enter (e.g., it is stationary or moving and is within a threshold distance of another object that could come into contact with the user or the door if the user attempts to use the door). To enable the user to safely enter the vehicle, the door interface component may output an indicator for the user to enter the vehicle using another door, while also outputting an indicator that the door is locked. Examples of additional visual door indicators are described throughout this disclosure in relation to Figures 1–7.

[0013]

[0020] The door interface component may determine a second set of visual indicators (e.g., communication indicator, moving vehicle indicator, door lock / unlock indicator, etc.) to communicate options to the user when the user is unauthenticated. In such an example, one or more visual indicators may be presented in or adjacent to a button that allows the user to rent a vehicle (initiating the renting process in the vehicle and / or using a remote operator), request the vehicle's moving location (e.g., navigating to another location in the environment), and / or request that a communication be sent to a remote operator and / or emergency personnel. Based on the control and interaction with the currently presented visual indicators, the door interface component may allow the control (e.g., a single button with two positions) to initiate several different types of operations or actions.

[0014]

[0021] In some cases, user authentication may include an authentication component that verifies the current authentication state of the user interacting with the control. Functionality associated with the authentication component may be included in the vehicle computing device and / or door interface component 104. The authentication state may include an authenticated state or an unauthenticated state. The authenticated state may be based on, to name a few, the proximity of the user's mobile device to the vehicle, an application on the user's mobile device to the vehicle, machine-readable codes, or biometric authentication. That is, the authentication component of the vehicle computing device may decide whether to authorize the user to enter the vehicle as an authenticated state based on whether the distance between the user and the vehicle is within a threshold distance (e.g., a sensor that detects the user's mobile device within 3 meters of the vehicle). The authentication component may also authorize the user to use the vehicle, or to interact with machine-readable codes (e.g., Quick Response (QR) Code®, barcode, radio frequency identification (RFID) tag, etc.) or biometric authentication (e.g., facial recognition, iris recognition, fingerprint, etc.) based on the user's use of an application on a mobile device or other computing device. An example of adding an authentication component that determines the authentication status is described through this disclosure, including the following drawings.

[0015]

[0022] In some examples, a vehicle may include an autonomous or semi-autonomous vehicle having a vehicle computing device configured to receive sensor data from one or more sensors on the vehicle. The vehicle may use one or more sensors to detect objects while navigating an environment. Objects may include static objects (e.g., ground surfaces, buildings, bridges, signs, etc.) and dynamic objects such as other vehicles (e.g., cars, trucks, motorcycles, mopeds, etc.), pedestrians, and cyclists. In some examples, objects may be detected based on sensor data from the vehicle's sensors (e.g., cameras, motion detectors, LiDAR sensors, radar sensors, etc.). In yet another example, objects may be detected based on sensor data received from remote sensors, such as sensors associated with another vehicle or sensor located in an environment configured to share data with multiple vehicles. Sensor data representing detected objects may be used to determine the presence of a user adjacent to a door interface component.

[0016]

[0023] The technology described in the present specification can improve the functionality of a vehicle computing device in many ways. For example, a door interface component of a vehicle computing device can receive an input at a control, process the input, and trigger different vehicle actions based on the authentication state of an operator of the control. In some examples, the door interface component improves vehicle functionality and safety by enabling the vehicle to navigate to a safer location to pick up an authorized user, or by responding to an unauthorized user who requests the vehicle to move away from blocking a person or another vehicle. In addition, the technologies described in the present specification can improve occupant comfort and / or vehicle safety, for example, by indicating the safest door among a plurality of doors to enter based on sensor data. In some examples, a single button can implement a plurality of different functions, thereby reducing the number of required buttons or other controls. By utilizing determinations made by the door interface component, the vehicle computing device can improve the accuracy and / or reduce the latency for the vehicle to respond to a potential emergency situation in the environment.

[0017]

[0024] In various examples, the vehicle computing device can improve vehicle safety by predicting a trajectory of the vehicle based at least in part on instructions or communications output by the door interface component. For example, a planning component of the vehicle computing device can determine a trajectory for the vehicle that is most likely to avoid an object, based on the door interface component sending a communication to the planning component indicating that a door is blocked or otherwise unsafe to enter.

[0018]

[0025] The techniques described herein can be implemented in many ways. Exemplary implementations are provided below with reference to the following drawings. Although described in the context of autonomous vehicles, the methods, apparatus, and systems described herein can be applied to a variety of systems and are not limited to autonomous vehicles. In another example, the techniques can be used in the context of aviation or navigation, or in any system that uses sensor data. Additionally, the techniques described herein can be used with real data (e.g., captured using sensors), simulated data (e.g., generated by a simulator), or any combination of the two.

[0019]

[0026] Figure 1 shows an exemplary environment 100 in which an exemplary door interface component determines the action of an exemplary vehicle. For example, the vehicle 102 may include a vehicle computing device (e.g., vehicle computing device 604 in Figure 6) and / or a remote computing device (e.g., computing device 636 in Figure 6) that implements the door interface component 104. Although described as separate systems, in some examples the interaction techniques described herein may be implemented by other vehicle systems, components, and / or computing devices. For example, as will be described in more detail with respect to Figure 6, the interaction techniques described herein may be implemented in part with, or in association with, a localization component 620, a perception component 622, a prediction component 624, and / or a planning component 626.

[0020]

[0027] In some examples, the door interface component 104 can comprise a control 106 including a visual indicator 108 and a visual indicator 110, a camera 112, an audio interface 114 (e.g., a speaker and / or a microphone), infrared sensors 116(1) and 116(2), a light emitting diode (LED) 118, and a communication component 120 (e.g., an antenna, a transceiver, etc.).

[0021]

[0028] The vehicle 102 may be an unmanned vehicle such as an autonomous vehicle configured to operate in accordance with the Level 5 classification issued by the U.S. Department of Transportation National Highway Traffic Safety Administration. The Level 5 classification describes a vehicle that is capable of performing all safety-critical functions throughout the entire journey, in a state where the driver (or occupant) is not expected to control the vehicle at any time. In such an example, the vehicle 102 can be configured to control all functions from the start to the completion of the journey, including all parking functions, and thus may not include a driver and / or controls for manually driving the vehicle 102 such as a steering wheel, an accelerator pedal and / or a brake pedal. This is merely an example, and the systems and methods described herein may be incorporated into any ground, airborne, or waterborne vehicle, including the range from vehicles that need to be always manually controlled by a driver to vehicles that are partially or fully autonomously controlled.

[0022]

[0029] The exemplary vehicle 102 can be any configuration of a vehicle, such as a van, a sports utility vehicle, a crossover vehicle, a truck, a bus, an agricultural vehicle, and / or a construction vehicle. The vehicle 102 can be powered by one or more internal combustion engines, one or more electric motors, hydrogen power, any combination thereof, and / or any other suitable power source. Although the exemplary vehicle 102 has four wheels, the systems and methods described herein can be incorporated into vehicles having fewer or more wheels, tires, and / or tracks. The exemplary vehicle 102 can have four-wheel steering and can generally operate with equal performance characteristics in all directions. For example, vehicle 102 may be configured such that when traveling in a first direction, the first end of vehicle 102 is the front end and the second end on the opposite side is the rear end, and when traveling in the opposite direction, the first end becomes the rear end and the second end becomes the front end. In other words, vehicle 102 may be a bidirectional vehicle capable of traveling forward in either the opposite direction. These exemplary features may promote better maneuverability in narrow spaces or congested environments such as parking lots and / or urban areas.

[0023]

[0030] An exemplary vehicle 102 may include a door 122 (e.g., an opening for occupants to enter and exit) on at least one side of the vehicle. Each door may consist of any number of movable sections, panels, or parts to create an opening for occupants. In some examples, the vehicle 102 may be configured to have a door on each side of the vehicle. A door interface component 104 may be included in each door or in a vehicle panel adjacent to each door to provide the interaction technology described herein.

[0024]

[0031] In some examples, the door interface component 104 can receive inputs in control 106 and determine the authentication status of the user associated with the input. For example, an authentication component (not shown) associated with a vehicle computing device or included as part of the door interface component 104 can determine the authentication status (e.g., authenticated or unauthenticated) based on receiving instructions for input from the door interface component 104. In response to receiving the authentication status from the authentication component, the door interface component 104 can output different instructions to control the vehicle 102 (e.g., send instructions to another component of the vehicle computing device), communicate with the user, and / or communicate with a computing device located away from the vehicle 102 (e.g., a device of a remote operator capable of authorizing the user, or a device associated with emergency services).

[0025]

[0032] In some examples, the door interface component 104 can be coupled to the door 122 of the vehicle 102. The control 106 can represent a button, such as a mechanical button, having a first position and a second position. In an example where the control 106 has a single button with two positions, the door interface component 104 can determine different types of interaction with the user based on the user's authentication status. As discussed herein, the door interface component 104 can provide more complex functionality than a typical two-position button. In particular, the door interface component 104 can determine actions based at least partially on visual indicators 108, 110, and / or the user's authentication status. For example, an unauthorized user may receive a first set of visual indicators (e.g., the visual indicator may be green to indicate an unlocked door), and an authorized user may receive a second set of visual indicators (e.g., the visual indicator may be red to indicate a locked door).

[0026]

[0033] Visual indicators 108 and 110 can change their appearance (e.g., color, size, animation, etc.) to correspond to different actions based on the authentication status. For example, the door interface component 104 can determine different configurations of visual indicators 108 and 110 for presentation on a display device such as the surface of the control 106, a liquid crystal display (LCD), or other display types. As shown in Figure 1, the control 106 includes a visual indicator 108 showing an arrow (various colors, sizes, light intensities, etc.) and a lock symbol to indicate the open, closed, or locked state of the door 122. For example, a green arrow indicating unlocked can indicate that an authenticated user can interact with the control 106 (e.g., press) to open the door. In some examples, a red arrow may be presented along with a lock symbol to indicate that the door 122 is locked.

[0027]

[0034] Generally, the door interface component 104 can determine a green lock symbol associated with the visual indicator 108 and / or visual indicator 110 to indicate that the user is authenticated but is attempting to use a door that is less secure than another available door. A green arrow can indicate that the door 122 may open in response to the user pressing the control 106, interacting with the application, and / or being identified while approaching the door 122 (for example, the door may open automatically without user input). A red lock and / or red arrow can be associated with the visual indicator 108 and / or visual indicator 110 to indicate that the door may not be open.

[0028]

[0035] In some examples, if a user attempts to enter a door that is less safe than another door, the visual indicator 108 may display a red arrow, a "no entry" symbol, or text such as "another door" or "use another door." The door interface component 104 may determine the safest door for the occupant to enter, at least partially based on sensor data associated with the vehicle 102. For example, the sensor data may indicate that an object (e.g., a parked car, a pedestrian, etc.) is blocking the door, that the door is inoperable due to a malfunction, or that another door is available for entry. In some examples, the door interface component 104 may determine the safest door to enter, at least partially based on receiving data from the user indicating that it is not physically possible for the user to enter.

[0029]

[0036] In some examples, the door interface component 104 may present a visual indicator 110 containing information including one or more words and / or one or more symbols (shown in Figure 1 as "HELP" and a telephone symbol) to convey that pressing the control 106 is the start of communication. In an example where an authenticated user initiates interaction with the vehicle 102 via the control 106 and / or via an application associated with the user's device, the control 106 is available for assistance from a remote operator (for example, the door interface component 104 may initiate communication with a remote computing device such as fleet services or emergency services). In this way, the authenticated user may receive emergency assistance from emergency services or assistance regarding the vehicle from fleet services (for example, requesting vehicle cleaning, relocation, or other reasons).

[0030]

[0037] The door interface component 104 may additionally or alternatively present a visual indicator 110 to communicate with or interact with an unauthenticated user who has pressed control 106 or has been detected in the vicinity of vehicle 102 using one or more sensors. In such an example, interaction with control 106 may include the user pressing control 106 or sensors detecting movement from an unauthenticated user indicating a selection of control 106 (e.g., interaction with control 106 may be possible using eye tracking, hand movements, verbal commands, etc.). For example, an unauthenticated user may be detected in the vicinity of vehicle 102, and the visual indicator interacted with by the unauthenticated user may be output for display, allowing vehicle 102 to determine a trajectory to navigate to a new location, and / or allowing vehicle computing devices to request assistance in vehicle 102.

[0031]

[0038] In various examples, the door interface component 104 may present the visual indicator 108 at a different time (e.g., before or after) than the visual indicator 110, and in other examples, both the visual indicator 108 and the visual indicator 110 may be presented simultaneously. In some examples, the visual indicators 108 and 110 may comprise a single visual indicator whose appearance changes based on the user's authentication status, associated with the number of times the input and control 106 is pressed or otherwise interacted with by the user. Additional examples for presenting the visual indicators 108 and 110 are described throughout this disclosure with reference to Figures 2 to 7.

[0032]

[0039] In some examples, the door interface component 104 can display “Help” text and / or a telephone symbol associated with a visual indicator 108 or visual indicator 110. For example, the telephone / help display can be displayed while the vehicle 102 is navigating within the environment 100 (with or without an occupant) and between occupants entering and exiting. In this way, the door interface component 104 can receive requests from authenticated and / or unauthenticated users to provide assistance for themselves or for another person inside or outside the vehicle 102. In one non-limiting example, an unauthenticated user outside the vehicle 102 can request help for an occupant inside the vehicle by interacting with control 106 while control 106 is displaying the telephone / help display. In such an example, the door interface component 104 can transmit a communication for emergency assistance directly from the vehicle 102 to an emergency service provider and / or transmit a communication to a remote operator to remotely operate the door 122 (e.g., to open the door so that the occupant can receive assistance).

[0033]

[0040] In some cases, an authenticated user can initiate the authentication process to enter the vehicle 102 by pressing control 106 while visual indicator 108 and / or visual indicator 110 are displaying the "Help" text and / or telephone symbol (for example, retrieving any remaining belongings, such as a mobile device available to use an application to operate the door for re-entry).

[0034]

[0041] As shown in Figure 1, the door interface component 104 can detect the presence of a user approaching the vehicle using one or more sensors, including the camera 112, the audio interface 114, infrared sensors 116(1) and 116(2), another sensor on the vehicle 102, or a sensor located away from the vehicle 102. For example, the camera 112 can capture an image of the user and determine the authentication status based on the image (e.g., by comparing the image with an image associated with an authorized user). In some examples, a user completing an authentication process (e.g., identity verification, payment verification) can be recognized by the vehicle 102 using the camera 112, and the visual indicator 108 can include a green arrow indicating that the door 122 is unlocked. In various examples, the camera 112 can capture images using one or more infrared sensors 116(1) and 116(2).

[0035]

[0042] In various examples, the LED 118 can change animation and color to communicate the operation of the camera 112 to the user. For example, the LED 118 can output a flashing green light to indicate that an image is about to be captured, or a continuous green light to indicate that the camera 112 is capturing an image. In some examples, the functions associated with the LED 118 may be included in the display of the control 106 or in other areas of the door interface component 104.

[0036]

[0043] Generally, the audio interface 114 is configured to output and / or receive audio signals to interact with the user (e.g., using a speaker to present information about the function of the door interface component 104, or receiving audio via a microphone). The audio interface 114 can, for example, indicate periods for receiving additional inputs, and can associate these additional inputs with actions based on the time period and the user's authentication status. For example, the audio interface 114 can provide audio to communicate when to press control 106 to initiate an action such as opening a door, renting a vehicle 102, requesting emergency assistance, or communicating with occupants inside the vehicle 102, but other actions are also conceivable. In some examples, the audio interface 114 may include an ultrasonic sensor to generate signals that can be used to enable the user to access the vehicle 102 (e.g., an ultrasonic signal to determine the distance to the user so that the door can operate automatically based on the distance between the user and the vehicle).

[0037]

[0044] In various examples, the audio from the audio interface 114 can be accompanied by various visual indicators to communicate with the user. For example, the audio interface 114 can indicate which door to use, when to press the control 106 to activate a specific function, how to pay and rent the vehicle, how to initiate communication (including, for example, emitting a ringtone), and warning signals when a door is in operation.

[0038]

[0045] As described above, the authentication component can determine whether a user approaching the door interface component 104 is associated with an authenticated or unauthenticated state. In some examples, the authentication component can determine the authentication status of a user who has interacted with the door interface component 104 via a camera 112, another sensor, control 106, and / or a communication component 120 (e.g., an antenna associated with proximity technology such as near-field communication (NFC)). For example, determining the authentication status can be based on the vehicle 102 detecting the proximity of the user's mobile device (e.g., Bluetooth, ultrawideband, NFC, WiFi, ultrasonic, etc.). In various examples, the authentication component may receive and / or access credentials from another entity, such as an application used by the user to rent the vehicle 102, and verify the authentication status at least partially based on the credentials.

[0039]

[0046] In some cases, user authentication by the door interface component 104 can be at least partially based on machine-readable codes (e.g., Quick Response (QR) Code®, barcodes, or radio frequency identification (RFID) tags). For example, authentication may be initiated by the user accessing a QR Code® associated with the vehicle (e.g., displayed on the exterior of the vehicle 102, within the control 106, within an application, etc.). The door interface component 104 may additionally or alternatively initiate authentication based on scanning a barcode or detecting an RFID tag (e.g., presented by the user's device). Machine-readable codes may also be scanned by the user to initiate other potential actions, such as renting the vehicle 102 and requesting the vehicle to move.

[0040]

[0047] In various examples, the authentication status (e.g., the association between the user and vehicle 102) can be based at least partially on biometric authentication associated with the user. For example, the authentication component can identify the user based on facial recognition, iris recognition, fingerprint recognition, etc., and determine whether the identified user is authorized to board the vehicle (e.g., whether they were authenticated while renting vehicle 102).

[0041]

[0048] In various examples, the door interface component 104 can output instructions to initiate, generate, or otherwise determine communication for transmission over a network to a device associated with an emergency service provider (e.g., police, fire, ambulance, etc.). For example, if an unauthenticated user interacts with control 106 and the visual indicator 110 presents an image representing a request for help (e.g., a help text, phone call, radio wave, etc.), the door interface component 104 can transmit communication indicating a request for help to an emergency service provider.

[0042]

[0049] In various examples, a vehicle computing device associated with the door interface component 104 may be configured to receive sensor data representing objects in the environment 100, such as via a perception component (e.g., perception component 622). In some examples, sensors may include, but are not limited to, sensors mounted on the vehicle 102, including ultrasonic sensors, radar sensors, light sensing and ranging (lidar) sensors, cameras, microphones, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, etc.), and Global Positioning System (GPS) sensors. In some examples, sensors may include one or more remote sensors, such as sensors mounted on another autonomous vehicle and / or sensors attached to the environment 100. In various examples, the vehicle 102 may be configured to transmit and / or receive data from other autonomous vehicles. The data may include sensor data and / or state data, such as sensor data associated with the environment 100.

[0043]

[0050] In various examples, the door interface component 104 can determine a function to associate with control 106 based at least in part on sensor data. For example, sensor data may be used to detect the presence, distance, and / or identification of one or more users around the vehicle 102. The door interface component 104 can output instructions to open the door, enable the vehicle to be rented, or communicate with another computing device associated with the user and / or remote operator. In one example, sensor data may be used to allow a user to enter the vehicle 102 when the user does not have a mobile device with them (or has left a mobile device inside the vehicle). As described herein, the mobile device may include a telephone, a wearable device (such as a watch or glasses), a key, a card with a machine-readable code, and the like.

[0044]

[0051] In general, the door interface component 104 can enable users inside or outside the vehicle to communicate with the vehicle 102 for a variety of reasons. For example, if a mobile device is unavailable or an application is inaccessible, the door interface component 104 can enable users to enter and exit the vehicle, reserve a ride (e.g., rent the vehicle), and / or change the vehicle's location within the environment. The door interface component 104 can additionally or alternatively enable communication between users inside and outside the vehicle (e.g., an occupant inside needs help and a non-occupant outside the vehicle can call for help, or an occupant inside can authorize another user to enter the vehicle).

[0045]

[0052] Users at airports, hotels, or other locations can rent a vehicle 102 using the door interface component 104 without requiring the user to register via an application, for example. In one particular example, a user may receive a machine-readable code (e.g., a QR code®, barcode, etc.) from a device associated with the airport, hotel, or other entity, and vehicle 102 may authenticate the user by capturing an image of the machine-readable code. For example, a user may obtain a travel voucher (e.g., from a hotel concierge) with a machine-readable code containing destination and / or payment information, and may present that voucher to vehicle 102 to rent the vehicle. Vehicle 102 may use a camera or scanner to scan the voucher to authenticate the user.

[0046]

[0053] In various examples, a key card, such as a hotel key card, can store credentials for accessing the vehicle 102, and the user can interact with the vehicle 102 by scanning the key card with the communication component 120 (e.g., an NFC antenna) of the door interface component 104.

[0047]

[0054] In some cases, enabling a user to rent vehicle 102 may include receiving destination and / or payment information from the user via voice commands. For example, the user may be prompted to verbally provide destination and payment information (e.g., using the audio interface 114) in order to rent vehicle 102.

[0048]

[0055] Figure 2 shows an exemplary door interface component 200 that implements exemplary interaction techniques as described herein. For example, the door interface component 104 can configure one or more visual indicators associated with control 106 to operate the door 122 or initiate communication. Generally, control 106 can represent a single button that outputs different functions based on the configuration of the visual indicator 202 (e.g., color, presentation time, brightness, animation, etc.) and the authentication status of user 204.

[0049]

[0056] As shown in Figure 2, the visual indicator 202 represents an arrow (e.g., two green arrows facing each other) and indicates to user 204 that door 122 is openable in response to door interface component 104 determining that user 204 has touched, hovered over, or otherwise interacted with control 106. In some examples, door interface component 104 may output the visual indicator 202 based on the fact that user 204 has been previously authorized to interact with door interface component 104. For example, user 204 may initiate authentication by interacting with sensors on door interface component 104 (e.g., camera 112 or other sensors), an application associated with door interface component 104, and / or a computing device associated with door interface component 104. User 204 may be authenticated to operate door 122 as an authenticated user based on door interface component 104 verifying user 204's authentication status.

[0050]

[0057] The position of the visual indicator 202 can vary relative to the control 106. For example, the visual indicator 202 can appear in different areas of the control 106, or in areas adjacent to the control 106. As shown in Figure 2, the visual indicator 202 occupies the central area of ​​the control 106, but in other examples, the visual indicator 202 may appear in the upper or lower area of ​​the control 106. In the example in Figure 2, the visual indicator 202 is shown as an arrow, but other symbols, words, and other representations are also possible. For example, the visual indicator 202 could represent other text or visual indicators for using a different door other than door 122.

[0051]

[0058] The visual indicator 202 may include a colored arrow to indicate whether the door is unlocked (e.g., green) or locked (e.g., red). The door interface component 104 may present the visual indicator 202 together with an audio indicator to indicate whether the door 122 is locked, unlocked, or functioning properly (e.g., whether the opening can be fully opened). In some examples, the visual indicator 202 may include a lock or other symbol instead of, or in addition to, an arrow. In various examples, the lock represented by the visual indicator 202 may change color (e.g., green if the door is unlocked, or red if the door is locked) and / or intensity (e.g., from brighter to darker, or from darker to brighter) to present to the user 204. As described herein, the visual indicator 202 may include one or more symbols (e.g., telephone, lock, etc.), text (e.g., "Help"), or a wave to communicate potential functions used by interacting with the control 106.

[0052]

[0059] In some examples, the visual indicator 202 may include at least the functions described in relation to the visual indicators 108 and 110 in Figure 1. In some examples, the door interface component 104 may include braille for implementing the interaction techniques described herein (for example, braille in control 106 may enable the user to move the vehicle, ask for help, or rent the vehicle).

[0053]

[0060] Figure 2 further shows the LED 118, the camera 112, and the sensor 206 adjacent to the audio interface 114. In various examples, the sensor 206 could represent an infrared sensor usable during image capture or detection of the user 204, and in at least some examples, it could determine the position of the user 204's contact points (e.g., fingertips) when interacting with the control 106.

[0054]

[0061] For example, user 204 can interact with the door interface component 104 to move vehicle 102 (e.g., navigate from its current location to another location). For example, user 204 (authenticated or unauthenticated) can interact with control 106 to cause vehicle 102 to determine a trajectory to move the vehicle to another location in the environment. In some examples, user 204 can request movement by initiating communication with a remote operator based on one or more interactions with sensors, cameras, an audio interface (audio interface 114), communication components (e.g., communication component 120, communication connection 610), and / or the control of the door interface component 104 (e.g., control 106). A remote operator can determine whether the vehicle should move in response to a request from user 204 (for example, the remote operator can "see" the environment of vehicle 102 via one or more user interfaces of a remote computing device by receiving data from door interface component 104 and / or sensor data from one or more sensors of vehicle 102). In cases where vehicle 102 can move safely, the remote operator can send a signal to vehicle 102 verifying the relocation request, and vehicle 102 can determine a trajectory to move the vehicle to another location. In cases where vehicle 102 cannot or should not move safely, the remote operator can send a signal indicating that the vehicle should remain in its location.

[0055]

[0062] In some cases, an unauthenticated user (or group of unauthenticated users) may interact with the door interface component 104, and a remote computing device may receive communications from the vehicle 102, analyze the environment, and determine whether the vehicle 102 should move. For example, a machine learning algorithm (or human operator) associated with the remote computing device may distinguish between a malicious request to move the vehicle (e.g., the vehicle is not blocking an area when an unauthenticated user interacts with the vehicle 102) and a legitimate request to move the vehicle (e.g., moving the vehicle 102 improves the safety of the occupants and / or when the occupants are picked up and limits the possibility of potential interaction with obstacles). In this way, vehicle 102 can move in scenarios where it is blocking a road, driveway, or person, but can also remain in place to pick up the occupant if an unauthenticated user requests that vehicle 102 move but the occupant is within a threshold distance to the vehicle's door and / or the occupant is inside vehicle 102 and relocation is not required.

[0056]

[0063] Figure 3A shows an exemplary visual indicator 302 of an exemplary door interface component (door interface component 104) for implementing exemplary interaction techniques as described herein. In some examples, the visual indicator 302 can be associated with control 106.

[0057]

[0064] In some examples, the visual indicator 302 may include, to name just a few, one or more of the following: left arrow, right arrow, up arrow, down arrow, circular or partially circular arrow, telephone symbol, lock symbol, communication symbol (e.g., waveform), "Help" text, "Use another door" text. In some examples, the visual indicator 302 may include a colored arrow to indicate that the door is unlocked (e.g., green) or locked (e.g., red). The door interface component 104 may present the visual indicator 302 in various colors, sizes, shapes, and brightness over a period of time to attract the user's attention (e.g., a green arrow and a green lock may be presented when the door is unlocked, and a red arrow and a red lock may be presented when the door is locked). The visual indicator 302 may include symbols, text, etc., to convey functions associated with the door 122 or vehicle 102.

[0058]

[0065] The door interface component 104 can output a visual indicator 302 "Use another door" if, for example, the user is near a door that is not safe to enter (e.g., sensor data detects an object moving near the door, or the door is not functioning properly (either malfunctioning or not fully open)). In this way, the door interface component 104 can provide a reason for not allowing the user to operate the door while informing the user that another door is available for entering the vehicle 102. Of course, other text or visual indicators may be used to clarify that the user should access a different door other than the one in front of them.

[0059]

[0066] Figures 3B, 3C, and 3D show an exemplary configuration of an exemplary door interface component 104 for implementing exemplary interaction techniques as described herein. For example, the door interface component 104 can output text, images, video, and / or audio to interact with authenticated and unauthenticated users.

[0060]

[0067] In general, the door interface component 104 can configure control 106 to perform different functions (e.g., presenting visual indicators) based on the authentication status of a user who can interact with control 106 or has already interacted with it. For example, the door interface component 104 can determine that a user (e.g., user 204) is adjacent to door 122 (within a few meters) based on sensor data associated with sensors in the door interface component 104 and / or sensors associated with the vehicle 102 (e.g., sensor system 606 in Figure 6). In some examples, the door interface component 104 can determine which of one or more visual indicators 302 to output to display to control 106 based on whether the user is associated with an authenticated or unauthenticated state. In some examples, the visual indicator 302 may include at least the functions described in relation to visual indicators 108 and 110 in Figure 1, and visual indicator 202 in Figure 2.

[0061]

[0068] As shown in Figure 3B, the door interface component 104 is capable of outputting a visual indicator 302 for display within region 304 of the control 106. As shown, region 304 can extend substantially to each edge of the control 106, and the visual indicator 302 can be presented in any part of region 304. Figure 3C shows an example in which the control 106 includes regions 306 and 308, each of which can be associated with a visual indicator 302.

[0062]

[0069] As shown in Figure 3D, control 106 can be associated with a first button 310 and a second button 312 that are available for user interaction. For example, the first button 310 can initiate a function associated with a visual indicator (e.g., an arrow for opening a door), and the second button 312 can be associated with another visual indicator to generate a call for help (e.g., to an emergency service provider or a remote operator to authorize the user). The door interface component 104 can receive input from the first button 310 to open or close the door based on a visual indicator 302 such as a red arrow, a green arrow, a green lock, or a red lock, and / or receive another input from the second button 312 to initiate communication (e.g., a message over the network to a device and / or application). When the door interface component 104 sends a communication to an emergency service provider or other remote operator, and in an example where a connection has not yet been established, a flashing waveform symbol may be shown along with a telephone symbol. When the door interface component 104 receives an instruction that communication is connected to the device, the waveform can remain visible during the communication session.

[0063]

[0070] In some examples, regions 304, 306, and / or 308 can be associated with a liquid crystal display or other display technology available for outputting the visual indicator 302. Regions 304, 306, and / or 308 can include functions for receiving user input. For example, one or more touch sensors, mechanical buttons, etc., can be included in the regions of control 106 to enable the door interface component 104 to receive one or more inputs from the user. Additionally or alternatively, the user can provide input by touching control 106 with sufficient force to change control 106 from a first position to a second position.

[0064]

[0071] Figure 4 shows an additional exemplary configuration 400 of an exemplary door interface component for implementing exemplary interaction techniques as described herein. For example, the door interface component 402 can be configured to perform at least the functions described in relation to the door interface component 104 in Figure 1.

[0065]

[0072] Figure 4 shows exemplary visual indicators such as arrows, locks, and the texts "Other Doors" and "Help" that may appear on the door interface component 402 based on different scenarios. In some examples, the control 404 may include one or more visual indicators in Figure 4 to enable potential user interaction. The visual indicators associated with the control 404 may also include one or more visual indicators 302.

[0066]

[0073] Figure 4 shows a door interface component 402, which includes a camera 112, an LED 118, a communication component 120, a control 404, a sensor 406, and some of the visual indicators shown in Figure 4. In various examples, the control 404 may include the functions associated with the control 106 in Figure 1. The sensor 406 may represent an infrared sensor, an ultrasonic sensor, a proximity sensor, or other sensor types.

[0067]

[0074] As shown in Figure 4, control 404 may include a lock, left arrow, right arrow, and “Help” text to give the user the option to operate a door associated with door interface component 402 (e.g., door 122) or to request help (e.g., emergency help or assistance from a remote operator authorizing the user to interact with door interface component 402). In some examples, the arrows and lock shown in Figure 4 may appear green when the user is authenticated or red (or yellow) when the user is not authenticated. Control 404 may be configured, for example, as a single button having a first position and a second position. In such examples, the user can initiate door operation or a help request by touching the area of ​​control 404 corresponding to each visual indicator. For example, control 404 may have different interactive zones to detect when the user touches an area near the lock and arrow, or another area containing the “Help” text. Alternatively, the control may consist of two buttons: one for operating the door and the other for initiating communication (for example, one button having a raised edge relative to the edge of the other). The placement of the visual indicator, camera 112, LED 118, control 404, sensor 406, etc., may vary in size, appearance, position, etc., while maintaining the interaction technology described herein.

[0068]

[0075] Figure 5 shows an exemplary configuration of an exemplary door interface component for implementing exemplary interaction technology as described herein. For example, the exemplary door interface component may represent door interface component 104 in Figure 1, or door interface component 402 in Figure 4.

[0069]

[0076] In 502, Figure 5 shows visual indicators 504 and 506 in an exemplary scenario in which an unauthenticated person approaches the door interface component 104. The door interface component 104 may present visual indicators 504 and 506 in response to the door interface component 104 determining that the user is within threshold distance of the control 106. For example, if the user approaches a door associated with the door interface component 104, visual indicator 504 may represent a green lock indicating that the door is locked. The door interface component 104 may also, or in addition, present visual indicator 506 representing a telephone symbol, and “Help” text informing the user that the control can be used to initiate communication.

[0070]

[0077] In 508, Figure 5 shows an example in which, when the user presses control 106 in 508, the door interface component 104 presents the visual indicator 504 as a red lock (shown with shading) and / or presents the visual indicator 506 as a telephone symbol with the text "Help" indicating to the user that access to the door is denied. In some examples, the visual indicator 504 and / or the visual indicator 506 may be displayed on the display device of control 106 for a period of time (e.g., 5 seconds) during which the visual indicator 504 may then return to a state showing a green lock.

[0071]

[0078] In 510, Figure 5 shows the configuration of the door interface component 104 in response to the user pressing control 106 while the red lock and / or telephone / help indicator in 508 is displayed. In Figure 5C, the visual indicator 504 (e.g., red lock) is removed, and the visual indicator 506 (telephone and help text) changes from a first color (e.g., white) to a second color (e.g., blue) while flashing as optional, indicating that the door interface component 104 has generated communication to authenticate the user. In some examples, LED 118 can flash in conjunction with the flashing of the visual indicator 506. In such examples, a pre-recorded message can instruct the user to press control 106 again to initiate a call for customer service. In examples where the user does not press control 106 for a predetermined time, the visual indicators 504 and 506 can return to the states described in relation to 502.

[0072]

[0079] In 512, Figure 5 shows an example in which, when the user presses control 106 in 510, the door interface component 104 presents a visual indicator 506 as a telephone symbol with the text "Help" having a higher intensity (e.g., greater brightness) and / or lack of flashing for 510. In this way, the visual indicator 506 in 512 can inform the user that an active call has been established (e.g., the door interface component 104 has established a communication session in response to the transmission of communication to the computing device). In some examples, the LED 118 can be illuminated while the communication session is active. The audio interface 114 can exchange voice between the user and the operator of the computing device as part of the communication session. Furthermore, the camera 112 can capture data representing images and / or video for transmission to the computing device as part of the communication session. The data can be used by the computing device to verify the user's identity. In some examples, the data from the camera 112 can be used by the vehicle computing device to verify the user's identity, independently of transmitting the data to the remote operator.

[0073]

[0080] The operator of the computing device can represent a person in a call center supporting a fleet of rental vehicles. The operator can instruct the user to access an application, website, QR code (registered trademark), etc., for authentication, and / or authenticate the user (for example, if the user does not have access to a mobile device), and optionally communicate with the user's mobile device. The operator's computing device can implement various authentication techniques, including verifying payment, verifying user identity based on images from camera 112, or verifying user identity based on information provided by the user before and / or during a communication session.

[0074]

[0081] In 514, Figure 5 shows the configuration of a door interface component 104 that responds to an authenticated user during communication in 512. For example, the door interface component 104 may present a visual indicator 504 as a green arrow that opens the door associated with the door interface component 104, if selected by the user. After the communication session ends, the door interface component 104 may change the visual indicator 506 to reduce its brightness and / or no longer need to animate the telephone symbol or the "Help" text.

[0075]

[0082] In various examples, audio may also be output by a door interface component 104 associated with any of the examples described in relation to 502, 508, 510, 512, and / or 514.

[0076]

[0083] Figure 6 is a block diagram of an exemplary system 600 for implementing the technology described herein. In at least one example, the system 600 may have a vehicle such as vehicle 602.

[0077]

[0084] The vehicle 602 may have a vehicle computing device 604, one or more sensor systems 606, one or more emitters 608, one or more communication connections 610, at least one direct connection 612, and one or more drive systems 614.

[0078]

[0085] The vehicle computing device 604 may have one or more processors 616 and a memory 618 communicably coupled to one or more processors 616. In the illustrated example, the vehicle 602 is an autonomous vehicle, but the vehicle 602 may be any other type of vehicle, such as a semi-autonomous vehicle, or any other system having at least an image capture device (e.g., a smartphone with a camera). In some examples, the autonomous vehicle 602 is configured to operate according to the Level 5 classification issued by the National Highway Traffic Safety Administration of the U.S. Department of Transportation. An autonomous vehicle may be one in which a vehicle is capable of performing all safety-critical functions throughout the entire journey without expecting any driver (or occupant) to take control of the vehicle at any given time. However, in other examples, an autonomous vehicle 602 may be a fully or partially autonomous vehicle having any other level or classification.

[0079]

[0086] In various examples, the vehicle computing device 604 may store sensor data associated with the actual position of an object at the end of a set of estimated states (e.g., the end of a time period), and may use this data as training data to train one or more models. In some examples, the vehicle computing device 604 may provide the data to a remote computing device (i.e., a computing device separate from the vehicle computing device, such as computing device 636) for data analysis. In such examples, the remote computing device may analyze the sensor data to determine the actual position, velocity, direction of travel, etc., of an object at the end of a set of estimated states. Additional details of training machine learning models based on stored sensor data by minimizing the difference between the actual position and the predicted position and / or predicted trajectory are described in U.S. Patent Application No. 16 / 282,201, filed March 12, 2019, entitled "Motion Prediction Based on Appearance," which is incorporated herein by reference.

[0080]

[0087] In the illustrated example, the memory 618 of the vehicle computing device 604 stores a localization component 620, a perception component 622, a prediction component 624, a planning component 626, one or more system controllers 628, one or more maps 630, a door interface component 632, and an authentication component 634. Although shown in Figure 6 as existing in memory 618 for illustrative purposes, the localization component 620, perception component 622, prediction component 624, planning component 626, one or more system controllers 628, one or more maps 630, a door interface component 632, and / or the authentication component 634 may be additionally or alternatively accessible from the vehicle 602 (for example, stored in memory located away from the vehicle 602, such as the memory 640 of the remote computing device 636, or otherwise accessible).

[0081]

[0088] In at least one example, the localization component 620 may include the ability to receive data from the sensor system 606 to determine the position and / or orientation of the vehicle 602 (e.g., one or more of x, y, z positions, roll, pitch, or yaw). For example, the localization component 620 may include and / or be able to request / receive maps of the environment, such as from map 630 and / or map component 646, and continuously determine the position and / or orientation of the autonomous vehicle within the map. In some examples, the localization component 620 may receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc., and accurately determine the position of the autonomous vehicle, using SLAM (simultaneous localization and mapping), CLAMS (calibration, localization and mapping, simultaneously), relative SLAM, bundle adjustment, nonlinear least-squares optimization, etc. In some examples, the positioning component 620 may provide data to various components of the vehicle 602, as described herein, to determine the initial position of the autonomous vehicle in order to determine the relationship of objects to the vehicle 602.

[0082]

[0089] In some examples, the perceptual component 622 may include the ability to perform object detection, segmentation, and / or classification. In some examples, the perceptual component 622 may provide processed sensor data indicating the presence of objects (e.g., entities) in proximity to the vehicle 602, and / or the classification of the objects as object types (e.g., cars, pedestrians, cyclists, animals, buildings, trees, road surfaces, curbs, sidewalks, unknowns, etc.). In some examples, the perceptual component 622 may provide processed sensor data indicating the presence of stationary entities in proximity to the vehicle 602, and / or the classification of stationary entities as types (e.g., buildings, trees, road surfaces, curbs, sidewalks, unknowns, etc.). In additional or alternative examples, the perceptual component 622 may provide processed sensor data indicating detected objects (e.g., tracked objects), and / or one or more characteristics associated with the environment in which the objects are located. In some examples, properties associated with an object may include, but are not limited to, x-position (global and / or local position), y-position (global and / or local position), z-position (global and / or local position), orientation (e.g., roll, pitch, yaw), object type (e.g., classification), object velocity, object acceleration, and object range (size). Properties associated with an environment may include, but are not limited to, the presence of other objects in the environment, the state of other objects in the environment, time of day, day of the week, season, weather conditions, and brightness / darkness indicators.

[0083]

[0090] The prediction component 624 can generate one or more probability maps representing the predicted probabilities of the possible locations of one or more objects in the environment. For example, the prediction component 624 can generate one or more probability maps for vehicles, pedestrians, animals, etc., within a threshold distance from vehicle 602. In some examples, the prediction component 624 can measure the trajectory of an object and, based on the observed and predicted behavior, generate discretized predicted probability maps, heat maps, probability distributions, discretized probability distributions, and / or trajectories for the object. In some examples, one or more probability maps can represent the intent of one or more objects in the environment.

[0084]

[0091] In some examples, the prediction component 624 may generate predicted trajectories for objects in the environment (e.g., objects) and / or predict candidate trajectories for the vehicle 602. For example, the prediction component 624 may generate one or more predicted trajectories for objects within a threshold distance from the vehicle 602. In some examples, the prediction component 624 may measure the trace of an object and generate the object's trajectory based on the observed and predicted behavior.

[0085]

[0092] In general, the planning component 626 can determine the path that the vehicle 602 will follow as it travels through the environment. For example, the planning component 626 can determine various paths and trajectories, as well as various levels of detail. For example, the planning component 626 can determine a path from a first location (e.g., current location) to a second location (e.g., target location). For the purposes of this explanation, the path may include a sequence of waypoints for traveling between the two locations. In non-limiting examples, waypoints may include roads, intersections, and Global Positioning System (GPS) coordinates. Furthermore, the planning component 626 can generate instructions for guiding the autonomous vehicle along at least part of the path from the first location to the second location. In at least one example, the planning component 626 can determine how to guide the autonomous vehicle from a first waypoint in a sequence of waypoints to a second waypoint in a sequence of waypoints. In some examples, the instructions may be a candidate trajectory or part of a trajectory. In some examples, multiple tracks may be generated substantially simultaneously (e.g., within technical tolerances) according to the reverse horizon technique. Of the multiple tracks, a single track of the reverse horizon data with the highest level of confidence may be selected for operating the vehicle. In various examples, the planning component 626 is a door interface It is possible to select the trajectory of vehicle 602 at least in part on receiving data representing the output of component 632 (for example, data indicating that vehicle 602 should be moved).

[0086]

[0093] In other examples, the planning component 626 may, alternatively or additionally, use data from the positioning component 620, the perception component 622, and / or the prediction component 624 to determine the path that the vehicle 602 should follow as it travels through the environment. For example, the planning component 626 may receive data from the positioning component 620, the perception component 622, and / or the prediction component 624 regarding objects associated with the environment. Using this data, the planning component 626 may determine a path to move from a first position (e.g., the current position) to a second position (e.g., the target position) to avoid objects in the environment. In at least some examples, such a planning component 626 may determine that no such collision-free path exists and then provide a path that leads the vehicle 602 to a safe stop that avoids all collisions and / or otherwise reduces damage. Additionally or alternatively, the planning component 626 may determine the path that the vehicle 602 should follow based at least in part on data received from the door interface component 632.

[0087]

[0094] In at least one example, the vehicle computing device 604 may have one or more system controllers 628 which may be configured to control the steering, propulsion, brakes, safety, emitter, communication, and other systems of the vehicle 602. The system controllers 628 may communicate with and / or control the drive system 614 and / or the corresponding systems of other components of the vehicle 602.

[0088]

[0095] Memory 618 may further have one or more maps 630 that can be used by the vehicle 602 to navigate within the environment. For the purposes of this description, a map can be any number of data structures modeled in two, three, or N dimensions that can provide information about the environment, such as topology (like intersections), streets, mountain ranges, roads, terrain, and the general environment. In some examples, a map may include, but is not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), intensity information (e.g., Lidar information, radar information), spatial information (e.g., image data projected onto a mesh, individual "surfaces" (e.g., polygons associated with individual colors and / or intensities)), and reflectivity information (e.g., specularity information, retroreflection information, BRDF information, BSSRDF information, etc.). In one example, a map may include a three-dimensional mesh of the environment. In some examples, the vehicle 602 may be controlled at least in part based on a map 630. In other words, the map 630 can be used in conjunction with the positioning component 620, the perception component 622, the prediction component 624, and / or the planning component 626 to determine the position of the vehicle 602, detect objects in the environment, generate a path, and determine actions and / or trajectories for guiding it through the environment.

[0089]

[0096] In some examples, one or more maps 630 may be stored on a remote computing device (such as computing device 636) accessible via the network 642. In some examples, multiple maps 630 may be stored based on characteristics, for example, the type of entity, time of day, day of the week, season of the year, etc. Storing multiple maps 630 may have similar memory requirements but may increase the speed at which data in the maps can be accessed.

[0090]

[0097] As shown in Figure 6, the vehicle computing device 604 may have a door interface component 632. Door Interface Component 632 may be configured to perform the functions of door interface components 104 and 402, including controlling door operation and initiating communication. In various examples, door interface component 632 may receive one or more characteristics associated with detected objects from perception component 622 and / or sensor system 606. In some examples, door interface component 632 may receive environmental characteristics (e.g., environmental factors) and / or weather characteristics (e.g., weather factors such as snow, rain, and ice) from perception component 622 and / or sensor system 606. Although shown separately in Figure 6, door interface component 632 may be part of prediction component 624, planning component 626, or other components of vehicle 602.

[0091]

[0098] In various examples, Door Interface Component 632 makes decisions that may be used by the prediction component 624 and / or the planning component 626, based on those prediction components, etc. vehicle One or more predicted trajectories (e.g., direction of travel, velocity, etc.) 、 and / or may transmit to generate one or more predicted trajectories of an object (e.g., direction of travel, velocity, etc.). In some examples, the planning component 626 may determine one or more actions for the vehicle 602 (e.g., a base action, and / or sub-actions), such as a candidate vehicle trajectory. In some examples, Door Interface Component 632 may be configured to determine visual indicators (e.g., visual indicators 108, 110, 202, and / or 302) at least in part based on one or more actions for the vehicle 602.

[0092]

[0099] The authentication component 634 can provide the functionality to authenticate a user to enter the vehicle 602 or otherwise associate with the vehicle 602 by determining whether the user is authenticated or unauthenticated. The authentication component 634 can determine the user's authentication status at least in part on receiving data, machine-readable codes (e.g., Quick Response (QR) Code®, barcodes, radio frequency identification (RFID) tags, etc.) or biometric authentication (e.g., facial recognition, iris recognition, fingerprints, etc.) from an application on a mobile device associated with the user.

[0093]

[0100] The authentication component 634 can determine the authentication state at least in part based on data received from a camera (e.g., camera 112), a sensor, and / or proximity technology that detects the proximity of the user's mobile device (e.g., using Bluetooth®, ultrawideband, NFC, WiFi®, ultrasound, etc.). In various examples, the authentication component 634 may receive and / or access credentials from another entity, such as an application used by the user to rent a vehicle 602, and verify the authentication state at least in part based on the credentials.

[0094]

[0101] For the purposes of understanding, the components discussed herein (e.g., the localization component 620, the perception component 622, the prediction component 624, the planning component 626, one or more system controllers 628, one or more maps 630, the door interface component 632, and the authentication component 634) are described separately for illustrative purposes. However, the operations performed by various components may be combined or performed by any other component.

[0095]

[0102] While examples are given of the technologies described herein being implemented by the planning and / or model components of a vehicle, in some examples some or all of the technologies described herein may be implemented by another system of the vehicle, such as a secondary safety system. Generally, such an architecture may include a first computing device for controlling the vehicle 602, and a secondary safety system that operates on the vehicle 602 to verify the operation of the primary system and control the vehicle 602 to avoid collisions.

[0096]

[0103] In some instances, some or all aspects of the components discussed herein may include any model, technique, and / or machine learning technique. For example, in some instances, the components in memory 618 (and memory 640 as described below) may be implemented as a neural network.

[0097]

[0104] As described herein, an exemplary neural network is a technique that passes input data through sequentially connected layers to produce an output. Each layer in a neural network may also comprise another neural network, or any number of layers (whether convolutional or not). As can be understood in the context of this disclosure, a neural network may utilize machine learning, which can refer to a broad class of techniques that produce an output based on learned parameters.

[0098]

[0105] While described in the context of neural networks, all types of machine learning can be used in accordance with this disclosure. For example, machine learning techniques include regression techniques (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, MARS (multivariate adaptive regression splines), LOESS (locally estimated scatterplot smoothing)), instance-based techniques (e.g., ridge regression, LASSO (least absolute shrinkage and selection operator), Elastic Net, LARS (least-angle regression)), decision tree techniques (e.g., CART (classification and regression tree), ID3 (iterative dichotomiser 3), chi-squared automated interaction detection (CHAID), decision stock, conditional decision tree), Bayesian techniques (e.g., naive Bayes, Gaussian naive Bayes, polynomial naive Bayes, AODE (average one-dependence estimators), Bayesian confidence networks (BNN), Bayesian networks), clustering techniques (e.g., k-means clustering, k-median clustering, expectation maximization (EM), hierarchical clustering), association rule learning techniques (e.g., perceptron, backpropagation, Hopfield network, RBFN (Radial Basis Function)).This may include, but is not limited to, neural networks such as ResNet50, ResNet101, VGG, DenseNet, and PointNet. Additional architectural examples include neural networks such as ResNet50, ResNet101, VGG, DenseNet, and PointNet.

[0099]

[0106] In at least one example, the sensor system 606 may include lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measuring units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), and so on. The sensor system 606 may include multiple instances of each of these or other types of sensors. For example, the lidar sensors may include lidar sensors positioned at each of the corners, front, back, sides, and / or top of the vehicle 602. As another example, the camera sensors may include multiple cameras positioned at various locations on the exterior and / or interior of the vehicle 602. The sensor system 606 may provide input to the vehicle computing device 604. Additionally or alternatively, the sensor system 606 may transmit sensor data to one or more computing devices 636 via one or more networks 642 in near real-time, such as at a specific frequency, after a predetermined period of time has elapsed.

[0100]

[0107] Vehicle 602 may also include one or more emitters 608 that emit light and / or sound. The emitters 608 may include internal audio and visual emitters for communicating with the occupants of vehicle 602. Internal emitters may include, but are not limited to, speakers, lighting, signs, display screens, touchscreens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, headrest positioners, etc.). The emitters 608 may also include external emitters. External emitters may include, but are not limited to, lights for indicating the direction of travel, or other indicators of vehicle action (e.g., indicator lights, signs, light arrays, etc.), as well as one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for audible communication with pedestrians or one or more other nearby vehicles, one or more of which include acoustic beam steering technology.

[0101]

[0108] The vehicle 602 may also include one or more communication connections 610 that enable communication between the vehicle 602 and one or more other local or remote computing devices. For example, a communication connection 610 may facilitate communication with other local computing devices and / or drive systems 614 on the vehicle 602. A communication connection 610 may also enable the vehicle to communicate with other nearby computing devices (e.g., remote computing device 636, other nearby vehicles, etc.) and / or one or more remote sensor systems 644 to receive sensor data. A communication connection 610 may also enable the vehicle 602 to communicate with remote teleoperation computing devices or other remote services.

[0102]

[0109] The communication connection 610 may include physical and / or logical interfaces for connecting the vehicle computing device 604 to another computing device or to a network such as network 642. For example, the communication connection 610 may enable Wi-Fi-based communication via frequencies defined by the IEEE 802.11 standard, short-range radio frequencies such as Bluetooth, cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol that enables each computing device to interface with other computing devices.

[0103]

[0110] In at least one example, the vehicle 602 may include one or more drive systems 614. In some examples, the vehicle 602 may have a single drive system 614. In at least one example, if the vehicle 602 has multiple drive systems 614, the individual drive systems 614 may be located at opposing ends of the vehicle 602 (e.g., front and rear). In at least one example, the drive system 614 may include one or more sensor systems for detecting the conditions around the drive system 614 and / or the vehicle 602. For illustrative purposes only and not limiting, the sensor systems may include one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the wheels of the drive module, inertial sensors (e.g., inertial measuring units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the direction and acceleration of the drive module, cameras or other image sensors, ultrasonic sensors, lidar sensors, radar sensors, etc. for acoustically detecting objects around the drive module. Some sensors, such as wheel encoders, may be specific to the drive system 614. In some cases, the sensor system on the drive system 614 may overlap with or complement the corresponding system on the vehicle 602 (e.g., sensor system 606).

[0104]

[0111] The drive system 614 may include many of the vehicle systems, including a high-voltage battery, a motor for propelling the vehicle, an inverter for converting DC from the battery to AC for use by other vehicle systems, a steering system (which may be electric) including a steering motor and steering rack, a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate traction loss and maintain control, an HVAC system, lighting (e.g., headlights / taillights for illuminating the exterior of the vehicle), and one or more other systems (e.g., a cooling system, safety systems, an on-board charging system, other electrical components such as DC / DC converters, a high-voltage junction, high-voltage cables, a charging system, a charging port, etc.). Additionally, the drive system 614 may include a drive module controller for controlling the operation of various vehicle systems, which may receive and preprocess data from sensor systems. In some examples, the drive module controller may include one or more processors and memory communicably coupled to the one or more processors. The memory can store one or more modules and perform various functions of the drive system 614. Furthermore, the drive system 614 may also include one or more communication connections that enable each drive system to communicate with one or more other local or remote computing devices.

[0105]

[0112] In at least one example, the direct connection 612 may provide a physical interface for coupling one or more drive systems 614 to the body of the vehicle 602. For example, the direct connection 612 may enable the transfer of energy, fluid, air, data, etc., between the drive system 614 and the vehicle. In some examples, the direct connection 612 may further allow the drive system 614 to be removably fixed to the body of the vehicle 602.

[0106]

[0113] In at least one example, the location component 620, perception component 622, prediction component 624, planning component 626, one or more system controllers 628, one or more maps 630, door interface component 632, and authentication component 634 may process sensor data as described above and transmit their respective outputs to a computing device 636 via one or more networks 642. In at least one example, the location component 620, perception component 622, prediction component 624, planning component 626, one or more system controllers 628, one or more maps 630, door interface component 632, and authentication component 634 may transmit their respective outputs to a remote computing device 636 at a specific frequency, after a predetermined period of time, or in near real-time.

[0107]

[0114] In some examples, vehicle 602 may transmit sensor data to computing device 636 via network 642. In some examples, vehicle 602 may receive sensor data via network 642 from computing device 636 and / or remote sensor system 644. Sensor data may include raw sensor data and / or processed sensor data and / or representations of sensor data. In some examples, (raw or processed) sensor data may be transmitted and / or received as one or more log files.

[0108]

[0115] The computing device 636 may include a processor 638 and a memory 640 that stores a map component 646, a sensor data processing component 648, and a training component 650. In some examples, the map component 646 may include the ability to generate maps of various resolutions. In such examples, the map component 646 may transmit one or more maps to the vehicle computing device 604 for navigation purposes. In various examples, the sensor data processing component 648 may be configured to receive data from one or more remote sensors, such as the sensor system 606 and / or the remote sensor system 644. In some examples, the sensor data processing component 648 may be configured to process data and transmit the processed sensor data to the vehicle computing device 604, for example, to be used by the door interface component 632 and / or the authentication component 634. In some examples, the sensor data processing component 648 may be configured to transmit raw sensor data to the vehicle computing device 604.

[0109]

[0116] In some examples, the training component 650 can include the functionality to train a machine learning model and output evaluated trajectories. For example, the training component 650 can receive sensor data representing objects moving through the environment over time periods such as 0.1 milliseconds, 1 second, 3 seconds, 5 seconds, and 7 seconds. At least a portion of the sensor data can be used as input to train a machine learning model.

[0110]

[0117] In some examples, the training component 650 may be executed by the processor 638 to train a machine learning model based on training data. The training data may include a wide variety of data, such as sensor data, audio data, image data, map data, inertial data, vehicle state data, historical data (log data), or a combination thereof, associated with values ​​(e.g., desired classifications, inferences, predictions, etc.). Such values ​​may generally be referred to as "ground truth." For illustrative purposes, the training data may be used to determine the risk associated with an evaluated trajectory and therefore may include data captured by an autonomous vehicle that represents the environment associated with one or more classifications or decisions. In some examples, such classifications may be based on user input (e.g., user input indicating that the data represents a particular risk) or on the output of another machine learning model. In some examples, such labeled classifications (or, more generally, labeled outputs associated with the training data) may be referred to as ground truth.

[0111]

[0118] In some examples, the training component 650 may include the ability to train a machine learning model and output classification values. For example, the training component 650 may receive data representing labeled collision data (e.g., publicly available data, sensor data, and / or a combination thereof). At least a portion of the data can be used as input for training a machine learning model. Thus, as described herein, by providing data of a vehicle traveling through an environment, the training component 650 can be trained to output potential intersections associated with objects.

[0112]

[0119] In some cases, the training component 650 can include training data generated by a simulator. For example, the simulated training data could represent examples of a vehicle colliding with an object in the environment, or nearly colliding with an object in the environment, to provide additional training examples.

[0113]

[0120] The processor 616 of vehicle 602 and the processor 638 of computing device 636 may be any suitable processor capable of executing instructions and processing data to perform operations as described herein. Processors 616 and 638 may include, but are not limited to, one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or part of a device that processes electronic data and converts that electronic data into other electronic data that can be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs), gate arrays (e.g., FPGAs), and other hardware devices may also be considered processors insofar as they are configured to execute encoded instructions.

[0114]

[0121] Memory 618 and memory 640 are examples of non-temporary computer-readable media. Memory 618 and 640 may store the operating system and one or more software applications, instructions, programs, and / or data to implement the functions resulting from the methods and various systems described herein. In various implementations, memory may be implemented using appropriate memory technology, such as static random-access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash-type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmatic, and physical components, some of which shown in the accompanying drawings are merely illustrative examples relevant to the description herein.

[0115]

[0122] While Figure 6 illustrates a distributed system, in alternative examples, components of vehicle 602 may be associated with computing device 636, and / or components of computing device 636 may be associated with vehicle 602. That is, vehicle 602 may perform one or more functions associated with computing device 636, and vice versa.

[0116]

[0123] Figure 7 is a flowchart illustrating an exemplary process 700 for determining interaction using an exemplary door interface component. For example, some or all of the process 700 can be carried out by a vehicle computing device 604 or computing device 636, as described herein.

[0117]

[0124] In operation 702, the process may include receiving input associated with the door interface component. In some examples, operation 702 may include a vehicle computing device that implements the door interface component 104 to receive input from control 106 (e.g., a button on control 106). In various examples, to name a few, users may include occupants who have rented vehicle 602 for a fee, potential occupants who wish to rent vehicle 602, pedestrians, people who wish to communicate with vehicle 602, or people who wish to communicate with occupants inside vehicle 602. In various examples, users may be authorized to enter the vehicle, or may not be authorized to enter the vehicle (based on being unauthenticated). In some examples, operation 702 may include the door interface component 104 determining that a button has been moved from a first position to a second position (e.g., the button has been pressed by a user).

[0118]

[0125] Operation 704 may include determining whether the user is authenticated. In some examples, operation 704 may include determining the authentication status associated with the user. For example, a vehicle computing device may implement an authentication component configured to access data indicating the authentication status of the user and / or the vehicle (e.g., credentials associated with a login, rental status from an application, etc.). In various examples, determining the authentication status may be at least partially based on the proximity of the user's mobile device to the vehicle, the application on the user's mobile device to the vehicle, machine-readable codes, and / or biometric authentication associated with the user.

[0119]

[0126] In some cases, the authentication state can include an authenticated state in which the user is associated with a vehicle (e.g., authorized to ride the vehicle) or an unauthenticated state in which the user is not associated with a vehicle (e.g., the user has not performed the operation to rent the vehicle). The authentication component can determine the authentication state based on accessing data from memory, a database, or other storage device that stores data associated with the ride service associated with a collection of autonomous vehicles such as vehicle 602.

[0120]

[0127] If the user is authenticated (for example, "yes" in action 704), action 706 may follow. If the user is not authenticated (for example, "no" in action 704), action 704 may proceed to action 708.

[0121]

[0128] In operation 706, the process may include outputting a command to operate the autonomous vehicle's door between a first position and a second position, at least partially based on the input and authentication state being authenticated. In some examples, operation 706 may include a door interface component 104 that changes the door 122 of the vehicle 602 between a closed state associated with the first position and an open state associated with the second position. Operation 706 may also include outputting a first visual indicator, at least partially based on the authentication state being authenticated. In such examples, the first visual indicator may include instructions to operate a button to open a door (e.g., door 122), or instructions to use a specific door of the vehicle that opens most safely.

[0122]

[0129] In operation 708, the process may include generating a signal to be output by the door interface component or to be transmitted to a remote operating system, at least in part on the input and authentication status being in an unauthenticated state. In some examples, operation 708 may include transmitting a communication over a network to authenticate the user by a remote operator. In some examples, the remote operating system may be associated with an emergency service provider, and the communication may request assistance for an emergency scenario near the vehicle. Operation 708 may also include outputting a second visual indicator (and / or audio), at least in part on the authentication status being in an unauthenticated state. In such examples, the second visual indicator may include an instruction to initiate the rental of the vehicle (e.g., from the vehicle or from a remote operator associated with the ride service) or an instruction to initiate the generation of a communication for emergency assistance to an emergency service provider (e.g., control 106 may include a visual indicator 110 which may be a telephone symbol and a transmit wave symbol).

[0123]

[0130] Figure 7 illustrates an exemplary process by example of the present disclosure. These processes are presented as logical flow graphs, where individual operations represent a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, an operation represents a computer-executable instruction stored in one or more computer-readable storage media that, when executed by one or more processors, performs the described operation. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a particular function or implement a concrete abstract data type. The order in which operations are described is not intended to be interpreted restrictively, and any number of described operations can be omitted and / or combined in any order and / or in parallel to implement the process. For example, a process may include performing either operation 706 or operation 708.

[0124]

[0131] The methods described herein represent sequences of actions that can be implemented in hardware, software, or a combination thereof. In the context of software, a block represents a computer-executable instruction stored in one or more computer-readable media that, when executed by one or more processors, performs the described actions. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a particular function or implement a particular abstract data type. The order in which the actions are described is not intended to be interpreted restrictively, and any number of described actions can be combined in any order and / or in parallel to implement a process. In some examples, one or more actions of a method may be omitted entirely. For example, an action may include determining a first and second action by a vehicle on a selected track without determining the cost of each of one or more actions by the vehicle. Furthermore, the methods described herein can be combined, whole or in part, with each other or with other methods.

[0125]

[0132] The various techniques described herein may be implemented in the context of computer executable instructions or software, such as program modules, which are stored in computer-readable storage and executed by the processor of one or more computing devices as illustrated. Generally, program modules include routines, programs, objects, components, and data structures, and define operational logic to perform a particular task or implement a particular abstract data type.

[0126]

[0133] Other architectures may be used to implement the described functions and are intended to be within the scope of this disclosure. Furthermore, while specific distributions of responsibility are defined above for illustrative purposes, various functions and responsibilities may be distributed and divided in different ways depending on the circumstances.

[0127]

[0134] Similarly, software may be stored and distributed in various ways and using different means, and the specific software storage and execution configurations described above may be modified in many different ways. Therefore, software implementing the above techniques may not be limited to the forms of memory specifically described, but may be distributed on various types of computer-readable media.

[0128] Example clause

[0135] Any of the illustrative provisions in this section may be used in conjunction with any other illustrative provisions and / or any other examples or embodiments described herein.

[0129]

[0136] A: A system comprising a door interface component coupled to the door of an autonomous vehicle, one or more processors, and one or more non-temporary computer-readable media storing instructions executable by the one or more processors, wherein when an instruction is executed, the system is made to perform at least one of the following actions: receiving an input associated with the door interface component; determining an authentication state associated with a user; and outputting an instruction to operate the door of the autonomous vehicle between a first and a second position, at least partially based on the input and the authentication state, and at least partially based on the authentication state being authenticated, or transmitting a signal to be output by the door interface component or a signal to be transmitted to a remote operation system, at least partially based on the authentication state being unauthenticated.

[0130]

[0137] B: The system according to paragraph A, wherein the operation further comprises outputting at least one of a first visual indicator, at least partially based on the input, that the authentication state is authenticated, or a second visual indicator, at least partially based on the authentication state being unauthenticated.

[0131]

[0138] C: The system according to either paragraph A or B, wherein the authentication state is unauthenticated, the input is a first input, and the operation further comprises receiving a second input from the remote operation system to set the authentication state associated with the user to an authenticated state.

[0132]

[0139] D: The system described in any one of paragraphs AC, wherein the instruction is a first instruction, and the operation further comprises outputting a second instruction to cause the autonomous vehicle to navigate to an alternative location in the environment, at least in part on the fact that the authentication state is an unauthenticated state.

[0133]

[0140] E: The system according to any one of paragraphs AD, wherein the door is a first door, and the operation further comprises determining that the first door is not safe to enter, and outputting a visual indicator for using a second door, at least partially based on the authentication state being an authenticated state and the first door being not safe to enter.

[0134]

[0141] F: A method comprising receiving an input signal from a vehicle door interface component, determining the authentication state of the vehicle, wherein the authentication state indicates whether a user is authorized to enter the vehicle, and triggering an action based at least in part on the input signal and the authentication state, wherein the action includes a first action based at least in part on the authentication state being authenticated, or a second action based at least in part on the authentication state being unauthenticated, wherein the second action includes outputting one or more options to the user for controlling the vehicle, or transmitting an additional signal to a remote computing device.

[0135]

[0142] G: The method according to paragraph F, wherein the door interface component includes a button, the first action of which includes opening the door of the vehicle or outputting a visual indicator indicating that it is safe to open the door, and the second action of which includes outputting one or more options to a display device for presentation to the user, including a request for the vehicle to move to another location, a request to initiate communication for emergency assistance, or a request to rent the vehicle.

[0136]

[0143] H: The method according to either paragraph F or G, further comprising outputting at least one of a first visual indicator at least partially based on the input signal, which is at least partially based on the authentication state being authenticated, or a second visual indicator at least partially based on the authentication state being unauthenticated.

[0137]

[0144] I: The method according to any one of paragraph FH, further comprising receiving a second signal which sets the authentication state of the vehicle to the authenticated state, wherein the authentication state is an unauthenticated state and the input signal is a first signal.

[0138]

[0145] J: A method of any one of paragraph FI, further comprising outputting an instruction to navigate the vehicle to an alternative location in the environment, at least in part on the fact that the authentication state is the unauthenticated state.

[0139]

[0146] A method according to any one of paragraph FJ, further comprising: determining that the first door is not safe to enter; outputting a visual indicator for using the second door, at least in part on the authentication state being the authenticated state; and the first door being not safe to enter.

[0140]

[0147] L: A method according to any one of paragraph FK, wherein the authenticated state is at least partially based on at least one of the following: the proximity of the user's mobile device to the vehicle, an application on the user's mobile device to the vehicle, a machine-readable code, or biometric authentication.

[0141]

[0148] M: Determining the authentication state of the vehicle is done by any one of the methods in paragraph FL, which occurs before the reception of the input signal.

[0142]

[0149] N: The method of any one of paragraph FM, wherein the second action includes moving the vehicle from a first location to a second location in the environment, initiating a first communication to authenticate the user, or initiating a second communication to request emergency assistance from an emergency service provider.

[0143]

[0150] O: The method according to paragraph N, wherein the second action is performed by the vehicle computing device of the vehicle.

[0144]

[0151] P: The method of either paragraph N or O, further comprising outputting a set of commands to the vehicle's display device or the user's mobile device, at least in part on the input signal and the authentication state being the unauthenticated state, and receiving a selection of a command from the set of commands via the display device or the mobile device, wherein triggering the second action is at least in part on the command.

[0145]

[0152] Q: One or more non-temporary computer-readable media storing instructions executable by one or more processors, wherein when the instructions are executed, the one or more processors are caused to perform an operation comprising: receiving an input signal from the control of a vehicle door interface component; determining the authentication state of the vehicle, wherein the authentication state indicates whether a user is authorized to enter the vehicle; and triggering an action based at least in part on the input signal and the authentication state, wherein the action is a first action at least in part on the authentication state being authenticated, or a second action at least in part on the authentication state being unauthenticated, wherein the second action includes outputting one or more options for controlling the vehicle to the user, or sending an additional signal to a remote computing device.

[0146]

[0153] R: One or more non-temporary computer-readable media according to paragraph Q, further comprising outputting at least one of a first visual indicator at least partially based on the input signal that the authentication state is the authenticated state, or a second visual indicator at least partially based on the authentication state being the unauthenticated state.

[0147]

[0154] S: One or more non-temporary computer-readable media as described in either paragraph Q or R, wherein the door interface component includes a button, and the first action includes opening the door of the vehicle or outputting a visual indicator indicating that it is safe to open the door.

[0148]

[0155] T: The second action includes moving the vehicle from a first location to a second location in the environment, initiating a first communication to authenticate the user, or initiating a second communication to request emergency assistance from an emergency service provider, one or more non-temporary computer-readable media as described in any one of paragraph QS.

[0149]

[0156] While the exemplary clauses described above are written with respect to one specific implementation, it should be understood that, in the context of this specification, the content of the exemplary clauses may also be implemented through methods, devices, systems, computer-readable media, and / or other implementations. Additionally, any of the exemplary clauses AT may be implemented alone or in combination with one or more other exemplary clauses AT.

[0150] knot

[0157] While one or more examples of the techniques described herein are provided, various modifications, additions, substitutions, and equivalents thereof are included within the scope of the techniques described herein.

[0151]

[0158] In the examples, references are made to accompanying drawings that form part of them, illustrating specific examples of the subject matter described in the claims. It should be understood that other examples may be used, and that modifications or substitutions, such as structural changes, are possible. Such examples, modifications, or variations do not necessarily deviate from the scope of the intended subject matter of the claims. While the steps in this specification may be presented in a certain order, in some cases, the order may be changed so that certain inputs are provided at different times or in different orders without changing the function of the described systems and methods. The disclosed procedures can also be performed in different orders. In addition, the various calculations in this specification do not need to be performed in the order disclosed, and other examples using alternative orders of calculations would be readily implementable. In addition to rearranging the order, calculations may also be broken down into subcalculations of the same result.

Claims

1. It is a system, A door interface component attached to the door of an autonomous vehicle, One or more processors, The system comprises one or more non-temporary computer-readable media that store instructions executable by the one or more processors, and which, when an instruction is executed, causes the system to perform an operation, wherein the operation is Receiving an input signal from the door interface component of the autonomous vehicle, The authentication status of the autonomous vehicle is determined, and the authentication status indicates whether the user is authorized to enter the autonomous vehicle. This includes triggering an action based at least partially on the input signal and the authentication state, wherein the action is A first action based at least in part on the authentication status being authenticated, A system comprising a second action at least in part based on the authentication state being an unauthenticated state, wherein the second action includes outputting one or more options for controlling the autonomous vehicle to the user, or transmitting an additional signal to a remote computing device.

2. The operation described above is based at least partially on the input signal, A first visual indicator, which is at least partially based on the authentication state being the authenticated state, The system according to claim 1, further comprising outputting at least one second visual indicator based at least in part on the authentication state being the unauthenticated state.

3. The authentication state is the unauthenticated state, the input signal is the first input signal, and the operation is The system according to claim 1 or 2, further comprising receiving a second input signal from a remote operation system to set the authentication state associated with the user to the authenticated state.

4. The aforementioned instruction is a first instruction, and the aforementioned operation is, The system according to claim 1 or 2, further comprising outputting a second command to navigate the autonomous vehicle to an alternative location in the environment, at least in part on the fact that the authentication state is the unauthenticated state.

5. The aforementioned door is the first door, and the aforementioned operation is, Determining that the first door is not safe to enter, The system according to claim 1 or 2, further comprising outputting a visual indicator for using a second door, at least in part on the fact that the authentication state is the authenticated state and that the first door is not safe to enter.

6. The aforementioned door interface component includes a button, The first action includes opening the door of the autonomous vehicle, or outputting a visual indicator indicating that it is safe to open the door, The system according to claim 1 or 2, wherein the second action includes outputting one or more options on a display device for presentation to the user, including a request for the autonomous vehicle to move to another location, a request to initiate communication for emergency assistance, or a request to rent the autonomous vehicle.

7. The system according to claim 1 or 2, wherein the authenticated state is at least partially based on at least one of the following: proximity of the user's mobile device to the autonomous vehicle, an application on the user's mobile device to the autonomous vehicle, a machine-readable code, or biometric authentication.

8. The system according to claim 1 or 2, wherein determining the authentication state of the autonomous vehicle occurs before receiving the input signal.

9. The second action described above is, The autonomous vehicle is to move from a first position to a second position in the environment. Initiating a first communication to authenticate the user, or The system according to claim 1 or 2, comprising one of initiating a second communication to request emergency assistance from an emergency service provider.

10. The system according to claim 9, wherein the second action is performed by the vehicle computing device of the autonomous vehicle.

11. Based at least partially on the input signal and the authentication state being the unauthenticated state, a set of commands is output to the autonomous vehicle's display device or the user's mobile device. Further including receiving a selection of a command from the set of commands via the display device or the mobile device, The system according to claim 9, wherein triggering the second action is at least partially based on the command.

12. It is a method, Receiving input signals from the vehicle's door interface component, The authentication status of the vehicle is determined, and the authentication status indicates whether the user is authorized to enter the vehicle. To trigger an action based at least partially on the input signal and the authentication state, wherein the action is A first action based at least in part on the authentication status being authenticated, A method comprising a second action based at least in part on the authentication state being an unauthenticated state, wherein the second action includes outputting one or more options for controlling the vehicle to the user, or transmitting an additional signal to a remote computing device.

13. Based at least partially on the aforementioned input signal, A first visual indicator, which is at least partially based on the authentication state being the authenticated state, A second visual indicator, which is at least partially based on the authentication state being the unauthenticated state, The method according to claim 12, further comprising outputting at least one of the following.

14. The authentication state is the unauthenticated state, and the input signal is the first input signal. The method according to either claim 12 or 13, further comprising receiving a second input from a remote operating system and setting the authentication state associated with the user to the authenticated state.

15. A computer program that, when executed on a computer, includes coded instructions that implement the method according to claim 12 or 13.

Images