System and method for controlling state transitions using a vehicle controller

Abstract

The present disclosure relates to systems and methods for controlling state transitions using a vehicle controller. In particular, a computing system may initiate an autonomous vehicle into an unauthorized state at startup. The computing system may receive an authorization request. The computing system determines whether the authorization request includes a request to enter a first operating mode or a request to enter a second operating mode, wherein the first operating mode is associated with operating the autonomous vehicle without a human operator and the second operating mode is associated with being able to operate the autonomous vehicle by a human operator. The computing system may transition the autonomous vehicle from an unauthorized state to a standby state in response to determining that the authorization request includes a request to enter the first operating mode, or transition the autonomous vehicle from an unauthorized state to a manual control state in response to determining that the authorization request is a request to enter the second operating mode.

Description

[0001] This application is a divisional application, and the parent case it is targeting is a Chinese invention patent application with an application date of March 30, 2021, the applicant is "Aurora Operations Company", the invention name is "System and method for controlling state transitions using a vehicle controller", and the application number is 202180026776.9.

[0002] Related Applications

[0003] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 001,838, filed on March 30, 2020, and U.S. Patent Application No. 16 / 910,554, filed on June 24, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0004] The present disclosure generally relates to autonomous vehicles. More specifically, the present disclosure relates to managing state transitions of an autonomous vehicle using a vehicle controller. Background Art

[0005] An autonomous vehicle is a vehicle that is capable of sensing its environment and navigating without human input. In particular, an autonomous vehicle may observe its surroundings using various sensors, and may attempt to understand the environment by performing various processing techniques on the data collected by the sensors. Given knowledge of its surroundings, an autonomous vehicle may identify appropriate paths of motion for navigating through such surroundings. Some autonomous vehicles can switch between an operator-controlled mode and a computer-controlled mode. Managing the transitions of the various states is important for the safe operation of autonomous vehicles. Summary of the invention

[0006] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0007] An exemplary aspect of the present disclosure relates to a computer-implemented method. The method may include initiating an autonomous vehicle to an unauthorized state at startup. The method may include receiving an authorization request. The method may include determining whether the authorization request includes a request to enter a first operating mode or a request to enter a second operating mode, wherein the first operating mode is associated with operating the autonomous vehicle without a human operator and the second operating mode is associated with being able to operate the autonomous vehicle by a human operator. The method may include transitioning the autonomous vehicle from an unauthorized state to a standby state in response to determining that the authorization request includes a request to enter the first operating mode, or transitioning the autonomous vehicle from an unauthorized state to a manual control state in response to determining that the authorization request is a request to enter the second operating mode.

[0008] Other aspects of the disclosure relate to various systems, apparatus, non-transitory computer-readable media, user interfaces, and electronic devices.

[0009] These and other features, aspects and advantages of various embodiments of the present disclosure will be better understood with reference to the following description and appended claims.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] A detailed discussion of embodiments for those of ordinary skill in the art is set forth in the specification with reference to the accompanying drawings, in which:

[0011] Figure 1 A block diagram of an exemplary autonomous vehicle is depicted, according to an exemplary embodiment of the present disclosure.

[0012] Figure 2 A block diagram of a vehicle controller is depicted according to an exemplary embodiment of the present disclosure.

[0013] Figure 3 A block diagram of a vehicle controller is depicted according to an exemplary embodiment of the present disclosure.

[0014] Figure 4 A state machine for mode transition management according to an exemplary embodiment of the present disclosure is depicted.

[0015] Figure 5 A state machine for mode transition management according to an exemplary embodiment of the present disclosure is depicted.

[0016] Figure 6 A flow chart of an exemplary method according to an exemplary embodiment of the present disclosure is depicted.

[0017] Figure 7 An exemplary system having means for performing operations and functions according to exemplary aspects of the present disclosure is depicted.

[0018] Figure 8 Depicted are exemplary system components according to exemplary aspects of the present disclosure. DETAILED DESCRIPTION

[0019] In general, the present disclosure relates to a system for allowing a vehicle controller (e.g., a vehicle interface module "VIM") to control mode transitions of an autonomous vehicle to ensure that transitions between different modes are safe. An autonomous vehicle can have two operating modes. A first operating mode is associated with a series of states of an autonomous vehicle that does not involve human control of the vehicle (e.g., by a human operator) at any point during the operating mode (e.g., a "no vehicle operator" (NVO) mode). A second operating mode can be associated with a series of states that can include a human operator (e.g., a "vehicle operator" (VO) mode). Thus, the second operating mode includes human-controlled states and / or computer-controlled states that allow human intervention.

[0020] To ensure that the vehicle does not directly transition between the first operating mode and the second operating mode, when the vehicle comes online, the vehicle controller (VIM) is initialized to an unauthorized mode. When in the unauthorized mode, the vehicle controller can restrict the autonomous vehicle from performing any tasks. The vehicle controller can receive an authorization request (e.g., from an onboard autonomous system of the vehicle, a physical key interface, a remote computing system, etc.). The vehicle controller determines whether the authorization request is a request to enter the first operating mode (NVO mode) or the second operating mode (VO mode) based on the received authorization request. If the vehicle controller determines that the authorization request is a request to enter the first operating mode, the vehicle controller transitions to a standby state, from which the vehicle controller can later transition to a fully autonomous state. If the vehicle controller determines that the authorization request is a request to enter the second operating mode, the vehicle controller transitions to a human operator controlled state. The vehicle controller can transition from a human operator controlled state to a computer driven state. Once the first operating mode or the second operating mode has been entered, the vehicle controller cannot transition to another operating mode without first transitioning to an unauthorized state.

[0021] For example, an autonomous vehicle can be turned on and its system can be started to an initial unauthorized state. Once in the unauthorized state, the vehicle controller, the associated autonomous system, and the vehicle itself can wait until an authorization request is issued. The authorization request can be received by the vehicle controller from another computing system (such as an autonomous system or a remote service system associated with the autonomous vehicle). The vehicle controller can determine whether the authorization request includes a request to enter a first operating mode or a request to enter a second operating mode. In this example, the request from the remote computing system includes an authorization request to enter the first operating mode. After verifying that the vehicle is ready to enter the first operating mode, the vehicle controller can transform the autonomous vehicle into a standby state associated with the first operating mode. When in the first operating mode, any user input (e.g., from a passenger in the vehicle) can be ignored. Therefore, once in the first operating mode, the autonomous vehicle or vehicle controller cannot be transformed into a human driving mode, but is restored to an unauthorized state to enter the second operating mode. In this way, the vehicle controller can safely and effectively help the autonomous vehicle smoothly transition between modes and states.

[0022] An autonomous vehicle may include a vehicle computing system. Among other functions, the vehicle computing system may be responsible for creating control signals required to effectively control the autonomous vehicle. The vehicle computing system may include an autonomous computing system. The autonomous computing system may include one or more systems that enable the autonomous vehicle to plan a route, receive sensor data about the environment, perceive objects (e.g., other vehicles) within the vehicle's surroundings, predict the movement of objects within the surroundings, generate trajectories based on the sensor data and the perceived / predicted movement of the objects, and based on the trajectories, send control signals to the vehicle control system so that the autonomous vehicle can move to its target destination. To accomplish these operations, the autonomous computing system may include, for example, a perception system, a prediction system, and a motion planning system.

[0023] To help maintain awareness of the vehicle's surroundings, the vehicle computing system (e.g., a perception system) can access sensor data from one or more sensors to identify static objects and / or dynamic objects (actors) in the autonomous vehicle's environment. To help determine its position within the environment (and relative to these objects), the vehicle computing system can use a positioning system and / or a communication system to determine its current position. Based on this position information, the vehicle computing system can access map data (e.g., HD map data, etc.) to determine the autonomous vehicle's current position relative to other objects in the world (e.g., bicycles, pedestrians, other vehicles, buildings, etc.), as well as map features such as, for example, lane boundaries, curbs, etc.

[0024] The vehicle computing system (e.g., a perception system) may utilize sensor data to identify one or more objects in the local environment of the autonomous vehicle. The sensor data may include, but is not limited to, data acquired via a camera sensor, a LIDAR sensor, and a RADAR sensor. Using the sensor data, the vehicle computing system may generate perception data (e.g., current location, speed, heading, shape / size, etc.) describing one or more objects in the vicinity of the autonomous vehicle.

[0025] The generated perception data may be used to predict the future motion of the object(s). For example, the vehicle computing system (e.g., a prediction system) may use the perception data to generate a prediction of the motion of one or more objects as an object trajectory including one or more future coordinates / points. In some implementations, the perception and prediction functions of the vehicle computing system may be included in the same system.

[0026] A vehicle computing system (e.g., a motion planning system) can use the perception data, prediction data, map data, and / or other data to generate a motion plan for the vehicle. For example, a route can describe a specific path for an autonomous vehicle to travel from a current location to a destination location. The vehicle computing system can generate potential trajectories for the autonomous vehicle to follow as it traverses the route. Each potential trajectory can be executed by the autonomous vehicle (e.g., feasible for the vehicle control system to implement). Each trajectory can be generated to include a specific amount of travel time (e.g., 8 seconds, etc.).

[0027] An autonomous vehicle may select and implement a trajectory of an autonomous vehicle to navigate a specific segment of a route. For example, a trajectory (or a representation thereof) may be provided to a vehicle controller. The vehicle controller may include a vehicle interface module (VIM) that may obtain data from one or more vehicle systems (such as an autonomous system, a collision mitigation system, and / or other systems). The vehicle controller may also communicate with an autonomous computing system associated with the autonomous vehicle. The autonomous computing system may be used as a communication interface to help confirm the state / condition / condition / capability of the vehicle. The vehicle controller may utilize the data it obtains and interface with (one or more) vehicle control systems to help implement the trajectory followed by the vehicle. For example, the vehicle controller may convert motion planning / trajectory data into instructions for controlling an autonomous vehicle (including adjusting the vehicle steering "X" degrees, applying a certain amount of braking force, etc.). (One or more) vehicle control systems may generate specific control signals for autonomous vehicles (e.g., adjusting steering, braking, speed, etc.). Specific control signals may enable the autonomous vehicle to move according to the selected trajectory. The vehicle controller may have other functions, such as, for example, safely transforming the mode and / or state of the vehicle, as described herein.

[0028] The vehicle controller may serve as an interface between the autonomous system and the autonomous vehicle itself. In some examples, the vehicle controller may include multiple control lanes. In some examples, the vehicle controller may have a dual lane control architecture. The vehicle controller may include a first control lane and a second control lane. Each lane control may constitute a full system that may be used to receive signals from the autonomous system, track the current state of the autonomous vehicle, and send control signals from the autonomous system to the vehicle itself.

[0029] In some examples, the vehicle controller can designate one of the first control lane or the second control lane as a primary lane and the other as a secondary lane (e.g., based on the capabilities of the first control lane and the capabilities of the second control lane). The vehicle controller can use the primary lane to control the autonomous vehicle so that the first control lane and the second control lane do not simultaneously attempt to control the operation of the autonomous vehicle.

[0030] In some examples, the vehicle controller can enable a first control lane to implement a motion plan from an autonomous system and silence a second control lane. By designating the first control lane as the primary lane, the autonomous vehicle can be controlled when the motion plan is implemented by the first control lane, but cannot be controlled when the motion plan is implemented by the second control lane.

[0031] In some examples, the primary lane may be synchronized with the secondary lane to ensure that both maintain accurate tracking of the vehicle's current state (including, but not limited to, the vehicle's current operating mode and current states within that operating mode). Thus, any request to change operating modes or change states of operating modes needs to be implemented in both lanes simultaneously or nearly simultaneously. Thus, if one controlling lane fails for any reason, the secondary controlling lane is ready to continue immediately.

[0032] The vehicle controller may include various subsystems to efficiently provide various services within the autonomous vehicle. The vehicle controller may obtain input from various sources including, but not limited to, the autonomous system, a vehicle platform interface, a collision mitigation interface, a network communication interface (e.g., for communicating with a remote computing system).

[0033] The following is a description of a state machine associated with a mode switch by a vehicle controller. The state machine may be implemented by a vehicle mode management system. The vehicle controller may also ensure that the primary lane control and the secondary lane control are synchronized (e.g., have the same state) and that both are synchronized with the state of the autonomous vehicle itself. Thus, each time the vehicle controller changes from one mode to another or from one state to another, the switch may be confirmed with the autonomous vehicle. If the autonomous vehicle does not confirm or authorize a change in mode or state, the vehicle controller may remain in a previous state or node.

[0034] When the vehicle controller (along with the rest of the autonomous vehicle) is started (or powered on), the vehicle controller can initialize the vehicle to an unauthorized state. The unauthorized state is not part of the first operating mode or the second operating mode. Instead, the unauthorized state can operate as a neutral state in which the autonomous vehicle does not take any action and waits for an authorization request.

[0035] The vehicle controller may receive the authorization request. The authorization request may be generated by one or more different sources, including but not limited to a system within the autonomous vehicle itself (e.g., an autonomous system), a remote computing system associated with the autonomous vehicle (e.g., a service system managing a ride-sharing service), or a physical key interface (e.g., via a user physically inserting a key or key-like device into the autonomous vehicle to initiate the autonomous vehicle into a drivable mode).

[0036] In response to receiving the authorization request, the vehicle controller can determine whether the request includes a request to enter a first operating mode or a request to enter a second operating mode. The first operating mode and the second operating mode are distinct modes such that when the vehicle is in either mode, the vehicle controller cannot change to a state associated with the other mode without first returning to an unauthorized state.

[0037] The first operating mode may be a mode in which no operator is present in the vehicle. When in the first operating mode, any user interaction with the vehicle may be ignored. When the vehicle controller determines that the authorization request includes a request to enter the first operating mode, the vehicle controller may cause the autonomous vehicle to change mode to the first operating mode and enter a standby state. As described above, before entering the first operating mode, the vehicle controller may send a request to the autonomous vehicle to determine whether the autonomous vehicle is ready to enter the first operating mode. Only upon receiving a confirmation that the autonomous vehicle is ready to enter the first operating mode does the vehicle controller change the mode of the vehicle and enter the standby state.

[0038] The standby state may be a state in which the autonomous vehicle may be ready to enter a fully computer controlled state. When in the standby state, the autonomous vehicle is immobile and cannot react to user input. Once the autonomous vehicle has signaled that it is ready to transition to a fully computer controlled state, the vehicle controller may wait for a state transition request from the autonomous system (or from a remote computing system). In some examples, the state transition request may be received from an autonomous system associated with the autonomous vehicle. In some examples, the state transition request may be received from a remote server system that controls or is associated with the autonomous vehicle.

[0039] Alternatively, once the autonomous vehicle has signaled that it is ready to transition to computer control, the vehicle controller may automatically transition from the standby state to the full computer control state without requiring additional state transition requests.

[0040] Once the vehicle controller has transitioned to the computer controlled state, the autonomous vehicle can be fully controlled by the autonomous system associated with the autonomous vehicle. The actions taken by the autonomous vehicle can be in response to commands generated by the autonomous system. In some embodiments, a remote system (e.g., a remote assistance system) can take precedence over the autonomous system to remotely assist the autonomous vehicle. While in the computer controlled state, the vehicle controller can receive a state transition request. The state transition request can request that the vehicle leave the computer controlled state and enter the standby state. Note that the state machine is configured so that the only state that can be reached from the computer controlled state is the standby state.

[0041] In some examples, the autonomous vehicle may ensure that movement has stopped before entering the standby state. Thus, when the vehicle controller receives a state transition request, the autonomous vehicle may be instructed by the autonomous system or the vehicle controller itself to take an action that will stop the autonomous vehicle. As with all other state transitions, the vehicle controller may synchronize the intent to change state with the service provider and any other controlled lanes. Once the intended transition is synchronized between all relevant components, the vehicle controller may transition the autonomous vehicle from the computer-controlled state to the standby state.

[0042] Once in the standby state, the vehicle controller may receive a transition request to the unauthorized state. If a request to transition to the second operating mode is received while in the standby state, the vehicle controller first transitions to the unauthorized state before transitioning to the second operating mode. In this way, the vehicle controller ensures that the autonomous vehicle does not transition directly between the first operating mode and the second operating mode.

[0043] In some examples, the autonomous vehicle can be powered off and restarted so that it is reinitialized to the unauthorized state before transitioning to the second operating mode. Alternatively, once in the standby state, the vehicle controller can transition to the unauthorized state without being powered off, and while in the unauthorized state, receive authorization to transition to the second operating mode.

[0044] When the vehicle controller receives an authorization request to enter an operating mode and determines that the request is to enter a second operating mode, the vehicle controller can transition to a user control state. The transition can be conditional on the vehicle controller synchronizing with the autonomous vehicle itself and any other controlled lanes (e.g., via an autonomous computing system). When in the manual control state, the operator can operate the on-board controls to drive the autonomous vehicle.

[0045] In some examples, while in the manual control state, the vehicle controller may receive a request to enter full computer control of the autonomous vehicle. The computer control state associated with the second operating mode is different from the full computer control state associated with the first operating mode. For example, in the first operating mode, the computer control state may ignore any human intervention and may not transition to the operator control state. In contrast, the computer control state associated with the second operating mode may transition to the operator control state upon request and may be responsive to user input. For example, if the user presses the brake pedal, the vehicle controller may respond by exiting the computer control state and re-entering the operator control state.

[0046] When in the computer control state associated with the second operating mode, the vehicle controller can receive a request to transition from the computer control state to the manual control state. As described above, the request can be generated by a user's interaction with the autonomous vehicle, by the autonomous system, or by a remote computing system that directs the autonomous vehicle back to operator control.

[0047] While in the manual control state, the vehicle controller may receive a request to transition to the unauthorized state. The request may be part of a request to transition to the first operating mode, or may be included in a request to cease operation and power off. The vehicle controller may establish that the service provider and the control lane are aware of the transition request and confirm that the transition is authorized. The vehicle controller may transition to the unauthorized state.

[0048] Each of the above states may have one or more sub-states that facilitate smooth transitions between states. A description of possible exemplary sub-states and how they interact with each other is provided below. As an example, an unauthorized state may include multiple possible sub-states. A first sub-state may be a sub-state in which the vehicle controller has not yet received an authorization request. The vehicle controller may transition to a subsequent sub-state in response to receiving an authorization request. For example, if the request is to enter a first operating mode, the vehicle controller may enter a second sub-state, or if the request is to enter a second operating mode, the vehicle controller may enter a third sub-state.

[0049] In both the second and third sub-states, the vehicle controller may request authorization from a third party (e.g., the vehicle itself) to ensure that the autonomous vehicle is ready to transition to another state. If the autonomous controller receives a receipt or authorization to transition to the requested mode, the vehicle controller may transition from the second sub-state to the standby state if entry into the first operating mode is requested, or from the third sub-state to the manual control state if entry into the second operating mode is requested.

[0050] However, if a receipt or response is not received within a predetermined time frame, the vehicle controller may not transition to the desired control mode, but may transition to a state indicating a potential failure. For example, if the vehicle controller exceeds the allocated time for receiving a receipt while in the second sub-state and waiting to transition to the first operating mode, the vehicle controller may transition to a fourth sub-state, which indicates a potential transition failure. The fifth sub-state may perform the same or similar function to the second operating mode. When in the fourth or fifth sub-state, respectively, if the authorization request is successful (e.g., the vehicle controller receives a receipt of the autonomous vehicle's intention to transition state), the vehicle controller may still transition to the first operating mode or the second operating mode. However, if a receipt is not received within a predetermined time frame, the vehicle controller may transition back to the first sub-state and wait for a new authorization request.

[0051] Similarly, when first entering the standby state, the initial sub-state can be a first nominal sub-state in which the vehicle controller waits to receive a state transition request to enter the computer controlled state. When the vehicle controller receives a request to transition to the computer controlled state, the vehicle controller can locally or remotely ensure that the state switch intent of the vehicle controller matches the intent of one or more other controlling lanes. For example, before transitioning to another sub-state within the standby state, the vehicle controller can request information from one or more other systems to ensure that all mode controllers (e.g., mode managers controlling lanes) have matching intent to transition to the computer controlled state.

[0052] The vehicle controller may transition to a sub-state in which the vehicle controller requests entry into a computer controlled state from the autonomous vehicle itself. This may occur, for example, once the vehicle controller has confirmed an intent to control a match between lanes. As described above, the vehicle controller may wait for confirmation from the vehicle itself before transitioning to the computer controlled state. If no confirmation is received within a predetermined time, the vehicle controller may enter a failed state. If, while in the failed state, a delayed confirmation arrives within a predetermined time period, the vehicle controller may transition to the computer controlled state. However, if no confirmation arrives, the vehicle controller may transition back to the first nominal sub-state to await further state transition requests.

[0053] When in the computer controlled state, the vehicle controller may initially transition to a first nominal sub-state associated with the computer controlled state. The vehicle controller may remain in this state until a transition request is received to transition back to the standby state. As described above, once the vehicle controller has received the state transition request, the vehicle controller may request authorization from the vehicle to transition to the standby state. If an acknowledgement is received within a predetermined time, the vehicle controller may transition to the standby state. However, if an acknowledgement is not received within a predetermined time, the vehicle controller may enter a request failed sub-state awaiting a delayed acknowledgement. If there is no upcoming delayed acknowledgement, the vehicle controller will transition back to the first nominal sub-state associated with the computer controlled state and await further requests.

[0054] When in the manual control state, the vehicle controller may first enter a nominal sub-state in which the vehicle controller waits to receive a request to transition to a computer control state associated with the second operating mode. The vehicle controller may remain in this state until a transition request to the computer control state is received. As described above, for example, once the vehicle controller has received a state transition request, the vehicle controller may request authorization from the vehicle to transition to the computer control state. If a confirmation is received within a predetermined time, the vehicle controller may transition to the computer control state. However, if a confirmation is not received within a predetermined time period, the vehicle controller may enter a request failure sub-state and wait for a delayed confirmation. If no delayed confirmation is forthcoming, the vehicle controller may transition back to the first nominal sub-state associated with the manual control state and wait for further requests.

[0055] In the computer control state, the vehicle controller may wait to receive a state transition request. When a state transition request is received in the computer control state associated with the second operating mode, the vehicle controller may immediately transition to the manual control state. Once in the manual control state, the vehicle controller may request authorization for the state transition and confirm that state synchronization is maintained.

[0056] An end-to-end example of the technology described herein is provided below. A vehicle controller may initiate an autonomous vehicle into an unauthorized state upon startup. The vehicle controller may receive an authorization request. The authorization request may be associated with an autonomous computing system onboard the autonomous vehicle. The authorization request may be transmitted from a remote computing system remote from the autonomous vehicle. In some embodiments, the authorization request may be associated with a physical key interface.

[0057] The vehicle controller may determine whether the authorization request includes a request to enter a first operating mode or a second operating mode, wherein the first operating mode is associated with an autonomous vehicle without a human operator and the second operating mode is associated with an autonomous vehicle including a human operator.

[0058] The vehicle controller may transition the autonomous vehicle to a standby state in response to determining that the authorization request includes a request to enter a first operating mode, or transition the autonomous vehicle to a manual control state in response to determining that the authorization request is a request to enter a second operating mode. When in the first operating mode, the autonomous vehicle may not respond to operator input.

[0059] While in the standby state, the vehicle controller may receive a request to transition to the first computer-controlled state. In response to receiving the request, the vehicle controller may determine that the autonomous vehicle is ready to enter the first computer-controlled state. In response to determining that the autonomous vehicle is ready to enter the first computer-controlled state, the vehicle controller may transition from the standby state to the first computer-controlled state.

[0060] While in the first computer-controlled state, the vehicle controller may receive a request from the autonomous system to exit the first computer-controlled state. The vehicle controller may send a control to the autonomous vehicle that causes the autonomous vehicle to reach a safe parking point and become immobile. In response to determining that the autonomous vehicle has become immobile, the vehicle controller may transition from the first computer-controlled state to a standby state.

[0061] While in the standby state, the vehicle controller may receive a request to transition to the second operating mode. The vehicle controller may transition from the standby state to the unauthorized state. Alternatively, the vehicle controller may send a command to cause the autonomous vehicle to power off and restart to the unauthorized state.

[0062] While in the manual control state, the vehicle controller may receive a request to transition to a second computer-controlled state. The vehicle controller may determine that the autonomous vehicle is ready to enter the second computer-controlled state. In response to determining that the autonomous vehicle is ready to enter the second computer-controlled state, the vehicle controller may transition from the manual control state to the second computer-controlled state.

[0063] The vehicle controller may send a state transition request to the autonomous vehicle, the state transition request including information describing a planned transition to the second computer-controlled state. The vehicle controller may receive confirmation from the autonomous vehicle that the autonomous vehicle state has been updated to the second computer-controlled state. In response to receiving confirmation that the autonomous vehicle has updated the autonomous vehicle state to the second computer-controlled state, the vehicle controller may update the vehicle controller state to the second computer-controlled state.

[0064] The vehicle controller may determine whether a confirmation has been received from the autonomous vehicle within a predetermined time limit. In response to determining that a confirmation has not been received from the autonomous vehicle within the predetermined time limit, the vehicle controller may enter an error or failure state and remain in the original state.

[0065] While in the second computer-controlled state, the vehicle controller can detect an operator input to the autonomous vehicle. In response to detecting the operator input, the vehicle controller can transition from the second computer-controlled state to the manual control state.

[0066] When in the manual control state, the vehicle controller may receive a request to transition to the first operating mode. The vehicle controller may send a command to cause the autonomous vehicle to power off and then restart to an unauthorized state.

[0067] In some examples, the vehicle controller may include a first control lane and a second control lane. The vehicle controller may facilitate a transition to the standby state by, for example, confirming that both the first control lane and the second control lane indicate a transition to the standby state. In another example, the vehicle controller may facilitate a transition to the manual control state by, for example, confirming that both the first control lane and the second control lane indicate a transition to the manual control state.

[0068] The systems and methods described herein provide many technical effects and benefits. More specifically, the systems and methods of the present disclosure provide improved techniques for controlling mode switching in autonomous vehicles. For example, the vehicle controller ensures that there is no situation where the autonomous vehicle without an operator can change the mode to a manually controlled state, because the vehicle controller ensures that the state associated with the absence of an operator cannot be transformed into a state where the operator is expected to be present. The vehicle controller achieves this by creating two separate state paths, one for the operator operation mode and one for the non-operator operation mode. The vehicle controller then ensures that the vehicle cannot transition between the two state paths without first entering a neutral state that is not associated with either state. Controlling the state of the autonomous vehicle in this way ensures that the state transitions of the autonomous vehicle are controlled in a safer and more efficient manner.

[0069] Additionally, by controlling the mode transitions to ensure safety at the vehicle controller, the autonomous vehicle can avoid any additional monitoring of the state during the transition process. By reducing the amount of data analysis and communication required during the state transition process, the vehicle controller can reduce the number of processor cycles used by the autonomous vehicle, reduce the amount of memory required to store information, and thereby reduce the energy consumption of the autonomous vehicle system as a whole. Reducing energy consumption increases the useful battery life of any battery system included in the autonomous vehicle.

[0070] The autonomous vehicle technology described herein can help improve the safety of passengers of autonomous vehicles, improve the safety of the surrounding environment of autonomous vehicles, improve the experience of passengers and / or operators of autonomous vehicles, and provide other improvements as described herein. In addition, the autonomous vehicle technology of the present disclosure can help improve the ability of autonomous vehicles to effectively provide vehicle services to others and support various members of the community in which the autonomous vehicle is operating, including people with limited mobility and / or people who are underserved by other transportation options. In addition, the autonomous vehicles of the present disclosure can reduce traffic congestion in the community, as well as provide alternative forms of transportation that can provide environmental benefits.

[0071] Various devices can be configured to perform the methods and processes described herein. For example, a computing system may include (one or more) initiating units, (one or more) receiving units, (one or more) request analysis units, (one or more) conversion units and / or other devices for performing the operations and functions described herein. In some embodiments, one or more of the units can be implemented separately. In some embodiments, one or more units can be part of one or more other units or included in one or more other units. These devices may include (one or more) processors, (one or more) microprocessors, (one or more) graphics processing units, (one or more) logic circuits, (one or more) dedicated circuits, (one or more) application-specific integrated circuits, programmable array logic, (one or more) field programmable gate arrays, (one or more) controllers, (one or more) microcontrollers and / or other suitable hardware. The device may also or alternatively include a software control device implemented, for example, with a processor or logic circuit. The apparatus may include or otherwise have access to memory, such as, for example, one or more non-transitory computer-readable storage media—such as random access memory, read-only memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, flash / (one or more) other memory devices, (one or more) data registers, (one or more) databases, and / or other suitable hardware.

[0072] The device may be programmed to execute one or more algorithms for performing the operations and functions described herein. For example, the device may be configured to initiate the autonomous vehicle into an unauthorized state at startup. For example, the vehicle controller may set the initial state of the autonomous vehicle at startup to an unauthorized state that is not associated with either the first operating mode or the second operating mode. The initiation unit is an example of a device for initiating the autonomous vehicle into an unauthorized state at startup.

[0073] The device may be configured to receive the authorization request. For example, the vehicle controller may receive the authorization request from an autonomous system, a remote computing system, or a physical key interface associated with the autonomous vehicle. The receiving unit is an example of a device for receiving the authorization request.

[0074] The device can be configured to determine whether the authorization request includes a request to enter a first operating mode or a second operating mode, wherein the first operating mode is associated with operating the autonomous vehicle without a human operator and the second operating mode is associated with being able to operate the autonomous vehicle by a human operator. For example, the vehicle controller can determine the specific operating mode requested based on data included in the authorization request. In some examples, the requested operating mode can be determined based on the source of the authorization request. The request analysis unit is an example of a device for determining whether the authorization request includes a request to enter a first operating mode or a request to enter a second operating mode, wherein the first operating mode is associated with operating the autonomous vehicle without a human operator and the second operating mode is associated with being able to operate the autonomous vehicle by a human operator.

[0075] The device can be configured to transition the autonomous vehicle from an unauthorized state to a standby state in response to determining that the authorization request includes a request to enter a first operating mode, or to transition the autonomous vehicle from an unauthorized state to a manual control state in response to determining that the authorization request is a request to enter a second operating mode. For example, the vehicle controller can transition to the requested operating mode (first or second operating mode) based on the authorization request. The vehicle controller can also synchronize with the autonomous vehicle or the secondary lane controller to ensure that the entire autonomous vehicle system has the same state at the same time. The transition unit is an example of a device for transitioning the autonomous vehicle from an unauthorized state to a standby state in response to determining that the authorization request includes a request to enter a first operating mode, or to transition the autonomous vehicle from an unauthorized state to a manual control state in response to determining that the authorization request is a request to enter a second operating mode.

[0076] With reference to the accompanying drawings, exemplary embodiments of the present disclosure will be discussed in more detail.

[0077] Figure 1 1 depicts a block diagram of an exemplary system 100 for controlling navigation of a vehicle according to an exemplary embodiment of the present disclosure. As shown, Figure 1System 100 is shown and may include vehicle 102; operating computing system 104; one or more remote computing devices 106; communications network 108; vehicle computing system 112; one or more autonomous system sensors 114; autonomous system sensor data 116; positioning system 118; autonomous computing system 120; map data 122; perception system 124; prediction system 126; motion planning system 128; state data 130; prediction data 132; motion planning data 134; communications system 136; (one or more) vehicle control systems 138; human-machine interface 140 and vehicle controller 142.

[0078] The operating computing system 104 may be associated with a service provider (e.g., a service entity) that may provide one or more vehicle services to a plurality of users via a fleet (e.g., service entity vehicles, third-party vehicles, etc.) including, for example, the vehicle 102. The vehicle services may include transportation services (e.g., ride-sharing services), courier services, delivery services, and / or other types of services.

[0079] The operating computing system 104 may include multiple components for performing various operations and functions. For example, the operating computing system 104 may include one or more computing devices remote from the vehicle 102 and / or otherwise be associated with one or more computing devices remote from the vehicle 102. The one or more computing devices of the operating computing system 104 may include one or more processors and one or more memory devices. The one or more memory devices of the operating computing system 104 may store instructions that, when executed by one or more processors, cause the one or more processors to perform operations and functions associated with the operation of one or more vehicles (e.g., a fleet), the provision of vehicle services, and / or other operations as discussed herein.

[0080] For example, the operating computing system 104 may be configured to monitor and communicate with the vehicle 102 and / or its user to coordinate vehicle services provided by the vehicle 102. To this end, the operating computing system 104 may manage a database that stores data including vehicle status data associated with the status of vehicles including the vehicle 102. The vehicle status data may include the status of the vehicle, the location of the vehicle (e.g., the latitude and longitude of the vehicle), the availability of the vehicle (e.g., whether the vehicle is available to pick up passengers and / or cargo, etc.), and / or the status of objects inside and / or outside the vehicle (e.g., the physical size and / or appearance of objects inside / outside the vehicle).

[0081] The operating computing system 104 may communicate with one or more remote computing devices 106 and / or the vehicle 102 via one or more communication networks including a communication network 108. The communication network 108 may exchange (send or receive) signals (e.g., electronic signals) or data (e.g., data from a computing device) and include any combination of various wired (e.g., twisted pair cables) and / or wireless communication mechanisms (e.g., cellular, wireless, satellite, microwave, and radio frequency) and / or any desired network topology (or multiple topologies). For example, the communication network 108 may include a local area network (e.g., an intranet), a wide area network (e.g., the Internet), a wireless LAN network (e.g., via Wi-Fi), a cellular network, a SATCOM network, a VHF network, a HF network, a WiMAX-based network, and / or any other suitable communication network (or combination thereof) for transmitting data to and / or from the vehicle 102.

[0082] Each of the one or more remote computing devices 106 may include one or more processors and one or more memory devices. The one or more memory devices may be used to store instructions that, when executed by the one or more processors of the one or more remote computing devices 106, cause the one or more processors to perform operations and / or functions including operations and / or functions associated with the vehicle 102, including exchanging (e.g., sending and / or receiving) data or signals with the vehicle 102, monitoring the status of the vehicle 102, and / or controlling the vehicle 102. The one or more remote computing devices 106 may communicate (e.g., exchange data and / or signals) with one or more devices including the operating computing system 104 and the vehicle 102 via the communication network 108.

[0083] The one or more remote computing devices 106 may include one or more computing devices (e.g., desktop computing devices, laptop computing devices, smart phones, and / or tablet computing devices) that can receive input or instructions from a user or exchange signals or data with an item or other computing device or computing system (e.g., operating computing system 104). In addition, the one or more remote computing devices 106 can be used to determine and / or modify one or more states of the vehicle 102, including the location (e.g., latitude and longitude), speed, acceleration, trajectory, and / or path of the vehicle 102, based at least in part on the signals or data exchanged with the vehicle 102. In some embodiments, the operating computing system 104 may include one or more remote computing devices 106.

[0084] The vehicle 102 may be a land-based vehicle (e.g., a car, a bicycle, a scooter, other light electric vehicles, etc.), an aircraft, and / or other types of vehicles. The vehicle 102 may be an autonomous vehicle that may perform various actions including driving, navigation, and / or operation with minimal and / or no interaction from a human driver. The autonomous vehicle 102 may be configured to operate in one or more modes, including, for example, a fully autonomous operating mode, a semi-autonomous operating mode, a parking mode, and / or a sleep mode. A fully autonomous (e.g., automatic driving) operating mode may be a mode in which the vehicle 102 may provide driving and navigation operations with minimal and / or no interaction from a human driver present in the vehicle. A semi-autonomous operating mode may be a mode in which the vehicle 102 may operate with some interaction from a human driver present in the vehicle. Parking and / or sleep modes may be used between operating modes when the vehicle 102 performs various actions including waiting to provide subsequent vehicle services and / or recharging between operating modes.

[0085] Indications, records, and / or other data indicating a state of the vehicle, a state of one or more occupants of the vehicle, and / or a state of the environment including one or more objects (e.g., physical size and / or appearance of one or more objects) may be stored locally in one or more memory devices of the vehicle 102. Additionally, the vehicle 102 may provide data indicating a state of the vehicle, a state of one or more occupants of the vehicle, and / or a state of the environment to the operating computing system 104, which may store the indications, records, and / or other data indicating a state of one or more objects within a predetermined distance of the vehicle 102 in one or more memory devices associated with the operating computing system 104 (e.g., remote from the vehicle). Additionally, the vehicle 102 may provide data indicating a state of one or more objects within a predetermined distance of the vehicle 102 to the operating computing system 104, which may store the indications, records, and / or other data indicating a state of one or more objects within a predetermined distance of the vehicle 102 in one or more memory devices associated with the operating computing system 104 (e.g., remote from the vehicle).

[0086] The vehicle 102 may include and / or be associated with a vehicle computing system 112. The vehicle computing system 112 may include one or more computing devices onboard the vehicle 102. For example, the one or more computing devices of the vehicle computing system 112 may be located on and / or within the vehicle 102. The one or more computing devices of the vehicle computing system 112 may include various components for performing various operations and functions. For example, the one or more computing devices of the vehicle computing system 112 may include one or more processors and one or more tangible, non-transitory computer-readable media (e.g., memory devices). The one or more tangible, non-transitory computer-readable media may store instructions that, when executed by one or more processors, cause the vehicle 102 (e.g., its computing system, one or more processors, and other devices in the vehicle 102) to perform operations and functions, including those described herein.

[0087] like Figure 1 As shown, the vehicle computing system 112 may include one or more autonomous system sensors 114; a positioning system 118; an autonomous computing system 120; a communication system 136; (one or more) vehicle control systems 138; and a human-machine interface 140; and / or a vehicle controller 142. One or more of these systems may be configured to communicate with each other via a communication channel. The communication channel may include one or more data buses (e.g., a controller area network (CAN)), an on-board diagnostic connector (e.g., OBD-II), and / or a combination of wired and / or wireless communication links. The on-board systems may exchange (e.g., send and / or receive) data, messages, and / or signals between each other via the communication channel.

[0088] One or more autonomous system sensors 114 may be configured to generate and / or store data including autonomous system sensor data 116 associated with one or more objects in proximity to the vehicle 102 (e.g., within the range or field of view of one or more of the one or more sensors 114). The one or more autonomous system sensors 114 may include a light detection and ranging (LIDAR) system, a radio detection and ranging (RADAR) system, one or more cameras (e.g., a visible spectrum camera and / or an infrared camera), a motion sensor, and / or other types of imaging capture devices and / or sensors. The autonomous system sensor data 116 may include image data, radar data, lidar data, and / or other data acquired by the one or more autonomous system sensors 114. The one or more objects may include, for example, pedestrians, vehicles, bicycles, and / or other objects. The one or more sensors may be located at various locations of the vehicle 102, including the front, rear, left, right, top, or bottom of the vehicle 102. The autonomous system sensor data 116 may indicate a location associated with one or more objects within the surrounding environment of the vehicle 102 at one or more times. For example, autonomous system sensor data 116 may indicate one or more lidar point clouds associated with one or more objects within the surrounding environment. One or more autonomous system sensors 114 may provide autonomous system sensor data 116 to autonomous computing system 120 .

[0089] In addition to autonomous system sensor data 116, autonomous computing system 120 may retrieve or otherwise obtain data including map data 122. Map data 122 may provide detailed information about the surroundings of vehicle 102. For example, map data 122 may provide information about: the identification and location of different roads, road segments, buildings, or other items or objects (e.g., lamp posts, crosswalks, and / or curbs); the location and orientation of traffic lanes (e.g., the location and orientation of parking lanes, turn lanes, bicycle lanes, or other lanes within a particular road or other travel route and / or one or more boundary markers associated therewith); traffic control data (e.g., the location and instructions of signs, traffic lights, or other traffic control devices); and / or any other map data that provides information that helps vehicle computing system 112 process, analyze, and perceive its surroundings and its relationship to the surroundings.

[0090] The vehicle computing system 112 may include a positioning system 118. The positioning system 118 may determine the current location of the vehicle 102. The positioning system 118 may be any device or circuit for analyzing the location of the vehicle 102. For example, the positioning system 118 may determine the location by using one or more of an inertial sensor, a satellite positioning system, based on an IP / MAC address, by using triangulation and / or proximity to a network access point or other network component (e.g., a cellular tower and / or a Wi-Fi access point), and / or other suitable techniques. The location of the vehicle 102 may be used by various systems of the vehicle computing system 112 and / or provided to one or more remote computing devices (e.g., the operating computing system 104 and / or the remote computing device 106). For example, the map data 122 may provide the vehicle 102 with a relative location of the surrounding environment of the vehicle 102. The vehicle 102 may identify its location within the surrounding environment (e.g., across six axes) based at least in part on the data described herein. For example, vehicle 102 may process autonomous system sensor data 116 (e.g., LIDAR data, camera data) to match it to a map of the surrounding environment to gain an understanding of the vehicle's position within that environment (e.g., transpose the vehicle's position within its surrounding environment).

[0091] The autonomous computing system 120 may include a perception system 124, a prediction system 126, a motion planning system 128, and / or other systems that cooperate to perceive the surrounding environment of the vehicle 102 and accordingly determine a motion plan for controlling the movement of the vehicle 102. For example, the autonomous computing system 120 may receive autonomous system sensor data 116 from one or more autonomous system sensors 114, attempt to determine the state of the surrounding environment by performing various processing techniques on the autonomous system sensor data 116 (and / or other data), and generate an appropriate motion plan through the surrounding environment. The autonomous computing system 120 may control one or more vehicle control systems 138 to operate the vehicle 102 according to the motion plan. One or more of these systems / computing functions may be included within the same system and / or share one or more computing resources.

[0092] The perception system 124 can identify one or more objects proximate to the vehicle 102 based on the autonomous system sensor data 116 received from the autonomous system sensor 114. In particular, in some embodiments, the perception system 124 can determine state data 130 for each object that describes the current state of such object. As an example, the state data 130 for each object can describe the following estimates of the object: current position (also referred to as position); current speed; current heading (which can also be collectively referred to as speed); current acceleration; current orientation; size / footprint (e.g., represented by a bounding shape such as a bounding polygon or polyhedron); characterized class (e.g., vehicle class vs. pedestrian class vs. bicycle class vs. other class); yaw rate; and / or other state information. In some embodiments, the perception system 124 can determine the state data 130 for each object in multiple iterations. In particular, the perception system 124 can update the state data 130 for each object at each iteration. Thus, the perception system 124 can detect and track objects (e.g., vehicles, bicycles, pedestrians, etc.) near the vehicle 102 over time, thereby generating a representation of the world around the vehicle 102 and its state (e.g., a representation of objects of interest within the scene at the current time and the states of the objects).

[0093] The prediction system 126 may receive state data 130 from the perception system 124 and predict one or more future positions and / or movement paths of each object based on such state data. For example, the prediction system 126 may generate prediction data 132 associated with each of the corresponding one or more objects in the vicinity of the vehicle 102. The prediction data 132 may indicate one or more predicted future positions of each corresponding object. The prediction data 132 may indicate a predicted path (e.g., a predicted trajectory) of at least one object within the surrounding environment of the vehicle 102. For example, the predicted path (e.g., trajectory) may indicate a path along which the corresponding object is predicted to travel over time (and / or a speed at which the object is predicted to travel along the predicted path). The prediction system 126 may provide the prediction data 132 associated with the one or more objects to the motion planning system 128.

[0094] The motion planning system 128 may determine a motion plan based at least in part on the prediction data 132 (and / or other data) and generate motion planning data 134 for the vehicle 102. The motion planning data 134 may include vehicle actions and predicted movements regarding objects near the vehicle 102. For example, the motion planning system 128 may implement an optimization algorithm that considers cost data associated with the vehicle actions and other objective functions (e.g., cost functions based on speed limits, traffic lights, and / or other aspects of the environment), if any, to determine optimization variables that make up the motion planning data 134. As an example, the motion planning system 128 may determine that the vehicle 102 can perform a certain action (e.g., pass an object) without increasing the potential risk to the vehicle 102 and / or violating any traffic regulations (e.g., speed limits, lane boundaries, signage). The motion planning data 134 may include the planned trajectory, speed, acceleration, and / or other actions of the vehicle 102.

[0095] As an example, in some implementations, motion planning system 128 may determine a cost function for each of one or more candidate motion plans for autonomous vehicle 102 based at least in part on the current position and / or predicted future position and / or movement path of an object. For example, a cost function may describe the cost (e.g., over time) of adhering to a particular candidate motion plan. For example, the cost described by the cost function may increase as autonomous vehicle 102 approaches a collision with another object and / or deviates from a preferred path (e.g., a predetermined route of travel).

[0096] Thus, given information about the current position and / or predicted future position and / or movement path of an object, the motion planning system 128 can determine the cost of adhering to a particular candidate path. The motion planning system 128 can select or determine a motion plan for the autonomous vehicle 102 based at least in part on the cost function(s). For example, a motion plan that minimizes the cost function can be selected or otherwise determined. The motion planning system 128 can then provide the selected motion plan to the vehicle controller 142, which controls one or more vehicle controls (e.g., actuators or other devices that control airflow, steering, braking, etc.) to execute the selected motion plan.

[0097] The motion planning system 128 may provide the motion planning data 134 having data indicative of vehicle maneuvers, planned trajectories, and / or other operating parameters to a vehicle control system 138 to implement the motion planning data 134 for the vehicle 102 .

[0098] The autonomous vehicle 102 may implement a planned trajectory for the autonomous vehicle 102 to navigate a particular segment of the route. For example, the trajectory (or a representation thereof) may be provided to the vehicle controller 142. The vehicle controller 142 may include a vehicle interface module (VIM) that may obtain data from one or more vehicle systems such as the autonomous computing system 128 and / or other systems. The vehicle controller 142 may also communicate with a service provider (e.g., its computing system) associated with the autonomous vehicle. The service provider may serve as a communication interface to help confirm the state / status / condition / capability of the vehicle. The vehicle controller 142 may utilize the data it obtains and interface with the vehicle control system(s) 138 to help implement the trajectory that the vehicle follows. For example, the vehicle controller 142 may convert the motion planning / trajectory data into instructions for controlling the autonomous vehicle 102 (including adjusting the vehicle steering "X" degrees, applying a certain amount of braking force, etc.). The vehicle control system(s) 138 generate specific control signals for the autonomous vehicle 102 (e.g., adjusting steering, braking, speed, etc.). The specific control signals may cause the autonomous vehicle 102 to move according to the selected trajectory. The vehicle controller 142 may have other functions, such as safely transitioning modes and / or states of the vehicle, as described herein.

[0099] The vehicle controller may serve as an interface between the autonomous computing system 120 and the autonomous vehicle 102 itself. In some examples, the vehicle controller 142 may include multiple control lanes. In some examples, the vehicle controller 142 may have a dual lane control architecture. The vehicle controller 142 may include a first control lane and a second control lane. Each lane control may constitute a full system that may be used to receive signals from the autonomous computing 120 system, track the current state of the autonomous vehicle 102, and send control signals from the autonomous computing system 120 to the vehicle control system(s) 138.

[0100] The vehicle controller 142 may include Figure 1 Various different systems not shown in the figure include vehicle health monitoring system, vehicle operator and passenger interface, vehicle dynamic monitor, input side and output side low-level control and estimation systems, posture management system, trajectory management system, motion control system, fault management system, unit help monitoring system, vehicle mode management system, application power management system and vehicle controller power management system.

[0101] The vehicle health monitoring system may collect information about vehicle degradation, including any damage or wear to the autonomous vehicle 102 that has been measured or detected by an onboard detection or analysis system. The vehicle operator interface may monitor the safety operator interface and the occupant interface to ensure that the operator and occupants are safe and able to communicate when needed. The vehicle dynamics monitoring system may measure various vehicle responses to inputs or controls to determine whether the autonomous vehicle 102 is properly monitoring and responding to inputs provided by various input sources.

[0102] The fault management system can receive information from the vehicle health monitoring system, the vehicle operator and passenger interface, the unit health monitoring system, and the vehicle dynamics monitor to determine when a fault occurs in autonomous vehicle 102. The fault management system can record any events that occur for later diagnosis. In some examples, the fault management system can receive information from a unit health monitor, which can monitor autonomous vehicle 102 for any equipment faults.

[0103] The low-level control estimation system can perform various tasks, including estimating the current friction level experienced by the autonomous vehicle 102, estimating the capabilities of the autonomous vehicle 102, performing collision mitigation arbitration, switching direct driving, serving as input and output to the vehicle controller 142, etc. The posture management system can receive information from the low-level control and estimation system to consider the plausibility of various postures and estimate their probabilities. The posture management system can also estimate the current state of the autonomous vehicle 102. The motion control system can have longitudinal and lateral control of the autonomous vehicle 102 and control the direction of motion processing.

[0104] The vehicle mode management system may be used to implement the improved mode transition techniques of the present disclosure, as will be further described herein. For example, each controlled lane of the vehicle controller 142 may include a vehicle mode management system (or mode manager). The vehicle mode management system may receive various inputs from various sources (such as, for example, a fault management system, (one or more) human-machine interfaces 140 (e.g., physical key interfaces, user devices, etc. associated therewith), passenger system interfaces, etc.) to determine the current mode of the vehicle controller. The vehicle controller 142 may communicate with a service provider to ensure that the vehicle controller mode matches the current state of the service provider's autonomous vehicle 102. The application power management system may coordinate with the vehicle controller power management system to power on and off the vehicle controller 142, the autonomous vehicle 102, and the autonomous computing system 120. The vehicle controller 142 (e.g., a vehicle mode management system) may monitor, manage, and control the mode of the autonomous vehicle 102.

[0105] Autonomous vehicle 102 may include a vehicle controller 142 configured to convert motion planning data 134 into instructions. As an example, vehicle controller 142 may convert the determined motion planning data 134 into instructions for controlling vehicle 102 (including adjusting the steering of vehicle 102 by “X” degrees and / or applying a certain amount of braking force). Vehicle controller 142 may send one or more control signals to the responsible vehicle control components (e.g., a braking control system, a steering control system, and / or an acceleration control system) to execute the instructions and implement motion planning data 134.

[0106] The vehicle computing system 112 may include a communication system 136 configured to allow the vehicle computing system 112 (and its one or more computing devices) to communicate with other computing devices. The vehicle computing system 112 may use the communication system 136 to communicate with the operating computing system 104 and / or one or more other remote computing devices (e.g., one or more remote computing devices 106) over one or more networks (e.g., via one or more wireless signal connections, etc.). In some embodiments, the communication system 136 may allow communication between one or more systems onboard the vehicle 102. The communication system 136 may also be configured to enable the autonomous vehicle to communicate with the remote computing device 106 and / or provide and / or receive data and / or signals from the remote computing device 106 - which are associated with a user and / or an item (e.g., an item to be picked up for a courier service). The communication system 136 may utilize a variety of communication technologies, including, for example, radio frequency signaling and / or Bluetooth low energy protocols. The communication system 136 may include any suitable components for interfacing with one or more networks, including, for example, one or more of the following: transmitters, receivers, ports, controllers, antennas, and / or other suitable components that can help facilitate communications. In some embodiments, the communication system 136 may include multiple components (e.g., antennas, transmitters, and / or receivers) that allow it to implement and utilize multiple-input multiple-output (MIMO) technology and communication technologies.

[0107] The vehicle computing system 112 may include one or more human-machine interfaces 140. For example, the vehicle computing system 112 may include one or more display devices located on the vehicle computing system 112. The display device (e.g., a screen of a tablet, laptop, and / or smart phone) may be viewed by a user of the vehicle 102 located in the front of the vehicle 102 (e.g., driver's seat, front passenger seat). Additionally or alternatively, the display device may be viewed by a user of the vehicle 102 located in the rear of the vehicle 102 (e.g., a passenger seat at the rear of the vehicle).

[0108] Figure 2 1 depicts a block diagram of a vehicle controller 142 according to an exemplary embodiment of the present disclosure. Figure 2 As shown, autonomous computer system 120 may be connected to vehicle controller 142. Vehicle controller 142 may include a first control lane 205A and a second control lane 205B. Each control lane may include a complete system for monitoring or controlling the current state of the autonomous vehicle as well as other responsibilities.

[0109] According to one aspect of the present disclosure, the vehicle controller 142 may control the autonomous driving system to simultaneously provide data representing the motion plan and data representing the autonomous vehicle (e.g., Figure 1 Therefore, the first control lane 205A and the second control lane 205B can both represent the current state of the autonomous vehicle.

[0110] According to one aspect of the present disclosure, the vehicle controller 142 may designate one of the first control lane 205A or the second control lane 205B as a primary lane and the other as a secondary lane (e.g., based on the capabilities of the first control lane 205A and the capabilities of the second control lane 205B). The vehicle controller 142 may use the primary lane to control the vehicle 102 so that the first control lane 205A and the second control lane 205B do not attempt to control the same operation of the autonomous vehicle 102 at the same time.

[0111] In some examples, first control lane 205A may determine that a request to switch states has been generated. First control lane 205A may ensure that second control lane 205B also transitions to the new state as part of the process of switching states. In this way, if first control lane 205A fails, or control needs to be transferred to second control lane 205B for any reason, second control lane 205B has an accurate internal representation of the state of autonomous vehicle 102.

[0112] Figure 3 A block diagram of a lane controller 304 according to an exemplary embodiment of the present disclosure is depicted. In this example, the lane controller 304 may be included in a vehicle controller (e.g., Figure 1 Lane controller 304 may receive input data 302, process input data 302, and output results 308. In some examples, lane controller 304 may include mode management system 306 to update the autonomous vehicle (e.g., Figure 1 The current mode of the autonomous vehicle 102).

[0113] The mode management system 306 may be used to implement the improved mode transition techniques of the present disclosure, as will be further described herein. For example, a vehicle controller (e.g., Figure 1 and / or Figure 2 Each controlled lane (e.g., Figure 2 The first controlled lane 205A and / or the second controlled lane 205B depicted in FIG. 1 may include a mode management system 306 (or mode manager). The mode management system 306 may receive various inputs from various sources such as, for example, a vehicle operator (e.g., a physical key interface associated therewith, a user device, etc.), a passenger system interface, etc. The inputs may be used to determine the vehicle controller (e.g., Figure 1 and / or Figure 2 The current mode of the vehicle controller 142 in the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the embodiment of the present invention may communicate with the service provider (e.g., a computing system associated therewith) to ensure that the vehicle controller mode is consistent with the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 itself may match the pattern of the autonomous vehicle 102.

[0114] Included in vehicle controllers (e.g. Figure 1 and / or Figure 2 The mode management system 306 in the vehicle controller 142 in the vehicle controller 142 can monitor, manage and control the mode of the autonomous vehicle. The state machine can be implemented by the mode management system 306. The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the embodiment may also ensure that the primary control lane (e.g. Figure 2 The first control lane 205A) and the secondary control lane (e.g., Figure 2 The second controlled lane 205B in the embodiment is synchronized (e.g., has the same state) and both are synchronized with the autonomous vehicle (e.g., Figure 1 Thus, whenever mode management system 306 changes from one mode to another or from one state to another, mode management system 306 can synchronize with autonomous vehicle (e.g., Figure 1 An autonomous computing system (e.g., Figure 1 The autonomous computing system 120) confirms the mode change. If the autonomous computing system (e.g., Figure 1 If the autonomous computing system 120) does not confirm or authorize the change in mode or state, the mode management system 306 can remain in the previous state.

[0115] Figure 4 A state machine 400 for mode transition management according to an exemplary embodiment of the present disclosure is depicted. The state machine 400 may represent an autonomous vehicle (e.g., Figure 1 102) (or its associated mode management system (e.g., Figure 3The mode management system 306 in the system manages the possible states and the transitions between different modes.

[0116] When the vehicle controller (e.g. Figure 1 and / or Figure 2 142 in the vehicle controller) (together with the autonomous vehicle (e.g., Figure 1 When the rest of the autonomous vehicle 102) is started (or powered on), the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the vehicle controller 142 may initialize the vehicle to an initial unauthorized state 402. The unauthorized state 402 may be outside of the first operating mode or the second operating mode. The first operating mode may be when the autonomous vehicle (e.g., Figure 1 The second operating mode may be a mode in which an operator is not present in the autonomous vehicle 102 (e.g., Figure 1 In contrast, unauthorized state 402 can operate as a neutral state, wherein an autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 1 does not take any action and waits for the authorization request.

[0117] Vehicle controller (e.g. Figure 1 and / or Figure 2 The authorization request may come from one of a number of different sources. For example, the authorization request may be received by an autonomous vehicle (e.g., Figure 1 systems within the autonomous vehicle 102 itself (e.g., Figure 1 ), a remote computing system associated with the autonomous vehicle (e.g., a service system that manages a ride-sharing service), or a physical key interface (e.g., via a user physically inserting a key or key-like device into the autonomous vehicle (e.g., Figure 1 In the autonomous vehicle 102) in order to initiate the autonomous vehicle into a drivable mode) is generated.

[0118] In response to receiving the authorization request, a vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the embodiment may determine whether the request includes a request to enter a first operating mode or a request to enter a second operating mode. The first operating mode and the second operating mode are different modes such that when the autonomous vehicle (e.g., Figure 1 When the autonomous vehicle 102 in FIG. 4 is in any mode, the vehicle controller cannot change to a state associated with another mode without first returning to the unauthorized state 402.

[0119] The first operating mode may be an autonomous vehicle (e.g., Figure 1When in this first operating mode, the operator's interaction with the autonomous vehicle (e.g., Figure 1 When a vehicle controller (e.g., Figure 1 and / or Figure 2 When the vehicle controller 142 in the example embodiment determines that the authorization request includes a request to enter the first operating mode, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may enable the autonomous vehicle (e.g., Figure 1 The mode of the autonomous vehicle 102 in FIG. 10A transitions to the first operating mode (at a) and enters a standby state 404 (labeled NVO standby). As described above, before entering the first operating mode, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may provide a signal to the autonomous vehicle (e.g., Figure 1 102 in the autonomous vehicle 102) sends a request to determine the autonomous vehicle (e.g., Figure 1 When the vehicle controller receives a signal from the autonomous vehicle (e.g., Figure 1 When the autonomous vehicle 102 in FIG. 1 is ready to enter the first operating mode, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may change the autonomous vehicle (e.g., Figure 1 mode of the autonomous vehicle 102) and enters a standby state 404.

[0120] Standby state 404 may be an autonomous vehicle (e.g., Figure 1 404 , the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 104 may be immobile and unable to respond to user input. Once the autonomous vehicle (e.g., Figure 1 ) has signaled that it is ready to transition to a fully computer controlled state 406 (at c), the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example can then wait for a signal from the autonomous system (e.g., Figure 1 ) or from a remote computing system (e.g., Figure 1 In some examples, a state transition request may be made from a remote computing system 106 associated with an autonomous vehicle (e.g., Figure 1In some examples, the state transition request may be received from an autonomous system associated with an autonomous vehicle 102 in the example. Figure 1 The state transition request is received by an autonomous vehicle 102 in the system or a remote server system associated with the autonomous vehicle.

[0121] Alternatively, once an autonomous vehicle (e.g. Figure 1 The autonomous vehicle 102 in FIG. 1 has signaled that it is ready to transition to computer control, and the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the FIG. 100 can automatically transition from the standby state 404 (at c) to the full computer control state 406 without requiring an additional state transition request.

[0122] Once the vehicle controller (e.g. Figure 1 and / or Figure 2 ) has transitioned to computer controlled state 406, the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in the example can be connected to the autonomous vehicle (e.g., Figure 1 Autonomous systems (e.g., Figure 1 The autonomous computing system 120 in the vehicle is fully controlled by the autonomous vehicle (e.g., Figure 1 The actions taken by the autonomous vehicle 102 in FIG. 104 may be in response to commands generated by the autonomous system. In some embodiments, a remote system (e.g., a remote assistance system) may take precedence over an autonomous system to remotely assist an autonomous vehicle (e.g., Figure 1 When in computer controlled state 406, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the embodiment may receive a state transition request. The state transition request may request the autonomous vehicle (e.g., Figure 1 d) leaves computer controlled state 406 and transitions back to standby state 404. Note that the state machine is configured such that the only state that can be reached from computer controlled state 406 is standby state 404.

[0123] In some examples, an autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 404 may ensure that movement has stopped before entering the standby state 404. Thus, when the vehicle controller (e.g., Figure 1 and / or Figure 2 When the vehicle controller 142 in FIG. 1 receives the state transition request, the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 1 may be provided by an autonomous system (e.g., Figure 1) or by a vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the vehicle itself instructs to take action that will stop it. As with all other state transitions, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may synchronize the intent to change state with the service provider and between the controlled lanes. Once the intended transition is synchronized between all relevant components, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example can then (at d) direct the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 1 transitions from the computer controlled state 406 to the standby state 404 .

[0124] Once in the standby state 404, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the vehicle controller 142 may receive (at b) a transition request to the unauthorized state 402. If a request to transition to the second operating mode is received while in the standby state 404, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may transition to the unauthorized state 402 (at b) before transitioning to the second operating mode. In this manner, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may ensure that the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 1 does not directly transition between the first operating mode and the second operating mode.

[0125] In some examples, an autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 4 may be powered down and restarted so that it is reinitialized to the unauthorized state 402 before transitioning to the second operating mode. Alternatively, once in the standby state 404, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the system can be switched to the unauthorized state 402 without powering off, and when in the unauthorized state, receive authorization to switch to the second operating mode.

[0126] When the vehicle controller receives the authorization request and determines that the request is to enter the second operating mode, the vehicle controller (e.g., Figure 1 and / or Figure 2The vehicle controller 142 in FIG. 140 may transition (at e) to a user control state, such as, for example, the manual control state 408. This transition may be performed by the vehicle controller (e.g., Figure 1 and / or Figure 2 142) and an autonomous vehicle (e.g., Figure 1 When in manual control state 408, the operator can operate the on-board controls to operate the autonomous vehicle (e.g., Figure 1 102 in the autonomous vehicle 102).

[0127] In some examples, when in manual control state 408, a vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example of FIG. 140 may receive a request to enter full computer control 410. In response to the request, the vehicle controller (e.g., Figure 1 and / or Figure 2 Once the vehicle controller 142 in the second operating mode has been synchronized with one or more other components, it is possible to transition (at g) from the manual control state 408 to the full computer control state 410. The computer control state 410 associated with the second operating mode is different from the full computer control state 406 associated with the first operating mode. For example, in the first operating mode, the computer controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may ignore any human intervention and may not transition to the operator control state. In contrast, the computer control state 410 associated with the second operating mode may transition to the operator control state 408 upon request and may be responsive to user input. For example, if a user presses the brake pedal, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in may respond by exiting the computer control state 410 (at h) and re-entering the operator control state 408.

[0128] When in the computer controlled state 410 associated with the second operating mode, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the embodiment of the present invention may receive a request to change from the computer control state 410 to the manual control state 408. As described above, the request may be made by a user communicating with the autonomous vehicle (e.g., Figure 1 interaction with the autonomous vehicle 102), through the autonomous system (e.g., Figure 1 ) or by instructing an autonomous vehicle (e.g., Figure 1The autonomous vehicle 102 in the example returns to the remote computing system of the operator control state 408 to generate.

[0129] When in manual control, the vehicle controller (e.g. Figure 1 and / or Figure 2 The vehicle controller 142 in the example may receive a request to transition to the unauthorized state 402. The request may be part of a request to transition to the first operating mode, or may be included in a request to cease operation and power off. Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may establish that both the service provider and the controlling lane are aware of the change request and confirm that the change is authorized. The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in may transition (at f) to the unauthorized state 402.

[0130] Figure 5 A detailed state machine 500 for mode transition management according to an exemplary embodiment of the present disclosure is depicted. For example, each state of the state machine 500 (eg, Figure 4 The states 402, 404, 406, 408, 410 described in the foregoing may have one or more sub-states that facilitate smooth transitions between states. The following provides possible exemplary sub-states and a description of how the state machine transitions between states and each sub-state. As an example, the unauthorized state 402 may include multiple possible sub-states (e.g., 510, 512, 514, 516, 518, etc.). The first sub-state may be the nominal sub-state 510, in which the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in has not yet received the authorization request.

[0131] Vehicle controller (e.g. Figure 1 and / or Figure 2 and Figure 2 The vehicle controller 142 in the example may transition to a subsequent substate in response to receiving an authorization request. For example, if the request is to enter a first operating mode, the vehicle controller may enter a first mode request substate 512, or if the request is to enter a second operating mode, the vehicle controller may enter a second mode request substate 516.

[0132] In both the first mode request substate 512 and the second mode request substate 516, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may request authorization from a third party (eg, a service provider) to ensure that the autonomous vehicle (eg, Figure 1102 in the autonomous vehicle 102) is ready to transition to another state. If the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 1 receives a receipt or authorization to change to the requested mode, then the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the vehicle controller 142 may transition the state from the first mode request substate 512 to the no operator standby state 520 upon requesting to enter the first operating mode, or may transition the state from the second mode request substate 516 to the operator manual control state 540 upon requesting to enter the second operating mode.

[0133] However, if no acknowledgement or response is received within a predetermined time frame, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the first mode request substate 512 may not transition to the desired operating mode, but may transition to a state indicating a potential failure. For example, if the vehicle controller (e.g., Figure 1 and / or Figure 2 If the vehicle controller 142 in FIG. 1 exceeds the allotted time for receiving the receipt, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the first mode request failure substate 514 may transition to the first mode request failure substate 514. For example, the first mode request failure substate 514 may indicate a potential transition failure. The second mode request failure substate 518 may perform the same or similar actions for a failure to transition to the second operating mode. When in the first mode request failure substate 514 or the second mode request failure substate 518, if the authorization request succeeds (e.g., the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the system receives the autonomous vehicle (e.g. Figure 1 102) in the autonomous vehicle 102) to change the state of the intention), the autonomous vehicle (e.g. Figure 1 The autonomous vehicle 102 in FIG. 100 may then transition to the first operating mode or the second operating mode (depending on which mode is requested). However, if no acknowledgement is received within a predetermined time frame, the vehicle controller (e.g., Figure 1 and / or the vehicle controller 142 in 2) can transition back to the nominal sub-state 510 and wait for a new authorization request.

[0134] Similarly, when the No Operator Standby state 520 is first entered, the initial substate may be the No Operator Standby nominal substate 522, wherein the vehicle controller (e.g., Figure 1 and / or Figure 2The vehicle controller 142 in FIG. 1 is waiting to receive a state transition request to enter the no operator computer control state 530. When the vehicle controller (e.g., Figure 1 and / or Figure 2 When the vehicle controller 142 in FIG. 1 receives a request to transition to the non-operator computer control state 530, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the state can locally or remotely ensure that the intention of the state switch matches the intention of one or more other controlled lanes. For example, before transitioning to another sub-state within the no operator standby state 520, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example embodiment may request information from one or more other systems to ensure that all mode controllers (e.g., mode managers of other controlled lanes) have matching intent to transition to the no operator computer control state 530.

[0135] Vehicle controller (e.g. Figure 1 and / or Figure 2 142 in the vehicle controller 144 may transition to the autonomous request substate 524, wherein the vehicle controller (e.g., Figure 1 and / or Figure 2 142 in the vehicle controller 142) from the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 104 itself requests to enter the no operator computer control state 530. For example, once the vehicle controller (e.g., Figure 1 and / or Figure 2 This can occur when the vehicle controller 142 in the control lane has confirmed the intent to match the control lanes. When in the autonomous request substate 524, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may wait for a signal from the autonomous vehicle (e.g., Figure 1 If the confirmation is not received within a predetermined time, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the autonomous request failure substate 526 may be entered. If a delayed acknowledgement arrives while in this failure state, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may transition to the no operator computer control state 530. However, if no confirmation is received, the vehicle controller (e.g., Figure 1 and / or Figure 2The vehicle controller 142 in the state may transition back to the no operator standby nominal sub-state 522 to await further state transition requests.

[0136] When in the no operator computer control state 530, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may initially transition to the no operator computer control nominal substate 532 associated with the no operator computer control state 530. The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may remain in this state until a transition request is received to transition back to the no operator standby state 520. As described above, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may enter the exit autonomous request substate 534 to request a request from the autonomous vehicle (e.g., Figure 1 520. If the confirmation is received within the predetermined time, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may transition to the no operator standby state 520. However, if no confirmation is received within a predetermined time, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may enter the Exit Autonomous Request Failed substate 536 while waiting for the delayed confirmation. If no delayed confirmation is forthcoming, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may transition back to the no operator computer control nominal sub-state 532 associated with the no operator computer control state 530 and await further requests.

[0137] If the request is a request to enter the second operating mode, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may transition to the operator manual control state 540. When in the operator manual control state 540, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may first enter the operator manual control state nominal substate 542, wherein the vehicle controller (e.g., Figure 1 and / or the vehicle controller 142 in 2) waits to receive a request to transition to the operator presence computer control state 550 associated with the second operating mode. The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may remain in this state until a transition request to the operator presence computer control state 550 is received. As with other state transitions, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may be from an autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 100 ) requests authorization to transition to the operator presence computer controlled state 550 by transitioning to the operator presence autonomous request substate 544 upon receiving the state transition request. If confirmation is received within a predetermined time, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may transition to the operator presence computer control state 550. However, if no confirmation is received within a predetermined time, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may enter the operator presence autonomous request failed substate 546 while waiting for the delayed confirmation. If no delayed confirmation is forthcoming, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the operator manual control state 540 may transition back to the operator manual control nominal sub-state 542 associated with the operator manual control state 540 and await further requests.

[0138] In the operator presence computer control state 550, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the second mode of operation may enter the nominal operator presence computer controlled substate 552 while waiting to receive the state transition request. When the state transition request is received in the nominal operator presence computer controlled substate 552 associated with the second mode of operation, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may immediately transition to the operator manual control state 540. Once in the operator manual control state 540, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the vehicle controller 142 may enter the autonomous exit request substate 554 to request authorization for the state transition and confirm that state synchronization is maintained. If no authorization is forthcoming, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may enter the autonomous exit request failed substate 556. If authorization is not received, the vehicle controller (e.g., Figure 1 and / or Figure 2The vehicle controller 142 in the control state may transition to the operator manual control state nominal substate 542 to await further requests.

[0139] Figure 6 A flow chart of an exemplary method 600 according to an exemplary embodiment of the present disclosure is depicted. One or more portions of the method 600 may be implemented by one or more computing devices (such as, for example, the computing devices described herein). In addition, one or more portions of the method 600 may be implemented as algorithms on hardware components of (one or more) devices described herein. For purposes of illustration and discussion, Figure 6 Elements are depicted as being performed in a particular order. Using the disclosure provided herein, one of ordinary skill in the art will appreciate that elements of any method discussed herein may be adjusted, rearranged, expanded, omitted, combined, and / or modified in various ways without departing from the scope of the present disclosure. Method 600 may be performed by one or more computing devices (such as Figure 1-3 The invention may be implemented as one or more computing devices depicted in .

[0140] At 602, a vehicle controller (eg, Figure 1 and / or Figure 2 The vehicle controller 142 in the example may initiate the autonomous vehicle into an unauthorized state upon startup. At 604, the vehicle controller (e.g., Figure 1 and / or Figure 2 The authorization request may be received by the vehicle controller 142 in the autonomous vehicle (e.g., Figure 1 An autonomous computing system (e.g., Figure 1 The authorization request may be made from a remote location (e.g., Figure 1 A remote computing system (e.g., Figure 1 In some embodiments, the authorization request may be associated with a physical key interface.

[0141] At 606, a vehicle controller (eg, Figure 1 and / or Figure 2 The vehicle controller 142 in the example embodiment may determine whether the authorization request includes a request for authorization to enter a first operating mode or a request for authorization to enter a second operating mode, wherein the first operating mode is associated with an autonomous vehicle without a human operator and the second operating mode is associated with an autonomous vehicle including a human operator.

[0142] At 608, a vehicle controller (eg, Figure 1 and / or Figure 2The vehicle controller 142 in FIG. 1 may, in response to determining that the authorization request includes a request to enter the first operating mode, direct the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in the first operating mode may be switched to a standby state, or the autonomous vehicle may be switched to a manual control state in response to determining that the authorization request is a request to enter the second operating mode. When in the first operating mode, the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 1 may not respond to operator input.

[0143] When in standby mode, the vehicle controller (e.g. Figure 1 and / or Figure 2 The vehicle controller 142 in the example may receive a request to transition to the first computer control state. In response to receiving the request, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may determine that the autonomous vehicle (e.g., Figure 1 In response to determining that the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in the embodiment of the present invention is ready to enter the first computer control mode, and the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in can be transitioned from a standby state to a first computer controlled state.

[0144] When in the first computer control state, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may receive a request from the autonomous system to exit the first computer controlled state. The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may provide a signal to the autonomous vehicle (e.g., Figure 1 102 in the autonomous vehicle 102) sends a signal that causes the autonomous vehicle (e.g., Figure 1 In response to determining that the autonomous vehicle (e.g., Figure 1 ) has become immobile, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in can be changed from a first computer controlled state to a standby state.

[0145] When in standby mode, the vehicle controller (e.g. Figure 1 and / or Figure 2 The vehicle controller 142 in the vehicle controller 142 may receive a request to transition to the second operating mode. The vehicle controller (e.g., Figure 1 and / or Figure 2Alternatively, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may send commands to cause the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 100 is powered off and restarted into an unauthorized state.

[0146] When in manual control, the vehicle controller (e.g. Figure 1 and / or Figure 2 The vehicle controller 142 in the vehicle control state may receive a request to transition to the second computer control state. The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example may determine that the autonomous vehicle (e.g., Figure 1 In response to determining that the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in the embodiment of the present invention is ready to enter the second computer-controlled state, and the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in can be changed from the manual control state to the second computer control state.

[0147] Vehicle controller (e.g. Figure 1 and / or Figure 2 The vehicle controller 142 in the example may provide a signal to the autonomous vehicle (e.g., Figure 1 The vehicle controller (e.g., autonomous vehicle 102 in FIG. 10A ) sends a state transition request, the state transition request including information describing a planned transition to a second computer-controlled state. Figure 1 and / or Figure 2 The vehicle controller 142 in the example may receive an autonomous vehicle (e.g., Figure 1 In response to receiving the autonomous vehicle (e.g., Figure 1 The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142) in the vehicle controller can update the vehicle controller state to a second computer controlled state.

[0148] Vehicle controller (e.g. Figure 1 and / or Figure 2 The vehicle controller 142 in the example may determine whether a vehicle has been received from an autonomous vehicle (e.g., Figure 1 In response to determining that no confirmation has been received from the autonomous vehicle (e.g., Figure 1 102) receives the confirmation, then the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142) in can enter an error or failure state and remain in the original state.

[0149] When in the second computer control state, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the embodiment may detect an operator's control of the autonomous vehicle (e.g., Figure 1 In response to detecting the operator input, a vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in can be changed from the second computer control state to the manual control state.

[0150] When in manual control, the vehicle controller (e.g. Figure 1 and / or Figure 2 The vehicle controller 142 in the vehicle controller 142 may receive a request to transition to the first operating mode. The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may send commands to cause the autonomous vehicle (e.g., Figure 1 The autonomous vehicle 102 in FIG. 100 is powered off and restarted into an unauthorized state.

[0151] In some examples, a vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 1 may include a first controlled lane and a second controlled lane. The vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in FIG. 140 may facilitate the transition to the standby state by, for example, confirming that both the first control lane and the second control lane indicate a transition to the standby state. In another example, the vehicle controller (e.g., Figure 1 and / or Figure 2 The vehicle controller 142 in the example embodiment may facilitate a transition to the manual control state by, for example, confirming that both the first control lane and the second control lane indicate a transition to the manual control state.

[0152] Figure 7Depicted is an exemplary system with units for performing operations and functions according to exemplary aspects of the present disclosure. Various devices may be configured to perform the methods and processes described herein. For example, a computing system may include (one or more) initiating units 702, (one or more) receiving units 704, (one or more) request analysis units 706, (one or more) conversion units 708 and / or other devices for performing the operations and functions described herein. In some embodiments, one or more of the units may be implemented separately. In some embodiments, one or more units may be part of or included in one or more other units. These devices may include (one or more) processors, (one or more) microprocessors, (one or more) graphics processing units, (one or more) logic circuits, (one or more) dedicated circuits, (one or more) application-specific integrated circuits, programmable array logic, (one or more) field programmable gate arrays, (one or more) controllers, (one or more) microcontrollers and / or other suitable hardware. The device may also or alternatively include, for example, a software control device implemented using a processor or logic circuit. The apparatus may include or otherwise have access to a memory, such as, for example, one or more non-transitory computer-readable storage media, such as random access memory, read-only memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, flash / (one or more) other memory devices, (one or more) data registers, (one or more) databases, and / or other suitable hardware.

[0153] The device may be programmed to execute one or more algorithms for performing the operations and functions described herein. For example, the device may be configured to initiate the autonomous vehicle to an unauthorized state at startup. For example, the vehicle controller may set the initial state of the autonomous vehicle at startup to an unauthorized state that is not associated with either the first operating mode or the second operating mode. Initiating unit 702 is an example of a device for initiating the autonomous vehicle to an unauthorized state at startup.

[0154] The device may be configured to receive the authorization request. For example, the vehicle controller may receive the authorization request from an autonomous system, a remote computing system, or a physical key interface associated with the autonomous vehicle. The receiving unit is an example of a device for receiving the authorization request.

[0155] The device can be configured to determine whether the authorization request includes a request to enter a first operating mode or a second operating mode, wherein the first operating mode is associated with operating the autonomous vehicle without a human operator and the second operating mode is associated with being able to operate the autonomous vehicle by an operator. For example, the vehicle controller can determine the specific operating mode requested based on data included in the authorization request. In some examples, the requested operating mode can be determined based on the source of the authorization request. Request analysis unit 706 is an example of a device for determining whether the authorization request includes a request to enter a first operating mode or a second operating mode, wherein the first operating mode is associated with operating the autonomous vehicle without a human operator and the second operating mode is associated with being able to operate the autonomous vehicle by a human operator.

[0156] The device can be configured to transition the autonomous vehicle from an unauthorized state to a standby state in response to determining that the authorization request includes a request to enter a first operating mode, or to transition the autonomous vehicle from an unauthorized state to a manual control state in response to determining that the authorization request is a request to enter a second operating mode. For example, the vehicle controller can transition to the requested operating mode (first or second operating mode) based on the authorization request. The vehicle controller can also synchronize with the autonomous vehicle or secondary lane controller to ensure that the entire autonomous vehicle system has the same state at the same time. Transition unit 708 is an example of a device for transitioning the autonomous vehicle from an unauthorized state to a standby state in response to determining that the authorization request includes a request to enter a first operating mode, or to transition the autonomous vehicle from an unauthorized state to a manual control state in response to determining that the authorization request is a request to enter a second operating mode.

[0157] Figure 8 Depicted are exemplary system components according to exemplary aspects of the present disclosure. Figure 8 The exemplary system 800 shown is provided by way of example only. Figure 8 The components, systems, connections, and / or other aspects shown in the drawings are optional and are provided as possible but not required examples for implementing the present disclosure. The computing system 800 may be and / or include Figure 1 The computing system 800 may be connected to a central operating system and / or to a vehicle (e.g., Figure 1 The system may be associated with an entity associated with the autonomous vehicle 102 (such as, for example, a vehicle owner, a vehicle manager, a fleet operator, a service provider, etc.).

[0158] The computing device(s) 805 of the computing system 800 may include a processor(s) 815 and at least one memory 820. The processor(s) 815 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and may be one processor or a plurality of processors operatively connected. The memory 820 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, one or more memory devices, flash memory devices, disks, data registers, etc., and combinations thereof.

[0159] The memory 820 may store information that may be accessed by the one or more processors 815. For example, the memory 820 (e.g., one or more non-transitory computer-readable storage media, memory devices) may include computer-readable instructions 825 that may be executed by the one or more processors 815. The instructions 825 may be software written in any suitable programming language, or may be implemented in hardware. Additionally or alternatively, the instructions 825 may be executed in logically and / or virtually separate threads on the processor(s) 815.

[0160] For example, vehicles (e.g. Figure 1 Memory 820 onboard autonomous vehicle 102 (e.g., in vehicle computing system 112) may store instructions 825 that, when executed by one or more processors 815, cause one or more processors 815 (e.g., in vehicle computing system 112) to perform operations, such as any operations and functions of computing device(s) 805 and / or vehicle computing system 112, any operations and functions configured for vehicle computing system 112, and / or any other operations and functions described herein.

[0161] Memory 820 can store data 830 that can be obtained (e.g., received, accessed, written, manipulated, created, generated, etc.) and / or stored. Data 830 can include, for example, service data (e.g., trip data, route data, user data, etc.), sensor data, map data, perception data, prediction data, motion planning data, object state and / or status data, and / or other data / information as described herein. In some embodiments, computing device(s) 805 can obtain data from one or more memories remote from autonomous vehicle 102.

[0162] The computing device(s) 805 may also include a communication interface 840 for communicating with one or more other systems (e.g., remote computing systems). The communication interface 840 may include any circuits, components, software, etc. for communicating via one or more networks (e.g., network(s)). In some implementations, the communication interface 840 may include, for example, one or more of: a communication controller, a receiver, a transceiver, a transmitter, a port, a conductor, software, and / or hardware for communicating data.

[0163] The computing tasks discussed herein that are performed at a computing device (or devices) remote from the autonomous vehicle may alternatively be performed at the autonomous vehicle (e.g., via a vehicle computing system), and vice versa. Such configurations may be implemented without departing from the scope of the present disclosure. The use of computer-based systems allows for various possible configurations, combinations, and divisions of tasks and functions between and within components. Computer-implemented operations may be performed on a single component or across multiple components. Computer-implemented tasks and / or operations may be performed sequentially or in parallel. Data and instructions may be stored in a single memory device or across multiple memory devices.

[0164] Aspects of the present disclosure have been described according to the illustrative embodiments of the present disclosure. By reading this disclosure, a person of ordinary skill in the art may conceive of many other embodiments, modifications and / or variations within the scope and spirit of the appended claims. Any and all features in the following claims may be combined and / or rearranged in any possible way.

[0165] Although the subject matter has been described in detail with respect to various specific exemplary embodiments of the subject matter, each example is provided by way of explanation rather than limitation of the present disclosure. Those skilled in the art, after obtaining an understanding of the foregoing, can easily produce changes, modifications and / or equivalents to these embodiments. Therefore, the present disclosure does not exclude including these modifications, changes and / or additions to the subject matter, which are obvious to those of ordinary skill in the art. For example, a feature shown and / or described as part of an embodiment can be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover these changes, modifications and / or equivalents.

Claims

1. A computer-implemented method for controlling an operating mode of an autonomous vehicle, comprising: When in a first operating mode, wherein the first operating mode is associated with the autonomous vehicle operating without a human operator: receiving a request to exit the first operating mode; determining that the autonomous vehicle is ready to exit the first operating mode; in response to receiving a signal indicating that the autonomous vehicle is ready to exit the first operating mode, transitioning the autonomous vehicle to a standby state; receiving confirmation to transition the autonomous vehicle to an unauthorized state while the autonomous vehicle is in a standby state; as well as The autonomous vehicle is transitioned to the unauthorized state based on the confirmation.

2. The computer-implemented method of claim 1 , wherein: The unauthorized state is associated with a neutral state in which the autonomous vehicle takes no action to switch operating modes and awaits an authorization request.

3. The computer-implemented method of claim 1 , wherein: The first operating mode includes a plurality of sub-states.

4. The computer-implemented method of claim 3, wherein: Transitioning the autonomous vehicle to the standby state further comprises: transitioning the autonomous vehicle to a first sub-state of a plurality of sub-states of the first operating mode; and Authorization is requested to transition to the standby state while the autonomous vehicle is in the first sub-state.

5. The computer-implemented method of claim 1 , wherein: The standby state includes a plurality of sub-states.

6. The computer-implemented method of claim 5, wherein: Transitioning the autonomous vehicle to the standby state includes: transitioning the autonomous vehicle to a first sub-state of a plurality of sub-states of the standby state; and A delayed confirmation is received to transition the autonomous vehicle to the standby state, wherein the delayed confirmation is received within a predetermined time.

7. The computer-implemented method of claim 5, wherein: Transitioning the autonomous vehicle to the standby sub-state includes: determining that confirmation to transition the autonomous vehicle to the standby state is not received within a predetermined time; transitioning the autonomous vehicle to a standby nominal sub-state; determining that confirmation to transition the autonomous vehicle to the unauthorized state has not been received; and The autonomous vehicle is transitioned to a standby request substate.

8. The computer-implemented method of claim 1 , wherein: The autonomous vehicle preparing to exit the first operating mode is based on movement of the autonomous vehicle.

9. The computer-implemented method of claim 1 , wherein: The confirmation is associated with a remote computing system providing remote assistance to the autonomous vehicle.

10. The computer-implemented method of claim 1, wherein: The unauthorized state includes a plurality of sub-states, at least one of the sub-states being associated with the transition to the standby state.

11. An autonomous vehicle control system for controlling an operating mode of an autonomous vehicle, the autonomous vehicle control system comprising: one or more processors; as well as a non-transitory computer-readable memory storing instructions executable by the one or more processors to perform operations comprising: When in a first operating mode, wherein the first operating mode is associated with operating the autonomous vehicle without a human operator: receiving a request to exit a first operating mode; determining that the autonomous vehicle is ready to exit a first operating mode; in response to receiving a signal indicating that the autonomous vehicle is ready to exit a first operating mode, transitioning the autonomous vehicle to a standby state; When the autonomous vehicle is in the standby state, receiving confirmation to transition the autonomous vehicle to an unauthorized state; and The autonomous vehicle is transitioned to the unauthorized state based on the confirmation.

12. The autonomous vehicle control system of claim 11, wherein: The unauthorized state is associated with a neutral state in which the autonomous vehicle takes no action to switch operating modes and awaits an authorization request.

13. The autonomous vehicle control system of claim 11, wherein: The first operating mode includes a plurality of sub-states.

14. The autonomous vehicle control system of claim 11, wherein: Transitioning the autonomous vehicle to the standby state further comprises: transitioning the autonomous vehicle to a first sub-state of a plurality of sub-states of a first operating mode, the first sub-state indicating receipt of a request to exit the first operating mode; and Authorization is requested to transition to the standby state while the autonomous vehicle is in the first sub-state.

15. The autonomous vehicle control system of claim 11, wherein: The standby state includes a plurality of sub-states.

16. The autonomous vehicle control system of claim 15, wherein: Transitioning the autonomous vehicle to the standby sub-state includes: determining that confirmation to transition the autonomous vehicle to the standby state is not received within a predetermined time; transitioning the autonomous vehicle to a standby nominal sub-state, wherein the standby nominal sub-state is associated with a no operator standby state; determining that confirmation to transition the autonomous vehicle to the unauthorized state has not been received; and The autonomous vehicle is transitioned to a standby request substate.

17. The autonomous vehicle control system of claim 11, wherein: The autonomous vehicle preparing to exit the first operating mode is based on movement of the autonomous vehicle.

18. The autonomous vehicle control system of claim 11, wherein: The confirmation is provided via a remote computing system.

19. An autonomous vehicle comprising: one or more processors; as well as a non-transitory computer-readable memory storing instructions executable by the one or more processors to perform operations comprising: When in a first operating mode, wherein the first operating mode is associated with the autonomous vehicle operating without a human operator: receiving a request to exit a first operating mode; determining that the autonomous vehicle is ready to exit a first operating mode; in response to receiving a signal indicating that the autonomous vehicle is ready to exit a first operating mode, transitioning the autonomous vehicle to a standby state; When the autonomous vehicle is in the standby state, receiving confirmation to transition the autonomous vehicle to an unauthorized state; and The autonomous vehicle is transitioned to the unauthorized state based on the confirmation.

20. The autonomous vehicle of claim 19, wherein: The unauthorized state is associated with a neutral state in which the autonomous vehicle does not take any action to switch operating modes.

Images