Monitor allocation system and method

Abstract

A method, computer program product, and computing system for defining a supervision level for each of a plurality of autonomous vehicles, thereby defining a plurality of tiered autonomous vehicles; assigning responsibility for each tiered autonomous vehicle to one of a plurality of vehicle monitors, thereby defining a vehicle workload for each of the plurality of vehicle monitors; sensing a change in the supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor; and re-examining the vehicle workload associated with the particular vehicle monitor.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 893,990, filed on August 30, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to supervisor assignment, and more particularly, to supervisor assignment for autonomous vehicles. Background Art

[0004] As the transportation industry moves toward autonomous (ie, driverless) vehicles, manufacturers and designers of these autonomous vehicles must determine contingency plans in the event that one or more systems within these autonomous vehicles fail.

[0005] As is well known, autonomous vehicles contain multiple electronic control units (ECUs), each of which can perform specific functions. For example, these different ECUs can calculate a safe trajectory for the vehicle (e.g., for navigating the vehicle to a predetermined destination) and can provide control signals to the vehicle's actuators, propulsion systems, and braking systems. Typically, one ECU (e.g., an autonomous control unit) can be responsible for planning and calculating the vehicle's trajectory and can provide commands to other ECUs to move the vehicle (e.g., by controlling the steering, braking, and powertrain ECUs).

[0006] As expected, such autonomous vehicles need to make navigation decisions that take their surroundings / environment into account. Moreover, these autonomous vehicles sometimes require varying degrees of supervision by a vehicle supervisor. Unfortunately, the level of supervision required for an autonomous vehicle may vary depending on how the autonomous vehicle is used, which may result in an unacceptably high workload for the vehicle supervisor. Summary of the invention

[0007] In one embodiment, a computer-implemented method is performed on a computing device and includes: defining a supervision level for each of a plurality of autonomous vehicles, thereby defining a plurality of tiered autonomous vehicles; assigning responsibility for each tiered autonomous vehicle to one of a plurality of vehicle monitors, thereby defining a vehicle workload for each of the plurality of vehicle monitors; sensing a change in the supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor; and re-examining the vehicle workload associated with the particular vehicle monitor.

[0008] One or more of the following features may be included. Sensing a change in the supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor may include: sensing that the particular tiered autonomous vehicle requires the full attention of the particular vehicle monitor. If the vehicle workload associated with the particular vehicle monitor exceeds a defined level, the responsibility for some other tiered autonomous vehicles assigned to the particular vehicle monitor may be reallocated to another vehicle monitor. Reallocating the responsibility for one or more tiered autonomous vehicles assigned to a particular vehicle monitor to another vehicle monitor may include: reallocating the responsibility for all other tiered autonomous vehicles assigned to the particular vehicle monitor to another vehicle monitor. The supervision level defined for each of the plurality of autonomous vehicles may include one or more category-based supervision levels. The supervision level defined for each of the plurality of autonomous vehicles may include one or more score-based supervision levels. The plurality of vehicle monitors may include: a plurality of human vehicle monitors.

[0009] In another embodiment, a computer program product resides on a computer-readable medium and has a plurality of instructions stored thereon. When executed by a processor, the instructions cause the processor to perform operations including: defining a supervision level for each of a plurality of autonomous vehicles, thereby defining a plurality of tiered autonomous vehicles; assigning responsibility for each tiered autonomous vehicle to one of a plurality of vehicle watchers, thereby defining a vehicle workload for each of the plurality of vehicle watchers; sensing a change in the supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle watcher; and rechecking the vehicle workload associated with the particular vehicle watcher.

[0010] One or more of the following features may be included. Sensing a change in the supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor may include: sensing that the particular tiered autonomous vehicle requires the full attention of the particular vehicle monitor. If the vehicle workload associated with the particular vehicle monitor exceeds a defined level, the responsibility for some other tiered autonomous vehicles assigned to the particular vehicle monitor may be reallocated to another vehicle monitor. Reallocating the responsibility for one or more tiered autonomous vehicles assigned to a particular vehicle monitor to another vehicle monitor may include: reallocating the responsibility for all other tiered autonomous vehicles assigned to the particular vehicle monitor to another vehicle monitor. The supervision level defined for each of a plurality of autonomous vehicles may include one or more category-based supervision levels. The supervision level defined for each of a plurality of autonomous vehicles may include one or more score-based supervision levels. The plurality of vehicle monitors may include: a plurality of human vehicle monitors.

[0011] In another embodiment, a computing system includes a processor and a memory configured to perform operations including: defining a supervision level for each of a plurality of autonomous vehicles, thereby defining a plurality of tiered autonomous vehicles; assigning responsibility for each tiered autonomous vehicle to one of a plurality of vehicle watchers, thereby defining a vehicle workload for each of the plurality of vehicle watchers; sensing a change in the supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle watcher; and re-examining the vehicle workload associated with the particular vehicle watcher.

[0012] One or more of the following features may be included. Sensing a change in the supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor may include: sensing that the particular tiered autonomous vehicle requires the full attention of the particular vehicle monitor. If the vehicle workload associated with the particular vehicle monitor exceeds a defined level, the responsibility for some other tiered autonomous vehicles assigned to the particular vehicle monitor may be reallocated to another vehicle monitor. Reallocating the responsibility for one or more tiered autonomous vehicles assigned to a particular vehicle monitor to another vehicle monitor may include: reallocating the responsibility for all other tiered autonomous vehicles assigned to the particular vehicle monitor to another vehicle monitor. The supervision level defined for each of a plurality of autonomous vehicles may include one or more category-based supervision levels. The supervision level defined for each of a plurality of autonomous vehicles may include one or more score-based supervision levels. The plurality of vehicle monitors may include: a plurality of human vehicle monitors.

[0013] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a schematic diagram of an autonomous vehicle according to one embodiment of the present disclosure;

[0015] Figure 2A According to one embodiment of the present disclosure, Figure 1 A schematic diagram of one embodiment of various systems within an autonomous vehicle;

[0016] Figure 2B According to one embodiment of the present disclosure, Figure 1 A schematic diagram of another embodiment of various systems within an autonomous vehicle;

[0017] Figure 3 According to one embodiment of the present disclosure, Figure 1A schematic diagram of another embodiment of various systems within an autonomous vehicle;

[0018] Figure 4 is a schematic diagram of multiple vehicle monitors according to one embodiment of the present disclosure; and

[0019] Figure 5 According to one embodiment of the present disclosure, Figure 4 Flowchart of a monitor assignment process for a vehicle monitor.

[0020] Like reference numerals in the various drawings represent like elements. DETAILED DESCRIPTION

[0021] Autonomous Vehicle Overview

[0022] Reference Figure 1 , an autonomous vehicle 10 is shown. As is known in the art, an autonomous vehicle (such as autonomous vehicle 10) is a vehicle that is capable of sensing its environment and moving with little or no human input. An autonomous vehicle (such as autonomous vehicle 10) may incorporate various sensor systems to sense its surroundings, examples of which may include, but are not limited to, radar, computer vision, lidar, GPS, odometer, temperature, and inertial, wherein such sensor systems may be configured to interpret lanes and markings on the road, road signs, traffic lights, pedestrians, other vehicles, roadside objects, hazards, etc.

[0023] Autonomous vehicle 10 may include a plurality of sensors (e.g., sensor 12), a plurality of electronic control units (e.g., ECU 14), and a plurality of actuators (e.g., actuator 16). Thus, sensor 12 within autonomous vehicle 10 may monitor the environment in which autonomous vehicle 10 is located, wherein sensor 12 may provide sensor data 18 to ECU 14. ECU 14 may process sensor data 18 to determine in what manner autonomous vehicle 10 should move. ECU 14 may then provide control data 20 to actuator 16 so that autonomous vehicle 10 may move in the manner determined by ECU 14. For example, a machine vision sensor included within sensor 12 may “read” a speed limit sign indicating that the speed limit on the road on which autonomous vehicle 10 is traveling is now 35 miles per hour. This machine vision sensor included within sensor 12 may provide sensor data 18 to ECU 14 indicating that the speed on the road on which autonomous vehicle 10 is traveling is now 35 miles per hour. Upon receiving sensor data 18, ECU 14 may process sensor data 18 and may determine that autonomous vehicle 10 (currently traveling at 45 mph) is traveling too fast and needs to slow down. Accordingly, ECU 14 may provide control data 20 to actuator 16, where control data 20 may, for example, apply the brakes of autonomous vehicle 10 or eliminate any actuation signal currently applied to the accelerator (thereby allowing autonomous vehicle 10 to coast until the speed of autonomous vehicle 10 decreases to 35 mph).

[0024] System Redundancy

[0025] It is conceivable that since autonomous vehicle 10 is controlled by various electronic systems included therein (such as sensors 12, ECU 14, and actuators 16), possible failures of one or more of these systems should be considered when designing autonomous vehicle 10, and appropriate contingency plans may be adopted.

[0026] For example, and also with reference to Figure 2A , the various ECUs (e.g., ECU 14) included in the autonomous vehicle 10 can be separated so that the responsibilities of the various ECUs (e.g., ECU 14) can be logically grouped. For example, ECU 14 can include an autonomous control unit 50 that can receive sensor data 18 from sensor 12.

[0027] Autonomous control unit 50 may be configured to perform a variety of functions. For example, autonomous control unit 50 may receive and process external sensor data (e.g., sensor data 18), may estimate the position of autonomous vehicle 10 in its operating environment, may calculate a representation of the surroundings of autonomous vehicle 10, may calculate a safe trajectory for autonomous vehicle 10, and may command other ECUs (particularly vehicle control units) to cause autonomous vehicle 10 to perform desired maneuvers. Autonomous control unit 50 may include a large amount of computing power, persistent storage, and memory.

[0028] Thus, autonomous control unit 50 may process sensor data 18 to determine the manner in which autonomous vehicle 10 should operate. Autonomous control unit 50 may then provide vehicle control data 52 to vehicle control unit 54, which may then process vehicle control data 52 to determine the manner in which various control systems (e.g., powertrain system 56, braking system 58, and steering system 60) should respond to achieve the trajectory defined by autonomous control unit 50 within vehicle control data 52.

[0029] The vehicle control unit 54 may be configured to control other ECUs included in the autonomous vehicle 10. For example, the vehicle control unit 54 may control steering, powertrain, and brake controller units. For example, the vehicle control unit 54 may provide: a powertrain control signal 62 to a powertrain control unit 64; a brake control signal 66 to a brake control unit 68; and a steering control signal 70 to a steering control unit 72.

[0030] The powertrain control unit 64 may process the powertrain control signal 62 so that appropriate control data (generally represented by the control data 20) may be provided to the powertrain system 56. Additionally, the brake control unit 68 may process the brake control signal 66 so that appropriate control data (generally represented by the control data 20) may be provided to the brake system 58. Furthermore, the steering control unit 72 may process the steering control signal 70 so that appropriate control data (generally represented by the control data 20) may be provided to the steering system 60.

[0031] Powertrain control unit 64 may be configured to control a transmission (not shown) and an engine / traction motor (not shown) within autonomous vehicle 10 ; brake control unit 68 may be configured to control a mechanical / regenerative braking system (not shown) within autonomous vehicle 10 ; and steering control unit 72 may be configured to control a steering column / steering rack (not shown) within autonomous vehicle 10 .

[0032] Autonomous control unit 50 may be a highly complex computing system that may provide extensive processing capabilities (e.g., a workstation-class computing system with multi-core processors, discrete co-processing units, several gigabytes of memory, and persistent storage). In contrast, vehicle control unit 54 may be a much simpler device that may provide processing capabilities comparable to other ECUs included within autonomous vehicle 10 (e.g., a computing system with a modest microprocessor (CPU frequency less than 200 MHz), less than 1 megabyte of system memory, and no persistent storage). Due to these simpler designs, vehicle control unit 54 may have greater reliability and durability than autonomous control unit 50.

[0033] To further improve redundancy and reliability, one or more ECUs (ECU 14) included in autonomous vehicle 10 may be configured in a redundant manner. For example, and with reference to Figure 2B , shows an embodiment of the ECU 14, wherein multiple vehicle control units are utilized. For example, this particular embodiment is shown as including two vehicle control units, namely a first vehicle control unit (e.g., vehicle control unit 54) and a second vehicle control unit (e.g., vehicle control unit 74).

[0034] In this particular configuration, the two vehicle control units (e.g., vehicle control units 54, 74) can be configured in various ways. For example, the two vehicle control units (e.g., vehicle control units 54, 74) can be configured as an active-passive configuration, wherein, for example, the vehicle control unit 54 performs the active role of processing the vehicle control data 52, while the vehicle control unit 74 assumes a passive role and is essentially in standby mode. In the event of a failure of the vehicle control unit 54, the vehicle control unit 74 can transition from the passive role to the active role and assume the role of processing the vehicle control data 52. Alternatively, the two vehicle control units (e.g., vehicle control units 54, 74) can be configured as an active-active configuration, wherein, for example, the vehicle control unit 52 and the vehicle control unit 74 both perform the active role of processing the vehicle control data 54 (e.g., sharing the workload), wherein in the event of a failure of the vehicle control unit 54 or the vehicle control unit 74, the surviving vehicle control unit can process all the vehicle control data 52.

[0035] Although Figure 2BOne example of the manner in which various ECUs (e.g., ECU 14) included within autonomous vehicle 10 may be configured in a redundant manner is presented, but this is for illustrative purposes only and is not intended to be a limitation of the present disclosure, as other configurations are possible and are considered to be within the scope of the present disclosure. For example, autonomous control unit 50 may be configured in a redundant manner, wherein a second autonomous control unit (not shown) is included within autonomous vehicle 10 and configured in an active-passive or active-active manner. In addition, it is contemplated that one or more sensors (e.g., sensor 12) and / or one or more actuators (e.g., actuator 16) may be configured in a redundant manner. Therefore, it will be appreciated that with respect to autonomous vehicle 10, the degree of redundancy that can be achieved may be limited only by the design criteria of autonomous vehicle 10 and budgetary constraints.

[0036] Autonomic computing subsystem

[0037] Also refer to Figure 3 , various ECUs of the autonomous vehicle 10 may be grouped / arranged / configured to implement various functions.

[0038] For example, one or more of the ECUs 14 may be configured to implement / form a perception subsystem 100. Among other things, the perception subsystem 100 may be configured to process data from onboard sensors (e.g., sensor data 18) to compute a concise representation of objects of interest near the autonomous vehicle 10 (examples of which may include, but are not limited to, other vehicles, pedestrians, traffic signals, traffic signs, road signs, hazards, etc.), and identify environmental features that may be helpful in determining the location of the autonomous vehicle 10. In addition, one or more of the ECUs 14 may be configured to implement / form a state estimation subsystem 102, wherein the state estimation subsystem 102 may be configured to process data from onboard sensors (e.g., sensor data 18) to estimate the position, orientation, and speed of the autonomous vehicle 10 in its operating environment. Additionally, one or more of the ECUs 14 may be configured to implement / form a planning subsystem 104, wherein the planning subsystem 104 may be configured to compute a desired vehicle trajectory (using the perception output 106 and the state estimation output 108). Still further, one or more of the ECUs 14 may be configured to implement / form a trajectory control subsystem 110 that uses the planning output 112 and the state estimation output 108 (in conjunction with feedback and / or feedforward control techniques) to calculate actuator commands (e.g., control data 20) that may cause the autonomous vehicle 10 to execute its predetermined trajectory within its operating environment.

[0039] For redundancy purposes, the above-described subsystems may be distributed across different devices (e.g., the autonomous control unit 50 and the vehicle control unit 54, 74). Additionally / alternatively, due to increased computational requirements, the perception subsystem 100 and the planning subsystem 104 may be located almost entirely within the autonomous control unit 50, which (as described above) has more computational horsepower than the vehicle control unit 54, 74. Conversely, due to their lower computational requirements, the state estimation subsystem 102 and the trajectory control subsystem 110 may be: located entirely on the vehicle control unit 54, 74 if the vehicle control unit 54, 74 has the necessary computational power; and / or located partially on the vehicle control unit 54, 74 and partially on the autonomous control unit 50. However, the location of the state estimation subsystem 102 and the trajectory control subsystem 110 may be critical in the design of any contingency planning architecture, as the location of these subsystems may determine how the contingency plans are computed, transmitted, and / or executed.

[0040] Trajectory calculation

[0041] During typical operation of autonomous vehicle 10, the autonomous subsystems described above repeatedly perform the following functions:

[0042] Using onboard sensors to measure the surrounding environment (e.g., using sensor 12);

[0043] estimating the position, velocity, and future trajectory of surrounding vehicles, pedestrians, bicyclists, other objects near the autonomous vehicle 10, and environmental features useful for determining position (e.g., using the perception subsystem 100);

[0044] estimating the position, orientation, and velocity of autonomous vehicle 10 in the operating environment (e.g., using state estimation subsystem 102);

[0045] planning a nominal trajectory for autonomous vehicle 10 to follow that brings autonomous vehicle 10 closer to a predetermined destination of autonomous vehicle 10 (e.g., using planning subsystem 104); and

[0046] Commands (eg, control data 20 ) are generated to cause autonomous vehicle 10 to execute a predetermined trajectory (eg, using trajectory control subsystem 110 ).

[0047] In each iteration, planning subsystem 104 may calculate a trajectory that may span many meters (distance) and many seconds (time). However, each iteration of the above loop may be calculated more frequently (e.g., every 10 milliseconds). Thus, autonomous vehicle 10 may be expected to execute only a small portion of each planned trajectory (which may differ from a previously calculated trajectory due to, for example, sensed changes in the environment) before calculating a new trajectory.

[0048] Trajectory Execution

[0049] The above trajectory can be represented as a parameterized curve describing the desired future path of the autonomous vehicle 10. While executing the above trajectory, there may be two major categories of techniques for controlling the autonomous vehicle 10: a) feedforward control and b) feedback control.

[0050] Under nominal conditions, the trajectory is executed using feedback control, where the feedback trajectory control algorithm can use, for example, a kinematic model of autonomous vehicle 10, configuration parameters of each vehicle, and continuously calculated estimates of the position, orientation, and velocity of autonomous vehicle 10 to calculate commands provided to various ECUs included within autonomous vehicle 10.

[0051] The feedforward trajectory control algorithm may use a kinematic model of autonomous vehicle 10, configuration parameters of each vehicle, and a single estimate of the initial position, orientation, and velocity of autonomous vehicle 10 to calculate a sequence of commands provided to various ECUs included within autonomous vehicle 10, wherein the command sequence is executed without using any real-time sensor data (e.g., from sensor 12) or other information.

[0052] To execute the above trajectory, the autonomous control unit 50 can communicate with (and can provide commands to) various ECUs, using the vehicle control unit 54 / 74 as a middleman. At each iteration of the above trajectory execution loop, the autonomous control unit 50 can calculate steering, powertrain, and braking commands that are provided to their respective ECUs (e.g., the powertrain control unit 64, the brake control unit 68, and the steering control unit 72, respectively), and can transmit these commands to the vehicle control unit 54 / 74. The vehicle control unit 54 / 74 can then verify these commands and can forward them to the respective ECUs (e.g., the powertrain control unit 64, the brake control unit 68, and the steering control unit 72, respectively).

[0053] Monitor Assignment Process

[0054] As described above, during a typical operation of the autonomous vehicle 10, the autonomous subsystems described above may repeatedly perform the following functions: measure the surrounding environment using onboard sensors (e.g., using sensors 12); estimate the position, velocity, and future trajectory of surrounding vehicles, pedestrians, cyclists, other objects near the autonomous vehicle 10, and environmental features useful for position determination (e.g., using perception subsystem 100); estimate the position, orientation, and velocity of the autonomous vehicle 10 in the operating environment (e.g., using state estimation subsystem 102); plan a nominal trajectory for the autonomous vehicle 10 to follow so that the autonomous vehicle 10 gets closer to the intended destination of the autonomous vehicle 10 (e.g., using planning subsystem 104); and generate commands (e.g., control data 20) to cause the autonomous vehicle 10 to execute the intended trajectory (e.g., using trajectory control subsystem 110).

[0055] The operation of autonomous vehicle 10 may be supervised by a vehicle monitor (e.g., a human vehicle monitor). Specifically, and in a manner similar to how an air traffic controller monitors the operation of one or more aircraft, a vehicle monitor may monitor the operation of one or more autonomous vehicles (e.g., autonomous vehicle 10).

[0056] For example, while referring to Figure 4 , vehicle monitors (e.g., vehicle monitors 200, 202, 204) may be located in a centralized location (e.g., a monitoring center) and may monitor the operation of various autonomous vehicles (e.g., autonomous vehicle 10). For example, vehicle monitors 200, 202, 204 may (in this example) monitor the operation of nine autonomous vehicles (e.g., autonomous vehicle No. 1 through autonomous vehicle No. 9), each of which is represented as a unique circle on the display of vehicle monitors 200, 202, 204. Specifically, for this example, it is assumed that vehicle monitors 200, 202 are advanced vehicle monitors who are able to monitor more autonomous vehicles than entry-level vehicle monitor 204.

[0057] Also refer to Figure 5 , monitor assignment process 250 can define 300 a monitoring level for each of a plurality of autonomous vehicles, thereby defining a plurality of autonomous vehicles assigned by level (eg, autonomous vehicle #1 through autonomous vehicle #9).

[0058] Monitor assignment process 250 may be a server application and may reside on and be executed by a computing device 252, which may be connected to a network 254 (e.g., the Internet or a local area network). Examples of computing device 252 may include, but are not limited to, a personal computer, a laptop, a personal digital assistant, a mobile phone with data capabilities, a laptop, a television having one or more processors embedded therein or coupled thereto, a cable / satellite receiver having one or more processors embedded therein or coupled thereto, a server computer, a series of server computers, a microcomputer, a mainframe, or a cloud-based computing network.

[0059] The instruction set and subroutines of the monitor assignment process 250 may be stored on a storage device 256 coupled to the computing device 252 and may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within the computing device 252. Examples of the storage device 256 may include, but are not limited to: a hard drive; a RAID device; a random access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices.

[0060] A network 254 (e.g., the Internet or a local area network) can couple the computing device 252 to client electronic devices (e.g., client electronic devices 258, 260, 262) utilized by the vehicle monitors 200, 202, 204 (respectively). Examples of client electronic devices 258, 260, 262 can include, but are not limited to, mobile phones with data capabilities, laptop computers, personal digital assistants, personal computers, notebook computers, workstation computers, smart televisions, and dedicated network devices. The client electronic devices 258, 260, 262 can each execute an operating system, examples of which can include, but are not limited to, Microsoft Windows tm 、Android tm 、WebOS tm 、iOS tm 、Redhat Linux tm , or a custom operating system.

[0061] The level of supervision assigned to each of a plurality of autonomous vehicles (in this example, nine autonomous vehicles) can vary from, for example, none to full. For example, when an autonomous vehicle is out of service (e.g., in storage, charging, or not in use), the level of supervision assigned to the vehicle can be none or minimal. Generally, when an autonomous vehicle is charging, a vehicle monitor may not be required to monitor the autonomous vehicle (because it is not moving). Alternatively, a vehicle monitor may provide minimal supervision to monitor, for example, the status of charging so that a time to return to service can be determined for the autonomous vehicle.

[0062] As can be imagined, the level of supervision that a vehicle monitor must apply to an autonomous vehicle may vary depending on the specific operating conditions of the autonomous vehicle. Thus, if an autonomous vehicle is traveling on a rural road with minimal traffic, a vehicle monitor may apply a low level of supervision, whereas if an autonomous vehicle is traveling on an urban road with heavy traffic, a vehicle monitor may apply a higher level of supervision. Furthermore, in certain circumstances, a vehicle monitor may need to devote 100% of its attention to a single autonomous vehicle. For example, if a passenger of an autonomous vehicle requests to speak with a vehicle monitor (e.g., to inquire about their route, change their destination, or complain about the condition of the vehicle), the vehicle monitor may need to devote 100% of its attention to the conversation. Furthermore, in certain circumstances, an autonomous vehicle may require guidance from a vehicle monitor. For example, if an accident occurs on a road and the travel lane is blocked, the autonomous vehicle may request permission from a vehicle monitor to cross the double yellow line and drive around the accident. This may require a vehicle monitor to review camera data to ensure that the autonomous vehicle can safely cross the double yellow line and drive around the accident.

[0063] Thus, the supervision level 300 defined for each of the plurality of autonomous vehicles may include one or more category-based supervision levels. For example, the supervisor assignment process 250 may define 300 category-based supervision levels such as:

[0064] None (e.g., for autonomous vehicles that are out of service or charging);

[0065] Low (e.g., for an autonomous vehicle operating on a rural road with minimal traffic);

[0066] Moderate (e.g., for an autonomous vehicle operating on a congested urban road during rush hour); and

[0067] High (e.g., for an autonomous vehicle with a passenger who wants to talk to the vehicle monitor or needs to cross a double yellow line).

[0068] Additionally / alternatively, the supervision level defined 300 for each of the plurality of autonomous vehicles may be much more granular, and may include one or more score-based supervision levels.

[0069] 0 (e.g., for an autonomous vehicle that is out of service or charging).

[0070] 1-25 (e.g., for an autonomous vehicle operating on a rural road with minimal traffic).

[0071] 50-99 (e.g., for an autonomous vehicle operating on a congested urban road during rush hour); and

[0072] 100 (e.g., for an autonomous vehicle with a passenger who wants to talk to the vehicle monitor or needs to cross a double yellow line).

[0073] The monitor assignment process 250 may assign 302 the responsibility of each hierarchically assigned autonomous vehicle (e.g., autonomous vehicle #1 through autonomous vehicle #9) to one of a plurality of vehicle monitors (e.g., vehicle monitors 200, 202, 204), thereby defining a vehicle workload for each of the plurality of vehicle monitors (e.g., vehicle monitors 200, 202, 204).

[0074] In this example, assume that the monitor assigns process 250:

[0075] • Assigning 302 responsibility for the five hierarchically assigned autonomous vehicles (ie, hierarchically assigned autonomous vehicles 1-5) to vehicle monitor 200, who is a senior vehicle monitor.

[0076] • Assigning 302 responsibility for three hierarchically assigned autonomous vehicles (ie, hierarchically assigned autonomous vehicles 6-8) to vehicle monitor 202, who is a mid-level vehicle monitor.

[0077] • Assigning 302 responsibility for a hierarchically assigned autonomous vehicle (ie, hierarchically assigned autonomous vehicle 9) to vehicle monitor 204, who is a primary vehicle monitor.

[0078] Specifically, monitor assignment process 250 may consider the experience level of the individual vehicle monitors when assigning 302 responsibility for each hierarchically assigned autonomous vehicle (e.g., autonomous vehicle #1 through autonomous vehicle #9) to one of a plurality of vehicle monitors (e.g., vehicle monitors 200, 202, 204) to avoid overloading them. For example,

[0079] A high-level vehicle monitor may have a maximum workload of seven autonomous vehicles assigned to the class (i.e., may monitor up to seven autonomous vehicles assigned to the class simultaneously);

[0080] A mid-level vehicle monitor may have a maximum workload of five class-assigned autonomous vehicles (i.e., may monitor up to five class-assigned autonomous vehicles simultaneously); and

[0081] • A primary vehicle monitor may have a maximum workload of three tiered autonomous vehicles (i.e., may monitor a maximum of three tiered autonomous vehicles simultaneously).

[0082] As described above, the level of supervision required for a tiered autonomous vehicle may vary from no supervision, low supervision, high supervision, to full supervision. Thus, the maximum workload description described above may be for, for example, a tiered autonomous vehicle with low supervision, where, for example, a tiered autonomous vehicle with high supervision may count as, for example, two tiered autonomous vehicles with low supervision. Thus, in such a configuration, tiered autonomous vehicles with low supervision and tiered autonomous vehicles with high supervision may be mixed and matched at different levels of vehicle supervisors. For example:

[0083] A high-level vehicle monitor may have a maximum workload of, for example, three high-supervision assigned autonomous vehicles and one low-supervision assigned autonomous vehicle;

[0084] A mid-level vehicle monitor may have a maximum workload of, for example, two high-supervision assigned autonomous vehicles and one low-supervision assigned autonomous vehicle; and

[0085] • A primary vehicle monitor may have a maximum workload of, for example, one high-supervision assigned autonomous vehicle and one low-supervision assigned autonomous vehicle.

[0086] As described above, for those class-assigned autonomous vehicles that require full supervision, any vehicle monitor monitoring the class-assigned autonomous vehicles with full supervision may only monitor that single class-assigned autonomous vehicle (and no others).

[0087] It is conceivable that the level of supervision required of autonomous vehicles will change during their operation (e.g., as they move from charging / de-service, to navigating rural roads, to navigating congested city streets, to navigating incidents, to facilitating communications between passengers and vehicle monitors, etc.). Accordingly, monitor assignment process 250 can monitor the supervision level of each of the level-assigned autonomous vehicles (e.g., autonomous vehicle #1 through autonomous vehicle #9) to determine whether they have changed.

[0088] When monitor assignment process 250 senses 304 a change in the supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor, monitor assignment process 250 may recheck 306 the vehicle workload associated with the particular vehicle monitor. If the vehicle workload associated with the particular vehicle monitor exceeds a defined level, monitor assignment process 250 may reassign 308 the responsibility for some other tiered autonomous vehicles assigned to the particular vehicle monitor to another vehicle monitor.

[0089] For example, if monitor assignment process 250 senses 304 that the supervision level of a particular tiered autonomous vehicle (e.g., tiered autonomous vehicle 3) assigned to a particular vehicle monitor (e.g., vehicle monitor 200) changes from low supervision to some other supervision level, monitor assignment process 250 may recheck 306 the vehicle workload associated with the particular vehicle monitor (e.g., vehicle monitor 200). And if the vehicle workload associated with the particular vehicle monitor (e.g., vehicle monitor 200) exceeds the defined level, monitor assignment process 250 may reallocate 308 the responsibility for some other tiered autonomous vehicles assigned to the particular vehicle monitor to another vehicle monitor.

[0090] As described above, assume that monitor assignment process 250 assigns 302 the responsibility of five hierarchically assigned autonomous vehicles (i.e., hierarchically assigned autonomous vehicles 1-5) to vehicle monitor 200, which is a senior vehicle monitor. Furthermore, as described above, a senior vehicle monitor may have a maximum workload of, for example, seven hierarchically assigned autonomous vehicles with low supervision, or three hierarchically assigned autonomous vehicles with high supervision, and one hierarchically assigned autonomous vehicle with low supervision. Therefore, if all five autonomous vehicles 1-5 assigned 302 to vehicle monitor 200 are hierarchically assigned autonomous vehicles with low supervision, the vehicle workload associated with vehicle monitor 200 is acceptable. Therefore, monitor assignment process 250 does not need to reallocate 308 the responsibility of any hierarchically assigned autonomous vehicles from vehicle monitor 200 to other vehicle monitors.

[0091] However, if all five autonomous vehicles 1-5 assigned 302 to vehicle monitor 200 were initially low-supervision tiered autonomous vehicles, and monitor assignment process 250 senses 304 that the supervision level of four of these low-supervision tiered autonomous vehicles (e.g., tiered autonomous vehicles 1-4) has changed from low supervision to high supervision, then a reallocation may be necessary 308. Therefore, and when monitor assignment process 250 re-examines 306 the vehicle workload associated with vehicle monitor 200, monitor assignment process 250 may determine that the vehicle workload associated with vehicle monitor 200 is unacceptable (because it is now four high-supervision tiered autonomous vehicles and one low-supervision tiered autonomous vehicle; this exceeds the maximum vehicle workload of the high-level vehicle monitor of three high-supervision tiered autonomous vehicles and one low-supervision tiered autonomous vehicle). Thus, monitor assignment process 250 may reallocate 308 the responsibility for one of the high-supervision assigned-level autonomous vehicles (e.g., assigned-level autonomous vehicle 3) from vehicle monitor 200 to one of vehicle monitor 202 and vehicle monitor 204, because both vehicle monitors 202, 204 have available workload bandwidth to handle the high-supervision assigned-level autonomous vehicle (e.g., assigned-level autonomous vehicle 3).

[0092] As described above, for those tiered assigned autonomous vehicles that require full supervision, any vehicle monitor that is monitoring a tiered assigned autonomous vehicle with full supervision may only monitor that single tiered assigned autonomous vehicle with full supervision (and no others). Thus, if a tiered assigned autonomous vehicle monitored by a vehicle monitor becomes a tiered assigned autonomous vehicle with full supervision, monitor assignment process 250 may reassign all other tiered assigned autonomous vehicles currently being monitored by the vehicle monitor.

[0093] For example, assume that upon sensing 304 a change in the level of supervision of a particular tiered autonomous vehicle assigned to a particular vehicle monitor, monitor assignment process 250 senses 310 that tiered autonomous vehicle 3 has transitioned from a tiered autonomous vehicle of low supervision to a tiered autonomous vehicle of high supervision; now requiring the full attention of vehicle monitor 200. In this example, assume that a passenger within tiered autonomous vehicle 3 wishes to converse with vehicle monitor 200, and therefore the full attention of vehicle monitor 200 is required.

[0094] Thus, and upon reassigning 308 responsibility for one or more tiered autonomous vehicles assigned to a particular vehicle monitor to another vehicle monitor, monitor assignment process 250 may reassign 312 responsibility for all other tiered autonomous vehicles assigned to vehicle monitor 200 to another vehicle monitor. For example, monitor assignment process 250 may reassign 312

[0095] Reassigning the responsibility for the autonomous vehicle 1 assigned by level from the vehicle monitor 200 to the vehicle monitor 202;

[0096] Reassigning the responsibility for the autonomous vehicle 2 assigned by level from the vehicle monitor 200 to the vehicle monitor 202;

[0097] Reassigning responsibility for the hierarchical allocation of autonomous vehicle 4 from vehicle monitor 200 to vehicle monitor 204; and

[0098] Reassigning the responsibility for the hierarchically assigned autonomous vehicle 5 from vehicle monitor 200 to vehicle monitor 204 .

[0099] Therefore, after this reallocation of 312, the following vehicle workloads will occur:

[0100] • Vehicle Supervisor 200 (who is the senior vehicle supervisor) will now be responsible only for the tier-assigned autonomous vehicles that are under full supervision (ie, tier-assigned autonomous vehicle 3).

[0101] Vehicle monitor 202 (who is a mid-level vehicle monitor) will now be responsible for their original three low-supervision tiered autonomous vehicles (i.e., tiered autonomous vehicles 6-8), plus the two reassigned low-supervision tiered autonomous vehicles (i.e., tiered autonomous vehicles 1-2).

[0102] Vehicle monitor 204 (who is the primary vehicle monitor) will now be responsible for their original one tiered autonomous vehicle (i.e., tiered autonomous vehicle 9) plus two reassigned low-supervision tiered autonomous vehicles (i.e., tiered autonomous vehicles 4-5).

[0103] Summary

[0104] As will be appreciated by those skilled in the art, the present disclosure may be embodied as a method, a system, or a computer program product. Thus, the present disclosure may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, which may all be generally referred to herein as a "circuit," "module," or "system." In addition, the present disclosure may take the form of a computer program product on a computer-usable storage medium having a computer-usable program code embodied in the medium.

[0105] Any suitable computer-usable or computer-readable medium may be utilized. A computer-usable or computer-readable medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device, apparatus or propagation medium. More specific examples of computer-readable media (a non-exhaustive list) may include the following: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a transmission medium such as one that supports the Internet or an intranet, or a magnetic storage device. A computer-usable or computer-readable medium may also be paper or another suitable medium on which the program is printed, because the program may be electronically generated, for example, by optical scanning of paper or other media, and then compiled, interpreted, or processed in a suitable manner as necessary, and then stored in a computer memory. In the context of this article, a computer-usable or computer-readable medium may be any medium that can contain, store, communicate, propagate or transport a program for use or in connection with an instruction execution system, apparatus or device. The computer usable medium may include a propagated data signal in which the computer usable program code is embodied, and the data signal is a baseband or as part of a carrier wave. The computer usable program code may be propagated using any appropriate medium, including but not limited to the Internet, wired, optical cable, radio frequency, etc.

[0106] The computer program code for performing the operation of the present disclosure can be written in an object-oriented programming language, such as Java, Smalltalk, C++ or similar languages. However, the computer program code for performing the operation of the present disclosure can also be written in a traditional procedural programming language, such as "C" programming language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package and partially on the user's computer, partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via a local area network / wide area network / internet (such as network 14).

[0107] The present disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It is understood that each block in the flowchart illustration and / or block diagram, as well as the combination of blocks in the flowchart illustration and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer / special-purpose computer / other programmable data processing device, so that instructions executed by a processor of a computer or other programmable data processing device can create a means for implementing the functions / behaviors specified in one or more blocks in the flowchart and / or block diagram.

[0108] These computer program instructions may also be stored in a computer-readable memory, which may instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product, including instruction means for implementing the functions / behaviors specified in the flowchart and / or block diagram.

[0109] Computer program instructions may also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions / behaviors specified in the flowchart and / or block diagram.

[0110] The flowcharts and block diagrams in the figures can illustrate the structure, function and operation of possible implementation schemes of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, segment or part of the code, which includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative embodiments, the functions indicated in the blocks may not appear in the order indicated in the figure. For example, two blocks displayed in succession may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart description, as well as the combination of blocks in the block diagram and / or flowchart description, can be implemented by a hardware-based special-purpose system that performs a specified function or behavior, or a combination of special-purpose hardware and computer instructions.

[0111] The terms used herein are only used to describe specific embodiments and are not intended to limit the disclosure. As used herein, the singular forms "one", "an" and "the" also include plural forms unless the context clearly indicates. It should be further understood that when the terms "include" and / or "comprise" are used in this specification, the specific existence of the features, integers, steps, operations, elements and / or components is specified, but the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof is not excluded.

[0112] The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the following claims are intended to include any structure, material, or act that performs the function in combination with other specifically claimed elements. The description of the present disclosure is presented for the purpose of illustration and description, but is not intended to be exhaustive or limited to the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the present disclosure. The embodiments are selected and described in order to best explain the principles and practical applications of the present disclosure and to enable those of ordinary skill in the art to understand the various embodiments of the present disclosure and to make various modifications thereto to suit the specific purposes contemplated.

[0113] Having described the disclosure of the present application in detail and by reference to embodiments thereof, it will be apparent that modifications and variations are possible without departing from the scope of the disclosure as defined in the appended claims.

Claims

1. A computer-implemented method, executed on a computing device, comprising: defining a supervision level for each of the plurality of autonomous vehicles, thereby defining a plurality of autonomous vehicles assigned by level; determining a respective maximum workload for each of a plurality of vehicle monitors; assigning responsibility for each tiered autonomous vehicle to a respective vehicle monitor of the plurality of vehicle monitors based on the respective maximum workload of each vehicle monitor and the supervision level of the respective tiered autonomous vehicle, thereby defining a vehicle workload for each of the plurality of vehicle monitors; sensing a change in a supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor, wherein sensing the change in supervision level includes determining that the particular tiered autonomous vehicle is traveling on a rural highway with minimal traffic; as well as In response to sensing a change in the supervision level of the particular tiered autonomous vehicle, a vehicle workload associated with the particular vehicle monitor is re-examined.

2. The computer-implemented method of claim 1, wherein: Sensing a change in the supervision level of a particular tier-assigned autonomous vehicle assigned to a particular vehicle monitor includes: Sensing a particular class-assigned autonomous vehicle requires the full attention of the particular vehicle observer.

3. The computer-implemented method of claim 1 , further comprising: If the vehicle workload associated with a particular vehicle monitor exceeds a defined level, the responsibility for some other tiered autonomous vehicles assigned to the particular vehicle monitor is reallocated to another vehicle monitor.

4. The computer-implemented method of claim 3, wherein: Reassigning responsibility for one or more tiered autonomous vehicles assigned to a particular vehicle monitor to another vehicle monitor includes: Responsibility for all other tier-assigned autonomous vehicles assigned to a particular vehicle monitor is reassigned to another vehicle monitor.

5. The computer-implemented method of claim 1, wherein: The supervision level defined for each of the plurality of autonomous vehicles includes one or more category-based supervision levels.

6. The computer-implemented method of claim 1, wherein: The supervision level defined for each of the plurality of autonomous vehicles includes one or more score-based supervision levels.

7. The computer-implemented method of claim 1, wherein: The plurality of vehicle monitors include: Multiple human vehicle monitors.

8. A computer program product residing on a computer-readable medium, the medium having a plurality of instructions stored thereon, which when executed by a processor, cause the processor to perform operations comprising: defining a supervision level for each of the plurality of autonomous vehicles, thereby defining a plurality of autonomous vehicles assigned by level; determining a respective maximum workload for each of a plurality of vehicle monitors; assigning responsibility for each tiered autonomous vehicle to a respective vehicle monitor of the plurality of vehicle monitors based on the respective maximum workload of each vehicle monitor and the supervision level of the respective tiered autonomous vehicle, thereby defining a vehicle workload for each of the plurality of vehicle monitors; sensing a change in a supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor, wherein sensing the change in supervision level includes determining that the particular tiered autonomous vehicle is traveling on a rural highway with minimal traffic; as well as In response to sensing a change in the supervision level of the particular tiered autonomous vehicle, a vehicle workload associated with the particular vehicle monitor is re-examined.

9. A computer program product according to claim 8, wherein: Sensing a change in the supervision level of a particular tier-assigned autonomous vehicle assigned to a particular vehicle monitor includes: Sensing a particular class-assigned autonomous vehicle requires the full attention of the particular vehicle observer.

10. The computer program product according to claim 8, further comprising: If the vehicle workload associated with a particular vehicle monitor exceeds a defined level, the responsibility for some other hierarchically assigned autonomous vehicles assigned to the particular vehicle monitor is reallocated to another vehicle monitor.

11. A computer program product according to claim 10, wherein: Reassigning responsibility for one or more tiered autonomous vehicles assigned to a particular vehicle monitor to another vehicle monitor includes: Responsibility for all other tier-assigned autonomous vehicles assigned to a particular vehicle monitor is reassigned to another vehicle monitor.

12. The computer program product according to claim 8, wherein: The supervision level defined for each of the plurality of autonomous vehicles includes one or more category-based supervision levels.

13. The computer program product according to claim 8, wherein: The supervision level defined for each of the plurality of autonomous vehicles includes one or more score-based supervision levels.

14. The computer program product according to claim 8, wherein: The plurality of vehicle monitors include: Multiple human vehicle monitors.

15. A computing system comprising a processor and a memory, the system being configured to perform operations comprising: defining a supervision level for each of the plurality of autonomous vehicles, thereby defining a plurality of autonomous vehicles assigned by level; determining a respective maximum workload for each of a plurality of vehicle monitors; assigning responsibility for each tiered autonomous vehicle to a respective vehicle monitor of the plurality of vehicle monitors based on the respective maximum workload of each vehicle monitor and the supervision level of the respective tiered autonomous vehicle, thereby defining a vehicle workload for each of the plurality of vehicle monitors; sensing a change in a supervision level of a particular tiered autonomous vehicle assigned to a particular vehicle monitor, wherein sensing the change in supervision level includes determining that the particular tiered autonomous vehicle is traveling on a rural highway with minimal traffic; as well as In response to sensing a change in the supervision level of the particular tiered autonomous vehicle, a vehicle workload associated with the particular vehicle monitor is re-examined.

16. The computing system of claim 15, wherein: Sensing a change in the supervision level of a particular tier-assigned autonomous vehicle assigned to a particular vehicle monitor includes: Sensing a particular class-assigned autonomous vehicle requires the full attention of the particular vehicle observer.

17. The computing system of claim 15, further comprising: If the vehicle workload associated with a particular vehicle monitor exceeds a defined level, the responsibility for some other hierarchically assigned autonomous vehicles assigned to the particular vehicle monitor is reallocated to another vehicle monitor.

18. The computing system of claim 17, wherein: Reassigning responsibility for one or more tiered autonomous vehicles assigned to a particular vehicle monitor to another vehicle monitor includes: Responsibility for all other tier-assigned autonomous vehicles assigned to a particular vehicle monitor is reassigned to another vehicle monitor.

19. The computing system of claim 15, wherein: The supervision level defined for each of the plurality of autonomous vehicles includes one or more category-based supervision levels.

20. The computing system of claim 15, wherein: The supervision level defined for each of the plurality of autonomous vehicles includes one or more score-based supervision levels.

21. The computing system of claim 15, wherein: The plurality of vehicle monitors include: Multiple human vehicle monitors.

Images