Vehicle prioritization

Abstract

A method is provided. The method includes, for a respective vehicle of a plurality of vehicles: (i) determining an urgency measure associated with the vehicle, the urgency measure indicating an urgency of performing a task associated with the vehicle, and (ii) determining a length of time associated with performing the task of the vehicle, the task being performed by an operator of one or more operators, the one or more operators being located remotely from the plurality of vehicles and responsible for monitoring and / or controlling the plurality of vehicles. The method further includes determining a preferred order for the one or more operators to perform the tasks based on the length of time and the urgency measure associated with the plurality of vehicles.

Description

Background Technology

[0001] Autonomous and partially autonomous vehicles are being tested and used more and more frequently, not only for convenience but also to improve road safety. When an autonomous vehicle encounters problems or malfunctions, it may require the assistance of a human operator. Attached Figure Description

[0002] The accompanying drawings are used for detailed description. The same reference numerals are used in different drawings to indicate similar or identical parts or features.

[0003] Figure 1 The diagram is based on an example system that includes multiple vehicles and a remote system for monitoring the vehicles. Figure 2 Example tables depict urgency metrics and time durations associated with different vehicles and tasks; Figure 3 An example table is depicted, showing the time used to perform multiple tasks and the total time to perform the multiple tasks, taking into account the time required to switch between tasks. Figure 4 Example table depicting the sequential costs of performing tasks in multiple different orders; Figure 5 An example table depicting the sequence cost of multiple updated preferred orders; Figure 6A An example table is depicted showing how tasks are assigned to several operators; Figure 6B An example table depicts the assignment of tasks to an increasing number of operators; Figure 7 An example table depicting perception data generated by vehicles; Figure 8 A flowchart based on the example method is depicted; and Figure 9 This is a block diagram of an example vehicle system. Detailed Implementation

[0004] This application relates to systems, methods, and computer-readable media for prioritizing the order in which one or more human operators (also referred to as remote operators) should handle the needs of multiple autonomous vehicles when assistance is required. For example, two or more vehicles may require operator assistance simultaneously (e.g., within the same timeframe). The operator may be located remotely from the vehicles and can therefore monitor and / or control the vehicles from a remote system. In some scenarios, vehicles may request assistance from the operator, such as when a vehicle experiences a malfunction, an inter-vehicle accident, contact with an object, or some other problem, or when it is uncertain what to do next. Therefore, the operator may need to perform tasks for each vehicle, such as: remotely controlling the vehicle, providing guidance to the vehicle, resetting the vehicle, providing input to the vehicle when the vehicle's computing device requires confirmation from the operator, assigning maintenance personnel to handle the vehicle personally, allowing passengers to leave the vehicle (when it is safe to do so), and so on.

[0005] This disclosure relates to determining the order in which one or a limited number of operators handle the needs of a fleet of vehicles requiring assistance. Thus, specific vehicles may be prioritized over others. The order in which operators (or one or more operators) handle tasks (and therefore vehicles) may depend on one or more factors, such as the urgency of the task (which itself may depend on one or more factors) and the length of time associated with performing the task (i.e., the amount of time required for operators to handle vehicle requests). This disclosure relates to determining a preferred order in which operators should perform tasks based on a urgency metric associated with each vehicle / task and the length of time associated with performing the task. The urgency metric can be a value, such as an unbounded value or a value within a certain range, such as, for example, values ​​between 0 and 0.5, 0 and 1, or 0 and 100. A higher urgency metric for a vehicle may mean a more urgent task that vehicle needs to perform. The time length may be longer for more complex tasks (such as tasks requiring multiple operations / processes) and shorter for simpler tasks. Less time-consuming tasks may involve operators providing input, such as associating objects detected by vehicles with object classifications. Time-consuming tasks may involve navigating a vehicle from one location to another, such as navigating from a busy intersection to a quiet side street.

[0006] There can be multiple different sequences in which an operator can handle tasks. For example, if there are 3 vehicles that need assistance, there are 6 different sequences / arrangements of vehicles and tasks. In general, for N vehicles, there are N! sequences (N factorial sequences) in which tasks can be arranged. The sequence of tasks or vehicles can be alternatively referred to as an arrangement.

[0007] In the example, a "cost" (referred to as "sequence cost" in this document) can be determined for each sequence, where the sequence cost is based on a metric of urgency associated with the vehicle / task and the length of time associated with performing the task. The sequence cost can be a single value, such as a value within a certain range. A higher sequence cost may mean that the sequence is less desirable when compared to lower sequence costs. The preferred sequence (which may be the final chosen sequence) is likely the sequence with the lowest cost. As an example, a high-urgency, short-time task may be prioritized over a high-urgency, long-time task, while a low-urgency, short-time task may be prioritized over a low-urgency, long-time task.

[0008] Therefore, this disclosure relates to determining a preferred order that balances the needs of the vehicle and, in some cases, the passengers inside the vehicle (which may be represented by a “urgency measure”) with the operator’s overhead (which may be represented by the length of time associated with performing the vehicle task).

[0009] In the example, the urgency metric for each vehicle / task can be determined based on one or more factors, such as the probability of the vehicle colliding or coming into contact with objects in its vicinity. Therefore, the urgency metric associated with a vehicle can be based on a contact metric associated with the vehicle, where the contact metric indicates or is based on the probability of the vehicle colliding or coming into contact with one or more objects in its surrounding environment. The contact metric can be a single value, such as an unbounded value or a value within a certain range, such as values ​​between 0 and 0.5, 0 and 1, or 0 and 100. A higher contact metric means that the vehicle is more likely to collide or come into contact with one or more objects. For example, a first vehicle located in a busy city surrounded by vehicles and pedestrians can be associated with a higher contact metric compared to a second vehicle located on a rural road where there are fewer or no other vehicles or pedestrians nearby. In this case, the urgency metric associated with the first vehicle can be higher than that associated with the second vehicle (but in this example, the urgency metric can be based on other factors besides the contact metric). In the example, the contact metric can be referred to as the collision metric.

[0010] Therefore, in the examples described herein, a system is provided comprising: one or more processors; and one or more non-transitory computer-readable media having instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform operations including: (a) for a respective vehicle among a plurality of vehicles: determining or receiving a urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing a task associated with the vehicle; and (b) determining or receiving a duration associated with performing a task of the vehicle, the task being performed by an operator among one or more operators located remotely from the plurality of vehicles and responsible for monitoring and / or controlling the plurality of vehicles; and (c) determining a preferred order in which the one or more operators perform the task based on the duration associated with the plurality of vehicles and the urgency metric.

[0011] Multiple vehicles requiring operator assistance may be described as being “stuck.” In this example, the multiple vehicles may be stationary. However, in other cases, it should be understood that one or more vehicles may not necessarily be stationary (e.g., a vehicle may be traveling at a low speed and pulling over before waiting for assistance, or a vehicle may be traveling at a normal speed).

[0012] The urgency metric and time length for each of the multiple vehicles can be determined or received. That is, each vehicle / task can be associated with (i) the urgency metric and (ii) the time length. Therefore, steps (a) and (b) above can be repeated for each of the multiple vehicles. For example, a first urgency metric and a first time length for a first vehicle can be determined or received, and a second urgency metric and a second time length for a second vehicle can be determined or received. Then, a preferred order in which the operator performs tasks is determined based on the first and second time lengths and the first and second urgency metrics (step (c)). It should be understood that other vehicles may be present in other examples, and therefore the above steps can be repeated for other vehicles (such as third, fourth, and fifth vehicles). More generally, throughout this disclosure, any reference to performing any process or method step for a given vehicle, task, or sequence (or for each vehicle, task, or sequence) means that the process or method step can be repeated for each of the multiple vehicles in the same manner as discussed above.

[0013] A vehicle's task can be described as something associated with the vehicle. A task may involve an operator performing one or more operations / processes on the vehicle (even if the operator may perform multiple steps, these one or more operations / processes can be collectively referred to as a task). For example, the vehicle may have one or more malfunctions or problems. Therefore, the length of time associated with performing a task can be based on the length of time required to perform one or more operations / processes on the vehicle.

[0014] In the example, the time length can be determined based on the type of one or more faults in the vehicle. For example, a fault may be associated with a specific subsystem of the vehicle. Multiple faults may be associated with a single component, and / or mitigation actions may be shared between faults. In such cases, the time to handle two faults may be shorter than simply adding the two time lengths separately, each corresponding to one fault. Therefore, the time length for repairing multiple faults can be determined with more precise granularity based on the vehicle subsystem associated with the fault, the mitigation actions associated with the fault, historical information related to the time of resolving multiple faults, etc. This information can be stored in a tabular format, where each fault or combination of faults has a correction coefficient, and / or determined via calculation.

[0015] It should be understood that throughout this disclosure, any reference to the urgency measure in relation to a vehicle may or may alternatively imply that the urgency measure is associated with the vehicle’s mission.

[0016] In the example, the urgency measure may be based on a contact metric associated with the vehicle, which is based on the probability of the vehicle colliding or coming into contact with one or more objects in the vehicle's surrounding environment; unless otherwise stated, "based on" may mean that the term (in this case, the urgency measure) is "at least partially based on" one or more factors (in this case, the contact metric). The contact metric may also be referred to as the "vehicle contact metric".

[0017] An operator (or one or more operators) can use the system to monitor and / or control multiple vehicles. The system can be communicatively coupled to multiple vehicles, such as via one or more wired and / or wireless networks.

[0018] In the examples, the system can determine the urgency metric of a vehicle independently. For instance, the system can receive data from each vehicle, enabling it to determine that vehicle's urgency metric. In an example where the urgency metric is based on a contact metric, the vehicle can determine its contact metric independently and transmit it to the system (so the system can determine the vehicle's urgency metric based on the corresponding contact metric), or the system can determine the vehicle's contact metric based on data received from the vehicle. For example, each vehicle can transmit sensing data to the system (discussed below), which the system can use to determine the vehicle's contact metric. In another example, the vehicle can determine its urgency metric independently and transmit it to the system. In an example where the urgency metric is based on a contact metric, the vehicle can thus determine both the contact metric and the urgency metric. In these examples, the contact metric for each of multiple vehicles can be determined.

[0019] Similarly, in the example, the system can determine the length of time associated with performing a vehicle task. For example, the system can receive data from each vehicle, enabling it to determine the length of time for that vehicle. This data can indicate the task associated with the vehicle, such as the type of vehicle malfunction or problem. In other examples, the vehicle can determine the length of time itself and transmit it to the system. The length of time can be a predicted time for performing the task.

[0020] In the example, the urgency metric can be determined based on data available to the vehicle or system at a specific point in time. In some cases, the urgency metric for a vehicle can be updated over time. Thus, for example, a new, higher urgency metric can replace the previously determined urgency metric.

[0021] In the example, the urgency measure may be based solely on the contact measure. In this case, therefore, the urgency measure can be the contact measure.

[0022] As briefly mentioned above, perception data can be determined by each vehicle, where the perception data can be associated with one or more objects in the vehicle's surrounding environment. In the example, a vehicle (such as a vehicle's planning component) can use the perception data to navigate in its environment. The perception data can instruct the vehicle how to perceive its environment and can be based on sensor data obtained from one or more sensors on the vehicle. In the example, the perception data can be generated by the vehicle's perception component. The perception component can determine one or more characteristics for each detected object. For example, characteristics can include any or all of the following: classification (or object type), location, size, speed, etc. Thus, the perception component can classify objects. Example classifications could include vehicles, pedestrians, buildings, road surfaces, cyclists, trees, traffic lights, etc. The perception data can include one or more characteristics associated with each object. Location can be the object's current location and / or predicted or future location. Therefore, the perception data can additionally include prediction data, where the prediction data is generated by the vehicle's prediction component. The prediction component can generate probabilistic data, such as one or more probability maps or one or more future trajectories, which represent the predicted probability of the possible future locations of one or more objects in the environment. In the example, the perception data can be used to determine an object's contact metric (referred to herein as the "object contact metric"). For example, object contact metrics can be based on predicted data. A vehicle's contact metric (i.e., "vehicle contact metric") can be based on one or more object contact metrics. For instance, a vehicle contact metric could be the sum, average, or weighted average of object contact metrics for one or more objects in the environment surrounding the vehicle. In another example, a vehicle contact metric could be the largest object contact metric among multiple object contact metrics.

[0023] In the example, a single operator can perform tasks for multiple vehicles. In other cases, multiple operators can perform the tasks. In the example, some operators may be assigned to handle certain types of vehicles, such as general vehicles, while others may be assigned to handle different types of vehicles, such as emergency vehicles. In some cases, depending on the nature of the task, multiple operators may be assigned to the same vehicle / task. For example, one operator may handle one or more processes / operations of a vehicle, while another operator may handle one or more other processes / operations of the vehicle. In some examples, each operator may need to handle a corresponding vehicle queue. The disclosed techniques can be used to assign vehicles to these queues and / or to sort vehicles in the queues.

[0024] In the example, one or more processors can also enable the execution of tasks based on one or more inputs from one or more operators. For example, the tasks can be performed sequentially by a single operator, or at least some tasks can be performed simultaneously by two or more operators. Enabling the execution of tasks can include sending data to multiple vehicles to cause the vehicles to perform actions.

[0025] The system may include one or more workstations or terminals, each operated by an operator. Workstations / terminals may be located in the same location or geographically dispersed (and away from multiple vehicles). Therefore, the system may be referred to as a remote system.

[0026] It should be understood that when determining the preferred order, there may be additional vehicles (i.e., vehicles beyond the initial number of vehicles) that do not require operator assistance. Therefore, urgency metrics and / or time durations can be determined or received only for the vehicles requiring assistance. In some examples, there may be an upper limit or limitation on the number of vehicles each operator can handle. For example, if 10 vehicles require assistance, and the operator is limited to handling 5, then the initial number of vehicles may consist of only 5. In other examples, the upper limit or limitation may be based on time duration. For example, the upper limit or limitation on the number of vehicles may be based on the total time spent performing all tasks. If the tasks are short, the upper limit / limit can be higher. In the examples, operators may monitor / control vehicles in a specific area. In some cases, if the number of vehicles requiring assistance increases, one or more additional operators may be needed to assist some of the vehicles.

[0027] In the example, determining the preferred order in which one or more operators should perform tasks may include: (A) determining multiple different orders in which one or more operators can perform tasks associated with multiple vehicles; and (B) for a given order among the multiple different orders, determining a sequence cost associated with the order, the sequence cost being based on: (i) a urgency measure associated with the multiple vehicles, (ii) the length of time associated with performing the tasks, and (iii) the order in which the tasks are to be performed; and (C) selecting a preferred order from the multiple different orders based on the sequence cost. The order in which tasks are performed may relate to the position of the tasks within the sequence (e.g., a task is performed first, second, last, etc.).

[0028] Therefore, the sequence cost can be determined for each of the multiple different sequences.

[0029] In the example, selecting a preferred order from multiple different orders based on order cost may include: selecting a preferred order from multiple different orders, the preferred order being associated with the lowest order cost.

[0030] In the example, after the preferred order has been determined, additional vehicles may require assistance (i.e., in addition to the original multiple vehicles associated with the preferred task order). Therefore, there are additional tasks (associated with the additional vehicles) that the operator must perform. In this case, instead of repeating the above process for the updated multiple vehicles (which includes the original multiple vehicles plus the additional vehicles), it is computationally more efficient to add the additional tasks / vehicles to specific locations in the preferred order. For example, if there are 3 vehicles in the preferred order (i.e., the original multiple vehicles include 3 vehicles), and now a fourth vehicle needs assistance, there are 4 different locations where the fourth vehicle can be added to the preferred order (first, second, third, or fourth). In the example, the order costs of these 4 updated preferred orders (referred to herein as multiple updated preferred orders) can be determined, and then the order cost with the lowest order cost can be selected.

[0031] The selected order can be referred to as the updated preferred order, and is therefore chosen from multiple updated preferred orders. Each of the multiple updated preferred orders has tasks in the same order as in the preferred order, but also includes additional tasks at specific positions within the preferred order. For example, if the preferred order has 3 tasks in the following order: task #2 (associated with vehicle 2), task #1 (associated with vehicle 1), and task #3 (associated with vehicle 3), then an example updated preferred order could have 4 tasks in the following order: task #2 (associated with vehicle 2), task #4 (associated with vehicle 4), task #1 (associated with vehicle 1), and task #3 (associated with vehicle 3). The original tasks can remain in the same order (relative to each other), while additional tasks are included at specific positions within the preferred order. It is this specific position that needs to be determined. In some cases, the position of tasks within the order may change. For example, tasks 1 and 4 may now be positioned third and fourth within the order, instead of second and third.

[0032] Therefore, in the example, after selecting the preferred order, the operation may also include: (i) determining the additional tasks associated with the additional vehicle that need to be performed by one or more operators, and (ii) determining the position of the additional tasks in the preferred order.

[0033] In the example, determining the execution position of the additional task in the preferred order may include: determining the execution position of the additional task in the preferred order based on the urgency measure associated with the additional vehicle and / or the length of time associated with performing the additional task.

[0034] Continuing with the previous example, the updated preferred order may not be the order with the lowest total order cost. However, it is computationally more efficient to determine the order cost associated with the 4 updated preferred orders than to determine the order cost associated with 24 (4 factorial) different orders, where the original multiple vehicles may be ordered in these different orders if additional vehicles are included.

[0035] Therefore, in the example, determining the execution position of the additional task in the preferred order may include: determining an updated preferred order in which the additional task is located at a specific position within the preferred order. Determining the updated preferred order may include: (A) determining a plurality of updated preferred orders that arrange tasks associated with multiple vehicles in the same order as in the preferred order, and arrange the additional task associated with the additional vehicle at different positions in the preferred order; (B) for a corresponding order in the plurality of updated preferred orders, determining a sequence cost associated with that order based on: (i) a urgency metric associated with the multiple vehicles and a urgency metric associated with the additional vehicle, (ii) a time length associated with executing the task and a time length associated with executing the additional task, and (iii) the execution order of the tasks in the updated preferred order; and (C) selecting the updated preferred order from the plurality of updated preferred orders based on the sequence cost. Therefore, a sequence cost for each of the plurality of updated preferred orders can be determined.

[0036] In the example, the selected updated preferred order can be the updated preferred order with the lowest order cost.

[0037] In the example, the number of updated preferred orders (within multiple updated preferred orders) is equal to the original number of tasks (equal to the number of vehicles) plus one (for new additional tasks and vehicles). Returning to the example above, there are 4 (3+1) possible updated preferred orders.

[0038] In the example, determining that additional tasks associated with an additional vehicle need to be performed may include determining that additional tasks associated with an additional vehicle need to be performed within a threshold time period. The threshold time period can be based on or after the preferred order has been determined. As an example, the threshold time period could be 30 seconds. Therefore, in some cases, if the additional vehicle needs assistance within the threshold time period (such as if the vehicle requests assistance within 30 seconds after the system determines the preferred order), the additional vehicle can be added to the multiple tasks to be performed by the operator. If the vehicle needs assistance outside the threshold time period (such as if the vehicle requests assistance within 45 seconds after the system determines the preferred order), the additional vehicle can be ignored (e.g., not added to the multiple tasks to be performed by the operator). The additional vehicle can then be handled by another operator, or by the same operator at a later point in time.

[0039] As mentioned above, sequence costs can be based on the execution order of tasks (that is, the position of tasks during execution). In practice, this can be considered by determining the time (from each vehicle's perspective) taken from the moment the operator begins executing the first task until the task for each specific vehicle is completed, for any task to be finished / resolved. The time taken to resolve a task (from the vehicle's perspective) can be called the vehicle's resolution time. If a task is later in the sequence, it will take longer to resolve because the resolution time is based on the cumulative time length associated with previous tasks in the execution sequence. For the first vehicle in the sequence, since there are no previous tasks to execute, the resolution time can be equal to the time length associated with executing the task for the first vehicle. For the second vehicle in the sequence, the resolution time will be based on the time length associated with executing the task for the second vehicle and the time length associated with executing the task for the first vehicle. More generally, for any vehicle / task, the resolution time for that vehicle / task will be based on the time length associated with executing the task for that vehicle and the cumulative time length associated with previous tasks in the execution sequence.

[0040] In the example, the resolution time associated with a specific task / vehicle can be a time period, such as the time between when the operator begins executing the task that is first in the sequence and when the operator completes the task associated with that vehicle.

[0041] Therefore, in the example, for a given sequence among multiple different sequences, determining the sequence cost associated with that sequence may include: for a given task in the sequence, (A) determining the resolution time based on: (i) the length of time associated with performing the task and (ii) the cumulative length of time associated with performing previous tasks in the sequence; and (B) determining the task cost based on: (i) the resolution time and (ii) a urgency measure associated with the vehicle. The sequence cost associated with the sequence may be based on the sum of the task costs of the sequence.

[0042] The resolution time and task cost of each task in a sequence can be determined. Therefore, the sequence cost of each sequence in multiple sequences is based on task costs. As an example, if there are 3 tasks to be performed (associated with 3 different vehicles), the costs of 3 tasks can be determined. In the example, the sequence cost associated with a sequence can be the sum of the task costs of that sequence. For example, the sequence cost can be the sum of the costs of these 3 tasks. Since task costs are based on resolution time (which in turn is based on the task's position within the sequence), the task cost is based on the task's position within the sequence.

[0043] In the example, the task cost associated with a vehicle task is based on or equal to the resolution time multiplied by the urgency measure associated with that task / vehicle.

[0044] In the example, the resolution time can also be based on at least one task switching time, which indicates the time an operator spends switching from one task to another. For example, if an operator needs to perform three tasks, the operator may need time to switch between tasks. For instance, if the operator spends 10 seconds switching between tasks, the resolution time for the second vehicle / task may additionally include one task switching time (equal to 10 seconds in this example), while the resolution time for the third vehicle / task may additionally include two switching times (equal to 20 seconds in this example, because the operator needs to switch tasks twice). In the example, the task switching time can also be based on the type of task being switched.

[0045] Similarly, when additional tasks need to be performed, the sequence cost of each of the updated preferred sequences can be determined. Therefore, in the example, for a given sequence among multiple updated preferred sequences, determining the sequence cost associated with that sequence can include: for the corresponding task in the sequence, (A) determining the resolution time based on (i) the length of time associated with performing the task and (ii) the cumulative length of time associated with performing previous tasks in the sequence; and (B) determining the task cost based on (i) the resolution time and (ii) a urgency measure associated with the vehicle. The sequence cost associated with the sequence can be based on the sum of the task costs of the sequence.

[0046] In addition to urgency measures being based on contact measures, or instead of urgency measures being based on contact measures, urgency measures can also be based on one or more other factors. Therefore, in the example, the urgency measure associated with the vehicle can be based on one or more of the following: one or more characteristics of one or more passengers in the vehicle, traffic conditions, the location of the vehicle, the time of day, or the type of fault associated with the vehicle.

[0047] The characteristics of one or more passengers in a vehicle may include: the number of passengers in the vehicle or the demographic information of one or more passengers. For example, demographic information may include whether the passengers are children, elderly people, or people with disabilities.

[0048] As an example, a larger number of passengers in a vehicle can lead to a higher urgency metric when the number of passengers is smaller. Therefore, prioritizing vehicles with larger passenger capacities may be more useful.

[0049] In the example, urgency metrics can be based on traffic conditions such as: the expected impact of vehicles on traffic patterns, the danger of traffic to passengers, the danger to people outside the vehicle, and traffic conditions at a specific time of day. Traffic conditions can take into account the vehicle's current location. Traffic conditions can be determined based on perceived data, historical data, and / or data from third parties.

[0050] In the examples, the type of malfunction associated with the vehicle (which may correspond to the type of task to be performed) may require a rescue operation (i.e., an operator may need to take control of the vehicle to allow passengers to exit from the vehicle in a safe location). In some examples, the vehicle may require manual handling by personnel to control it manually or perform maintenance. Therefore, the urgency metric can be based on the type of malfunction associated with the vehicle. In the examples, the complexity and / or duration of the rescue operation may be considered (e.g., rescue operations with more preparation time may need to start earlier, which may result in a higher urgency metric). In the examples, the malfunction type may pose a danger to occupants or the public. For example, a system malfunction may be flagged as potentially causing harm in the form of releasing gas or liquid. Therefore, a more dangerous malfunction may result in a higher urgency metric.

[0051] In the example where the urgency metric is based on vehicle position, the urgency metric can be based on one or more of the following: the vehicle's position relative to an intersection, such as a crossroads, or other road features (such as railroad tracks); the vehicle's position relative to traffic; the vehicle's position relative to a safe exit location (e.g., when the complexity and / or duration of a task requires passengers to exit the vehicle); the speed limit on the road where the vehicle is located; the lane the vehicle is in; or the number of lanes traveling in the same direction as the vehicle. For example, if the vehicle is currently in the middle of an intersection, the urgency metric may be higher than if the vehicle is on a rural road. If the vehicle is on a road with two or more lanes for traveling in the same direction as the vehicle, the urgency metric may be lower than if the vehicle is on a road with only one lane for traveling in the same direction as the vehicle (because the vehicle does not obstruct traffic along the road). Therefore, more lanes result in a lower urgency metric. If the vehicle is in the overtaking lane (such as the lane to the left of the inside lane), the urgency metric may be higher than if the vehicle is in the inside lane (because the overtaking lane can be considered more dangerous). In the example, the further to the left the vehicle is, the higher the urgency metric. The urgency metric may be higher if the vehicle is on a road with a relatively high speed limit (because that road may be considered more dangerous). Similarly, the urgency metric may be higher if the vehicle is far from a safe exit point than if it is closer to that exit point.

[0052] In this example, multiple operators can perform tasks associated with multiple vehicles, rather than a single operator. In this case, there may be N vehicles and N tasks, and M operators to perform tasks associated with multiple vehicles, where N > M. Operations may include: assigning the first M tasks in a preferred order to the M operators (such that each operator is assigned one task), and assigning the remaining tasks in the preferred order to the M operators based on each operator's total resolution time, which is based on the cumulative time associated with performing the task currently assigned to an operator. The remaining tasks can be assigned sequentially. In some examples, there may be an upper limit on the number of tasks / vehicles assigned to operators and / or the total resolution time of operators. When the upper limit is reached, no further tasks / vehicles are assigned to operators. This can alleviate the operator's load or burden. In some examples, tasks can be allocated to available operators based on a predicted total resolution time to avoid overloading operators. In some examples, an idle window is available when operators complete tasks earlier than predicted and / or are available, and corresponding tasks can be assigned, where the total resolution time is within the idle window.

[0053] In the example, each vehicle can alert the operator that assistance is needed (i.e., a task needs to be performed). Therefore, in this example, the operation could also include: receiving data from the appropriate vehicle among multiple vehicles at the system, and determining, based on the data, that the appropriate vehicle among the multiple vehicles requires the operator to perform a task. In this example, the data could be an urgency measure, and the remote system could infer the need to perform a task from the received urgency measure. In other examples, the data could include a help request, or it could be an indication that a vehicle is malfunctioning, and so on.

[0054] In the example, there may be multiple processes / operations that a vehicle needs to perform. In some cases, the nature of one process / operation of a vehicle may be that it is less important than another process / operation of the vehicle, such that it can be handled by the operator at a later time. For example, if both vehicles need assistance, the operator may handle the primary process or operation of the first vehicle, then handle the second vehicle, and then return to handle the secondary process or operation of the first vehicle. In this case, the urgency measure and / or time length of each process / operation can be determined, rather than the urgency measure of the vehicle itself or anything other than the urgency measure of the vehicle itself. In the same manner as discussed above, the preferred order in which these processes / operations should be performed can be determined based on the urgency measure and / or time length. Therefore, this disclosure can be extended to scenarios where, in the example, the urgency measure of the vehicle may be based on one or more urgency measures of one or more processes / operations.

[0055] According to a second aspect of this disclosure, a method is provided, the method comprising: (A) for a respective vehicle among a plurality of vehicles: (i) determining an urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing a task associated with the vehicle, and (ii) determining a time length associated with performing the task of the vehicle, the task being performed by an operator among one or more operators located remotely from the plurality of vehicles and responsible for monitoring and / or controlling the plurality of vehicles; and (B) determining a preferred order in which the one or more operators perform the task based on the time length associated with the plurality of vehicles and the urgency metric.

[0056] In the example, the above method steps can be performed by a system located remotely from multiple vehicles. In other examples, some method steps can be performed by the system while others can be performed by the vehicles. For example, as previously discussed, the system or vehicle can determine either or both of the following: (i) a measure of urgency and / or (ii) the length of time. The system can then determine the steps for determining the preferred order.

[0057] According to a third aspect of this disclosure, one or more non-transitory computer-readable media are provided, which store instructions that, when executed by one or more processors of a system, cause the system to perform operations including: for a respective vehicle among a plurality of vehicles: determining or receiving an urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing a task associated with the vehicle; and determining or receiving a duration associated with performing the task of the vehicle, the task being performed by one or more operators located remotely from the plurality of vehicles and responsible for monitoring and / or controlling the plurality of vehicles; and determining a preferred order in which the one or more operators perform the task based on the duration associated with the plurality of vehicles and the urgency metric.

[0058] More detailed examples of this disclosure, as well as systems, methods, and computer-readable media, will now be presented with reference to the accompanying drawings.

[0059] Figure 1 An example implementation of this disclosure is described. Specifically, Figure 1 An example system 100 is depicted, located remotely from multiple autonomous vehicles. In this example, three vehicles are present (including a first vehicle 112a, a second vehicle 112b, and a third vehicle 112c). In other examples, a different number of vehicles may be present.

[0060] Each vehicle 112a, 112b, and 112c is located in an environment and can autonomously navigate within that environment based on data obtained from sensors on vehicles 112a, 112b, and 112c. In this example, the first vehicle 112a is located in a certain area of ​​a town or city, the second vehicle 112b is located in another area of ​​a town or city (in this case, at an intersection), and the third vehicle 112c is located in a certain area of ​​a rural area. In this example, vehicles 112a, 112b, and 112c are far apart from each other, but in some cases, vehicles 112a, 112b, and 112c may be close to each other, such as within the same city or within a specific area or region of a city.

[0061] As shown, remote system 100 can be communicatively coupled to multiple vehicles 112a, 112b, 112c via one or more networks 114. Data can be transmitted between remote system 100 and any of vehicles 112a, 112b, 112c via network 114. In this example, data can be transmitted between vehicles 112. Therefore, each such vehicle may include one or more network interfaces (…). Figure 9 (As shown in the diagram) to enable the transmission of data to and reception of data from the remote system 100. The remote system 100 may also include one or more network interfaces.

[0062] System 100 includes one or more processors 102 and one or more non-transitory computer-readable media 104, on which instructions are stored, which, when executed by the one or more processors 102, cause the one or more processors 102 to perform a specific operation, which will be discussed in more detail below. It should be understood that any reference to the remote system 100 performing an action / operation should be construed as being performed by one or more processors 102 of system 100.

[0063] Human operator 108 can monitor and / or control one or more vehicles 112a, 112b, 112c via system 100. For example, operator 108 can observe data received from vehicles 112a, 112b, 112c and output to one or more displays 106 of system 100. For example, operator 108 can view one or more video feeds from one or more cameras on vehicle 112. In some examples, a 2D or 3D model of the environment surrounding vehicles 112a, 112b, 112c can be rendered on display 106, where the model includes representations of one or more objects around vehicle 112. Such a model can be generated based on perception data generated by each vehicle (discussed in more detail below). Thus, operator 108 can be able to monitor one or more vehicles 112a, 112b, 112c in this way. In some cases, operator 108 can provide instructions such as driving or navigation commands to vehicles 112a, 112b, 112c to remotely guide vehicle 112 when needed.

[0064] In the example, a single operator 108 can monitor multiple vehicles 112a, 112b, 112c, and / or can be responsible for multiple vehicles 112a, 112b, 112c when assistance is needed. In other examples, two or more operators can monitor multiple vehicles 112a, 112b, 112c, and / or can be responsible for multiple vehicles 112a, 112b, 112c when assistance is needed.

[0065] System 100 may also include one or more input devices 110 to receive user input from operator 108. For example, user input may cause instructions to be transmitted via network 114 to vehicles 112a, 112b, 112c, causing vehicles 112a, 112b, 112c to perform actions such as navigating the environment along a specific navigation path, troubleshooting or resolving errors, resetting or restarting the computing systems / devices of vehicles 112a, 112b, 112c, etc. In this example, one or more displays 106 are themselves input devices 110. For example, one or more displays 106 may include a touchscreen display 106 that can accept user input.

[0066] Each vehicle 112a, 112b, 112c may also include one or more processors. Figure 9 (as shown) and one or more non-transitory computer-readable media (as shown) Figure 9 As shown in the diagram, the one or more non-transitory computer-readable media have instructions stored thereon that, when executed by one or more processors, cause one or more processors to perform a specific operation, which will be discussed in more detail below. It should be understood that any reference to the performance of operations / actions by vehicles 112a, 112b, 112c should be construed as performance by one or more processors of vehicle 112.

[0067] As briefly mentioned, each vehicle 112a, 112b, 112c may include one or more sensors that acquire sensor data associated with a corresponding environment. Sensors may include ultrasonic sensors (for acoustically detecting objects in the surrounding environment), lidar sensors, radar sensors, infrared sensors, etc. The sensor data captured by one or more sensors of vehicle 112 may be processed by vehicles 112a, 112b, 112c (such as by a sensing component of vehicle 112) to generate perception data for vehicle 112. For example, data from one or more sensors may be fused or otherwise modified to generate perception data representing one or more objects in the environment surrounding vehicle 112. The perception data may be used by the planning components of vehicles 112a, 112b, 112c (see reference). Figure 9 (Discussed in more detail), the planning component determines instructions for controlling the operation of vehicles 112a, 112b, and 112c based on the sensing output data. In the example, the sensing data associated with each vehicle 112a, 112b, and 112c can be sent or otherwise transmitted to the remote system 100.

[0068] Each vehicle 112a, 112b, and 112c can also determine prediction data, which is generated by the prediction component of vehicles 112a, 112b, and 112c. Figure 9 (Discussed in more detail below). Predictive data can form part of the perceived data. In the example, the predictive data can be sent via network 114 or otherwise transmitted to remote system 100. For example, the predictive data can be transmitted to system 100 as part of the perceived data.

[0069] Stuck vehicle As mentioned, at times, multiple vehicles may require operator assistance simultaneously. For example, Figure 1 The three vehicles 112a, 112b, and 112c may all require the assistance of operator 108 simultaneously (e.g., within the same timeframe). Therefore, each vehicle can transmit data to system 100, and system 100 can determine based on that data that vehicles 112a, 112b, and 112c require assistance. In certain situations, when requesting assistance from operator 108, multiple vehicles 112a, 112b, and 112c may all be stationary (as an example of being "stuck"). For example, vehicles 112a, 112b, and 112c may be unable to move (or movement may be considered unsafe) before receiving input from operator 108.

[0070] exist Figure 1 In the example scenario, vehicles 112a, 112b, and 112c all require assistance from operator 108. For instance, vehicles 112a, 112b, and 112c may have experienced a malfunction, contact with an object, or some other problem, or they may be unsure of the next step. Therefore, operator 108 may need to perform tasks for each vehicle, such as: remotely controlling the vehicle, resetting the vehicle, providing input to the vehicle's computing device when operator confirmation is required, assigning maintenance personnel to handle the vehicle personally, allowing passengers to leave the vehicle (when it is safe to do so), and so on.

[0071] When multiple vehicles require assistance simultaneously, prioritizing the order in which operator 108 processes vehicles 112a, 112b, and 112c based on specific criteria can be useful. In this disclosure, system 100 determines the preferred order of processing vehicles 112a, 112b, and 112c (and the task) based on an urgency metric determined for each vehicle and the length of time required for each task associated with the task. Reference will now be made to... Figure 1 The example is described using multiple vehicles 112a, 112b, and 112c. It should be understood that although the following description involves three vehicles, the same principles can be applied or extended to any number of vehicles, such as two or more.

[0072] As mentioned, system 100 is configured to determine the preferred order in which operator 108 should perform tasks for multiple vehicles 112a, 112b, 112c based on the time length associated with the task and the urgency metric associated with the multiple vehicles.

[0073] In this example scenario, system 100 determines (or receives from the vehicles 112) an urgency metric for each of the plurality of vehicles 112a, 112b, 112c. For example, the urgency metric may take a value between 0 and 0.5. As discussed, in this example, the urgency metric for vehicles 112a, 112b, 112c may be based on one or more factors, such as the location of vehicles 112a, 112b, 112c (referred to herein as the “location metric” and having a value between 0 and 0.5), the number of passengers in vehicles 112a, 112b, 112c (referred to herein as the “passenger metric” and having a value between 0 and 0.5), and the vehicle contact metric (referred to herein as the “vehicle contact metric” and having a value between 0 and 0.5).

[0074] Figure 2 The table in the document depicts an example where the urgency measure 202 is based on these three factors; however, it should be understood that in other examples, the urgency measure may be based on fewer or more factors. In some cases, the urgency measure may be based entirely on the vehicle contact measure, while in others, it may not be based on the vehicle contact measure. Figure 2 In the example, the urgency metric 202 for vehicles 112a, 112b, and 112c is the average (arithmetic mean) of the vehicle's location metric 204, the vehicle's passenger metric 206, and the vehicle's vehicle contact metric 208. In other examples, the urgency metric 202 may be determined in another way, such as a weighted average or sum of metrics 204, 206, and 208, or the maximum value of metrics 204, 206, and 208. More generally, the urgency metric 202 for vehicles 112a, 112b, and 112c may be based on the vehicle's location metric 204, the vehicle's passenger metric 206, and the vehicle's contact metric 208. In a specific example, the urgency metric 202 may be a weighted average of metrics 204, 206, and 208, where a higher weight is assigned to the contact metric 208.

[0075] like Figure 2As shown, the first vehicle 112a in this example is located on a main road, where there is only one lane in the driving direction of the vehicle. This results in a position metric 204 of 0.3 for this vehicle 112a (indicating the position of the vehicle 112a). The number of passengers in the first vehicle 112a is 2 passengers (e.g., at most 5 passengers may be possible), and this results in a passenger metric 206 of 0.2. The objects around the first vehicle 112a include several other vehicles (some of which are stationary) and pedestrians, and this leads to a vehicle contact metric 208 of 0.25. A variety of different methods and schemes can be used to determine the metrics 204, 206, 208.

[0076] As Figure 2 Further shown, the second vehicle 112b in this example is located at an intersection, where there is only one lane in the driving direction of the vehicle. This results in a position metric 204 of 0.3. The number of passengers in the second vehicle 112b is 3 passengers (at most 5 passengers may be possible), and this results in a passenger metric 206 of 0.3. The objects around the second vehicle 112b include several other vehicles and pedestrians, and this leads to a vehicle contact metric 208 of 0.4. Thus, for example, compared with the vehicle contact metric of the first vehicle 112a, the vehicle contact metric of the second vehicle 112b is higher, which means that the second vehicle 112b is more likely to collide with or contact an object than the first vehicle 112a.

[0077] As Figure 2 Further shown, the third vehicle 112c in this example is on a rural road, where there are two lanes in the driving direction of the vehicle. This results in a position metric 204 of 0.1. The number of passengers in the third vehicle 112c is 1 passenger (at most 5 passengers may be possible), and this results in a passenger metric 206 of 0.1. There are no other vehicles or pedestrians around the third vehicle 112c, which leads to a vehicle contact metric 208 of 0.1. Thus, for example, compared with the vehicle contact metrics of the first vehicle 112a and the second vehicle 112b, the vehicle contact metric of the third vehicle 112c is lower, which means that the third vehicle 112c is less likely to collide with or contact an object than the first vehicle 112a and the second vehicle 112b.

[0078] Therefore, based on the urgency metric 202, it can be inferred that: the task associated with the second vehicle 112b is the most urgent because it has the highest urgency metric; and the task associated with the third vehicle 112c is the least urgent because it has the lowest urgency metric.

[0079] Figure 2 The table in also includes the length of time associated with each of the tasks of the vehicle 112. The length of time taken to complete the task of vehicle n (where 0 < n ≤ N, and N is the total number of tasks (and vehicles)) is: T任务_n For example, due to the nature of the task and work involved, processing the first vehicle 112a (and therefore the task #1 associated with the first vehicle 112a) could take the operator 180 seconds (T). 任务_1 = 180), processing the second vehicle 112b (and therefore the associated task #2) takes the operator 120 seconds (T) 任务_2 = 120), and processing the third vehicle 112c (and therefore the task #3 associated with the third vehicle 112c) can take the operator 60 seconds (T 任务_3 = 60). Figure 3 The table shows that the total time for the operator to perform the task was 360 seconds (180+120+60).

[0080] As mentioned earlier, this can also take the operator a certain amount of time (referred to as task switching time, T). 切换任务 Switch / move between tasks. In this example, T 切换任务 = 10 seconds. For example, it might take an operator 10 seconds to navigate from one task in one vehicle to another task in another vehicle using a user interface displayed on one or more monitors 106. In this example, with three tasks, the operator would need to switch tasks twice (from the first task to the second task, and from the second task to the third task). The total time T for the operator to complete the task. 总 (Including the time spent switching between tasks) is given by the following formula: Seconds, or as Figure 3 As shown.

[0081] Therefore, based on the time length 210, the task associated with the first vehicle 112a takes the longest execution time for the operator, while the task associated with the third vehicle 112c takes the shortest execution time for the operator.

[0082] Preferred order As discussed earlier, tasks associated with multiple vehicles can be executed in a variety of different orders. The total number of different orders is given by N!, where N is the number of tasks / vehicles. In this example, there are 6 (N! = 3! = 3 * 2 * 1 = 6) different orders in which the operator can process the tasks. Figure 4Shows how tasks / vehicles are arranged in each of a plurality of different orders 402. For example, for order #1 (also referred to as arrangement #1), the tasks can be sorted as follows: task #1 (associated with the first vehicle 112a), followed by task #2 (associated with the second vehicle 112b), followed by task #3 (associated with the third vehicle 112c). If the operator 108 is to perform the tasks according to order #1, the operator will perform the tasks sequentially and in that order.

[0083] To determine which of the plurality of different orders 402 is the preferred order (i.e., the order in which the operator 108 should process the vehicles and tasks), an order cost 404 can be determined for each of the plurality of different orders 402. As mentioned, the order cost 404 can be based on: (i) an urgency metric 202 associated with the plurality of vehicles 112, (ii) the length of time 210 associated with performing the tasks, and (iii) the order in which the tasks are performed.

[0084] In an example, the order cost for order i (where i is the order number and 0 < i ≤ N!) can be determined as follows: , for all j, 0 < j ≤ N, where is the task cost associated with the task at position j.

[0085] For example, for Figure 4 the order #2 (i = 2) shown in .

[0086] For example, in order #2, is the task cost of the task associated with the second vehicle 112b because task #2 (i.e., vehicle #2) is at position #3 in that order.

[0087] In addition, the task cost associated with each task / vehicle is based on the urgency metric associated with the vehicle / task and the resolution time of the task at position m in that order

[0088] The resolution time is the time taken to resolve the task (from the perspective of the vehicle) and is based on: (i) the length of time associated with performing the task, and (ii) the cumulative length of time associated with performing the previous tasks in order. Thus, the resolution time of a task depends on its position in the order in which the tasks are performed. The resolution time can be given by: , for all i, 0 < j ≤ m.

[0089] Therefore, the task cost of the vehicle / task at location m can be given by the following formula: .

[0090] in It is a measure of urgency associated with the vehicle at location m.

[0091] As an example, in sequence #2, the task cost of task #1 (vehicle #1, i.e., the first vehicle 112a) at position #1 in this sequence can be given by: 0.25*((1-1)*10+(180)) = 45, the task cost of task #3 (vehicle #3, i.e., the third vehicle 112c) at position #2 in this sequence can be given by: 0.1*((2-1)*10+(180+60)) = 25, and the task cost of task #2 (vehicle #2, i.e., the second vehicle 112b) at position #3 in this sequence can be given by: 0.33*((3-1)*10+(180+60+120)) = 125.4. Therefore, the sequence cost associated with sequence #2 can be given by: 45 + 25 + 125.4.

[0092] The order costs of other orders among multiple different orders can be determined in the same way.

[0093] The preferred order can be the order with the lowest sequential cost. In this particular example, the order with the lowest sequential cost is order #3, which means that the operator processes the second vehicle 112b first, then the third vehicle 112c, and then the first vehicle 112a.

[0094] Updated preferred order In the example, after the preferred order has been determined, additional vehicles may require assistance (that is, in addition to the original vehicles 112a, 112b, 112c associated with the preferred task order). Therefore, there are additional tasks associated with the additional vehicles that operator 108 needs to perform. In this case, instead of repeating the above process for the updated vehicles (which include the original vehicles plus the additional vehicles), it is more computationally efficient to add the additional task / vehicle to a specific position in the previously determined preferred order. Therefore, it is necessary to determine at what position in the preferred order to perform this additional task.

[0095] Continuing with the example above, in addition to the three tasks already required, operator 108 needs to perform a fourth vehicle and a fourth task (vehicle #4 and task #4). As discussed above, the fourth vehicle and task can also be associated with urgency metrics and time duration. In this example, there are four different locations / positions where the fourth task / vehicle can be added to the preferred order. Figure 5 Different sequences 504 of executable tasks are shown. Each of these sequences can be referred to as an updated preferred sequence, and thus there are multiple updated preferred sequences 502.

[0096] like Figure 5 As shown, each of the multiple updated preferred sequences 502 has tasks in the same order as the preferred sequence, and also includes additional tasks at specific positions within the preferred sequence. In this example, the preferred sequence has three tasks in the following order: Task #2 (associated with vehicle #2), Task #3 (associated with vehicle #3), and Task #1 (associated with vehicle #1). As an example, the second updated preferred sequence has four tasks in the following order: Task #2 (associated with vehicle #2), Task #4 (associated with vehicle #4), Task #3 (associated with vehicle #3), and Task #1 (associated with vehicle #1). The original tasks (relative to each other) maintain the same order, and additional tasks are included at specific positions within the preferred sequence. The same applies to the other updated preferred sequences.

[0097] Determining the execution position of additional tasks in the preferred order may include determining multiple updated preferred orders 502, wherein the multiple updated preferred orders 502 arrange tasks associated with multiple vehicles 112a, 112b, 112c in the same order as in the preferred order, and arrange additional tasks associated with additional vehicles at different positions in the preferred order.

[0098] Operator 108 can select one of several updated preferred sequences. The selected sequence can be referred to as the updated preferred sequence (or the selected updated preferred sequence). In the example, the sequence costs of several updated preferred sequences 502 can be determined, and then the sequence cost with the lowest sequence cost can be selected.

[0099] Therefore, in the same manner as discussed above, the sequence cost of each of the multiple updated sequences 502 can be determined. The sequence cost may be based on: (i) urgency metrics associated with the multiple vehicles 112a, 112b, 112c and urgency metrics associated with additional vehicles; (ii) the time length associated with performing the task and the time length associated with performing the additional task; and (iii) the execution order of the tasks in the updated preferred sequence.

[0100] In the example, the sequence cost of sequence i (where i is the sequence number and 0 < i ≤ N, and in this example, N = 4) can be determined in the same manner as discussed above: , for all j, 0 < j ≤ N, where is the task cost associated with the task at position j.

[0101] For example, for Figure 5 sequence #2 (i = 2) in the updated plurality of sequences 502 shown in .

[0102] For example, in sequence #2, is the task cost of the task associated with the first vehicle 112a because task #1 (i.e., vehicle #1) is at position #4 in this sequence.

[0103] In addition, the task cost associated with each task / vehicle is based on the urgency measure associated with the vehicle / task and the resolution time of the task at position m in this sequence , where 0 < m ≤ N. As mentioned previously, the resolution time can be given by: , for all i, 0 < j ≤ m.

[0104] Therefore, as mentioned above, the task cost of the vehicle / task at position m can be given by: .

[0105] where is the urgency measure associated with the vehicle at position m.

[0106] Therefore, the sequence cost of each of the updated preferred sequences 502 can be determined in the same manner as discussed above.

[0107] One of the updated preferred sequences can be selected based on the sequence cost. For example, the selected updated preferred sequence can be the sequence with the lowest sequence cost. Then, the operator 108 processes the vehicles / tasks in the order indicated in the selected updated preferred sequence.

[0108] Instead of all possible sequences of executable tasks, sequence costs can be determined for multiple updated preferred sequences 502. In this example, there are 4 updated preferred sequences because additional tasks / vehicles can be added to 4 different positions within a preferred sequence that itself contains 3 tasks / vehicles. This contrasts with the total number of possible sequences in which 4 tasks can be arranged if the original 3 tasks are not limited to being in the same order relative to each other. More specifically, as discussed earlier, the total number of ways tasks can be sorted is given by N!, where N is the number of tasks / vehicles. In this example, there are 4 (N! = 4! = 4*3*2*1 = 24) different sequences in which the operator can handle 4 tasks. Therefore, when performing additional tasks after a preferred sequence has been determined, it may be computationally more efficient to determine the sequence costs of only multiple updated preferred sequences, rather than the sequence costs of all possible sequences. In this example, this results in determining only 4 sequence costs instead of 24. Therefore, the sequence chosen from 4 sequence costs may not be the actual sequence with the lowest total cost, but this process provides a balance between processing efficiency and the requirements of vehicles and operators.

[0109] Multiple operators In this example, instead of a single operator performing tasks and handling the vehicle, multiple operators can be present to address the vehicle's needs. In this scenario, multiple operators can work on the vehicle in parallel. An example of how this can be achieved will be described.

[0110] In this example, there may be N vehicles requiring assistance and M available operators, where N > M. Following the same method described above for a single operator, the preferred order for processing the vehicles / tasks can be determined. From here, the top M vehicles / tasks in the preferred order can be assigned to the M operators, allowing the M most important vehicles / tasks to work in parallel.

[0111] As an example, the first vehicle (AV@1) in the preferred sequence can be assigned to the first operator (TO1), the second vehicle (AV@2) can be assigned to the second operator (TO2), and so on, until the Mth vehicle (AV@M) is assigned to TOM.

[0112] From this point onward, the vehicle at position M+1 in the preferred order (AV@(M+1)) is assigned to the operator with the shortest total resolution time. The total resolution time is based on the resolution times of all vehicles already in the operator's queue. For example, if there is only one task / vehicle in the operator's queue, the operator's total resolution time is equal to the time associated with executing that task, or in some cases, equal to the time associated with executing that task plus the task switching time. Therefore, each operator's total resolution time is based on the cumulative time associated with executing the task currently assigned to the operator. Thus, the total resolution time is similar to the T described above. 总 By assigning the next task / vehicle to that specific operator, the waiting time for operators to resolve vehicles is reduced.

[0113] The same process can be repeated to assign the vehicle to position M+2 (AV@(M+2)) in the preferred order. As before, this vehicle can be assigned to the operator with the shortest total resolution time.

[0114] In the example, the total solution time calculated for each operator in each round of the process can be stored in memory and reused in the next round to speed up the calculation. For example, only the queues of operators who were assigned additional vehicles in the previous round need to have their total solution time updated.

[0115] As an example, we can assume that N = 8 (AV1 to AV8) and M = 4 (TO1 to TO4). In this example, the time T taken to complete the vehicle task is... 任务_n The following (in seconds): T 任务_1 = 30, T 任务_2 = 40, T 任务_3 = 50, T 任务_4 = 60, T 任务_5 = 100, T 任务_6 = 120, T 任务_7 = 180, T 任务_8 = 240.

[0116] Furthermore, in this example, T 切换任务 = 30s.

[0117] Based on the process described above, we can assume that the preferred order of task execution (assuming there is only one operator) is as follows: AV3, AV5, AV2, AV7, AV8, AV4, AV1, AV6.

[0118] Therefore, the process can begin by assigning the first M (in this case, four) vehicles to the operator in the following manner: TO1 receives AV3, TO2 receives AV5, TO3 receives AV2, and TO4 receives AV7.

[0119] To determine which operator was assigned vehicle M+1 (AV8 in this case), the algorithm calculates the total resolution time for each operator's queue. For example, the total resolution time is 50s for TO1; 100s for TO2; 40s for TO3; and 180s for TO4.

[0120] As mentioned, the vehicle (AV8) at position M+1 in the preferred order is assigned to the operator with the shortest total resolution time. In this case, AV8 is assigned to TO3.

[0121] To determine AV4 (the next vehicle in the preferred order), the algorithm again calculates (or receives from memory) the total resolution time for each operator's queue. For example, the total resolution time is 50s for TO1; 100s for TO2; 40s + 30s + 240s = 310s for TO3 (thus including task switching time); and 180s for TO4. In this case, AV4 is assigned to TO1.

[0122] To determine AV1 (the next AV in the deployment), the algorithm again calculates (or receives from memory) the total resolution time for each operator's queue. For example, for TO1, the total resolution time is 50s + 30s + 60s = 140s; for TO2, the total resolution time is 100s; for TO3, the total resolution time is 40s + 30s + 240s = 310s; and for TO4, the total resolution time is 180s. In this case, AV1 is assigned to TO2.

[0123] To determine AV6 (the next AV in the deployment), the algorithm again calculates (or receives from memory) the total resolution time for each operator's queue. For example, for TO1, the total resolution time is 50s + 30s + 60s = 140s; for TO2, the total resolution time is 100s + 30s + 30s = 160s; for TO3, the total resolution time is 40s + 30s + 240s = 310s; and for TO4, the total resolution time is 180s. In this case, AV6 is assigned to TO1.

[0124] The final allocation results are as follows: TO1 consists of the allocated vehicles: AV3, AV4, and AV6. TO2 consists of the allocated vehicles: AV5 and AV1. TO3 consists of the allocated vehicles: AV2 and AV8. TO4 consists of the allocated vehicle: AV7.

[0125] In the example, additional vehicles may require assistance (that is, in addition to the original 8 vehicles). Therefore, there may be additional tasks that the operator needs to perform in connection with the additional vehicles.

[0126] In the same manner described above for a single operator, the preferred task / vehicle sequence can be updated to include additional tasks / vehicles.

[0127] Continuing this example, vehicle number nine (AV9) may require assistance. The updated preferred order can be determined as follows: AV3, AV5, AV2, AV7, AV8, AV4, AV9, AV1, AV6.

[0128] In this example, the first six vehicles already assigned to operators will remain unchanged, while the process for assigning AV9, AV1, and AV6 needs to be repeated. These vehicles can be assigned in the same manner as previously discussed by determining the total resolution time for each operator's queue. In this example, the first six vehicles could be assigned to operators as follows: TO1 has been assigned to AV3 and AV4, TO2 has been assigned to AV5, TO3 has been assigned to AV2 and AV8, and TO4 has been assigned to AV7.

[0129] To determine AV9 (the next AV in the deployment), the algorithm calculates (or receives from memory) the total resolution time for each operator's queue. In the same manner discussed above, this results in AV9 being assigned to TO2. This process can be repeated for AV1 and AV6.

[0130] Additional Operator In the example, it can be useful to determine or estimate the time each vehicle must wait for an operator to resolve. For instance, for the second vehicle in a particular operator's queue, the time taken to complete the task associated with that vehicle (i.e., its resolution time) would be based on the length of time associated with executing the vehicle's task and the cumulative length of time associated with executing previous tasks in the queue. The resolution time may also include at least one task switching time.

[0131] In the example, it may be useful to additionally or alternatively determine or estimate the time it takes for each operator to resolve all vehicle tasks assigned to that operator's queue.

[0132] These are preliminary estimates, which can be revised based on the actual completion time to provide a more accurate estimate.

[0133] In the example, additional criteria can be determined. For example, at least one of the following: (i) the maximum permissible waiting time for any vehicle, or (ii) the maximum permissible task load time for any operator. As an example, the maximum permissible waiting time for any vehicle can be set to 300s, meaning that the time a vehicle waits for an operator to resolve should not exceed 300s. For any operator, the maximum permissible task load time can be the maximum permissible time that the operator spends resolving all vehicle tasks assigned to that operator's queue.

[0134] In the example, these criteria and estimates can be used to determine whether one or more additional operators are needed to handle the vehicle.

[0135] As an example, it can be assumed that the time each vehicle must wait for an operator to resolve should be less than the maximum allowable waiting time for any vehicle. Continuing the example above, this would mean that a vehicle should not wait longer than 300 seconds to be resolved. The minimum number of operators required can be determined using the following algorithm: Step 1. Determine the preferred order of task execution, as described above. In some examples, Step 1 may also include initializing variables: =The number of currently available operators, where It is about the number of operators required to operate the vehicle.

[0136] Step 2. Using the "multiple operators" algorithm described above, assign tasks / vehicles among the operators.

[0137] Step 3. For each vehicle / task, calculate the time each vehicle must wait for the operator to resolve.

[0138] Step 4. If, for any vehicle / task, the time a vehicle must wait for an operator to resolve exceeds the maximum permissible waiting time for any vehicle, the required number of operators should be increased, such as by adding one. For example, +1. For example, step 4 could involve determining the highest / maximum time a vehicle has been waiting for operator processing across all vehicles and comparing it to the maximum allowable waiting time, rather than calculating it for each task / vehicle.

[0139] Step 5: If additional operators are needed, repeat steps 2 through 4 (and if additional operators are still needed, repeat step 5). Repeat the steps until, for all vehicles / tasks, the time a vehicle must wait for an operator to resolve the task is less than the maximum allowable waiting time for any vehicle. For example, repeat the steps until the maximum waiting time between vehicles is less than the maximum allowable waiting time, such as 300 seconds.

[0140] Continuing with the example of eight vehicles described earlier, Figure 6AA sample table is depicted, assigning tasks to several operators; in this example, there are four operators. The time shown in the "Estimated Start Time" column corresponds to the time a vehicle must wait for an operator to resolve it. As shown, vehicle AV8 is the vehicle that must wait the longest for an operator (operator 3 in this example) to resolve it. In this example, the time is 310 seconds. This exceeds the maximum allowable wait time of 300 seconds, meaning that additional operators need to be added, and the task / vehicle needs to be reassigned, as mentioned above.

[0141] Figure 6B The updated assignment with additional operators is depicted. In this example, vehicle AV6 is the vehicle that must wait the longest for operator (operator 2 in this example) to resolve. In this example, the time is 250s. This is less than the maximum allowable waiting time of 300s for any vehicle, so no additional operator is needed.

[0142] It should be understood that the same algorithm and process can be achieved by using the maximum permissible task load time for any operator, rather than the maximum permissible waiting time for any vehicle.

[0143] Contact measurement As discussed above, urgency measurement can be based on vehicle contact measurement. In the example, each vehicle 112a, 112b, 112c can determine its own vehicle contact measurement and transmit the vehicle contact measurement (and / or urgency measurement based on the vehicle contact measurement) to remote system 100. In other examples, system 100 can determine the vehicle contact measurement associated with vehicle 112 itself, such as based on perception data received from vehicle 112.

[0144] As mentioned above, vehicle contact metrics can be based on perception data generated by the vehicle, which can be associated with one or more objects in the vehicle's surrounding environment. In the example, vehicle contact metrics can be based on one or more object contact metrics. For instance, a vehicle contact metric could be the sum, average, weighted average, or maximum of object contact metrics for one or more objects in the vehicle's surrounding environment.

[0145] As an example, Figure 7 This describes perception data that can be determined or generated by specific vehicles 112a, 112b, 112c based on sensor data obtained from one or more sensors of vehicle 112. In this example, Figure 7 The sensing data in the middle has been changed from Figure 1 The second vehicle 112b was generated based on nine objects near the second vehicle 112b.

[0146] exist Figure 7 In the table, each row is associated with a different object in the environment, and each column corresponds to a property associated with that object. In this example, the property includes: object number. a (or other unique identifier associated with the object), category or object type, the object's current location in the environment, velocity, the object's predicted future location, and object contact metrics. Contact metric per object All based on vehicles and objects a The probability of contact (such as collision) between objects. It should be understood that in other examples, there may be fewer or more objects, and / or fewer or more properties associated with each object. In the examples, objects may be associated with a different number of properties depending on the object.

[0147] The object contact metric can be determined / calculated based on sensor data obtained from sensors on the vehicle, and / or can be determined based on other perception data generated by vehicle 112.

[0148] In the example, the object contact metric for each object can be determined / calculated based on (i) the object’s potential future path / trajectory or location, and (ii) the action or expected action of vehicle 112. For example, the action or expected action of vehicles 112a, 112b, and 112c can be represented by the path or trajectory of vehicle 112.

[0149] For example, Figure 1 The trajectory 116 of the second vehicle 112b is depicted, which vehicle 112b may have already followed, may currently be following, or may follow in the near future. Although Figure 1 Although not shown, the future path or position of each object can also be determined. For example, the prediction component of vehicle 112b can determine the future path or position of each object based on its current position and speed. The prediction component of vehicle 112b can determine more than one future path or position for each object. Each future path or position can be associated with a probability. For example, an object may be more likely to follow one path in the future than another.

[0150] Based on the object's future path or location and the vehicle's actions or anticipated actions, vehicle 112b can determine the probability of a future collision / contact between the object and vehicle 112b. For example, if vehicle 112b's future path 116 intersects with the object's future path, the probability of contact may be higher than if the paths do not intersect. If vehicle 112b's future path 116 does not intersect with the object's future path, it can be determined that the object and vehicle are unlikely or improbable to contact each other.

[0151] Once the object contact metric for each object has been determined, the vehicle contact metric for vehicle 112b can be determined. The vehicle contact metric can be based on the overall probability of a collision between the vehicle and objects within the field of view of the vehicle sensors. Therefore, the vehicle contact metric can be based on object contact metrics. For example, the vehicle contact metric can be based on the average, sum, or maximum of the object contact metrics associated with objects within the field of view of the vehicle sensors. Continuing with the example above, based on... Figure 7 Object contact measurement The vehicle contact metric for the second vehicle 112b can be equal to 0.4 (e.g., Figure 2 (as shown in the image).

[0152] It should be understood that various different techniques can be used to determine the contact metric for vehicles and / or objects. In the example, the object contact metric could be based on the probability of a future collision / contact between the object and the vehicle, and the severity of the contact. Severity can depend on the type of object, such as whether the object is a vehicle, a pedestrian, etc. In some cases, the object contact metric is not based on the severity of the contact.

[0153] Generally, the vehicle contact measurement can be given by the following formula:

[0154] in It is a vehicle contact metric. It is an object contact metric. It is the weight of each object type. It is the probability of an object colliding with / making contact with a vehicle, and This refers to the severity of the collision / contact between the object and the vehicle. Therefore, the vehicle contact metric can be a function of these terms.

[0155] As a more concrete example, vehicle contact metric can be given by the following formula:

[0156] in It is the weight of the contact probability in the contact metric, and This represents the weight of contact severity in the contact metric. The weights for object type, contact probability, and contact severity can be arbitrarily chosen. It should be understood that this is a specific example of a vehicle contact metric, and other examples are envisioned.

[0157] The vehicle contact metric for each vehicle 112a, 112b, 112c can be determined in the same manner. Therefore, the urgency metric for each vehicle 112a, 112b, 112c can be determined based on the vehicle contact metric associated with each respective vehicle 112, as discussed above.

[0158] Figure 8A flowchart of example method 700 is shown. Example method 700 may be implemented by system 100 alone, or by vehicles 112a, 112b, 112c and system 100. In the example, method 700 may be encoded and stored as instructions on one or more non-transitory computer-readable media, which, when executed by one or more processors, cause system 100, or system 100 and vehicles 112a, 112b, 112c, to implement method 700. In the example, the method is a computer-implemented method.

[0159] like Figure 8 As can be seen, method / process 700 may include: at step 702, for each of the plurality of vehicles 112 (or for a corresponding vehicle), determining an urgency metric associated with that vehicle, the urgency metric indicating the urgency of performing the task associated with that vehicle. Method / process 700 may include: at step 704, for each of the plurality of vehicles 112 (or for a corresponding vehicle), determining the length of time associated with performing the task of that vehicle. For example, steps 702 and 704 may be performed by the vehicles 112a, 112b, 112c themselves, or by system 100. Method / process 700 may include: at step 706, determining a preferred order in which one or more operators perform tasks based on the length of time associated with the plurality of vehicles and the urgency metric. For example, step 706 may be performed by system 100.

[0160] Figure 9 A block diagram of an example system 800 implementing the techniques discussed above and herein is shown. The example system 800 may include a vehicle 802, which can represent... Figure 1 Vehicles 112a, 112b, and 112c are mentioned. In some cases, vehicle 802 may be an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration (NHTSA), which describes a vehicle capable of performing all safety-critical functions throughout the journey without expecting the driver (or occupant) to control the vehicle at any time. However, in other examples, vehicle 802 may be a fully or partially autonomous vehicle with any other level or classification. Furthermore, in some cases, the techniques described herein may also be usable by non-autonomous vehicles.

[0161] Vehicle 802 may include vehicle computing device 804, sensor 806, transmitter 808, network interface 810, and / or drive system 812. Sensor 806 may represent the sensors discussed above.

[0162] In some cases, sensor 806 may include lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., Global Positioning System (GPS), compasses, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), image sensors (e.g., red-green-blue (RGB), infrared (IR), intensity, depth, time-of-flight cameras, etc.), microphones, wheel encoders, environmental sensors (e.g., thermometers, hygrometers, light sensors, pressure sensors, etc.), etc. Sensor 806 may include multiple instances of each of these or other types of sensors. For example, radar sensors may include individual radar sensors located at corners, front, rear, sides, and / or top of vehicle 802. As another example, cameras may include multiple cameras positioned at various locations near the exterior and / or interior of vehicle 802. Sensor 806 may provide input to vehicle computing unit 804 and / or computing unit 832.

[0163] Data captured by sensors can be referred to as sensor data.

[0164] Vehicle 802 may also include a transmitter 808 for emitting light and / or sound. Transmitter 808 may include internal audio and visual transmitters for communicating with passengers of vehicle 802. Internal transmitters may include speakers, lights, signs, displays, touchscreens, haptic transmitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seatbelt tensioners, seat positioners, headrest positioners, etc.), and so on. Transmitter 808 may also include external transmitters. External transmitters may include lights (e.g., indicator lights, signs, light arrays, etc.) for signaling other indications of direction of travel or vehicle movement, and one or more audio transmitters (e.g., speakers, speaker arrays, horns, etc.) for audible communication with pedestrians or other nearby vehicles, one or more of which may include beam steering technology.

[0165] Vehicle 802 may also include a network interface 810 that enables communication between vehicle 802 and one or more other local or remote computing devices. Network interface 810 may facilitate communication with other local computing devices and / or drive components 812 on vehicle 802. Network interface 810 may additionally or alternatively allow the vehicle to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic lights, etc.). Network interface 810 may additionally or alternatively enable vehicle 802 to communicate with computing device 832 via network 838. In some examples, computing device 832 may include one or more nodes of a distributed computing system (e.g., a cloud computing architecture). Computing device 832 corresponds to the remote system 100 discussed above.

[0166] Vehicle 802 may include one or more drive components 812. In some cases, vehicle 802 may have a single drive component 812. In some cases, drive component 812 may include one or more sensors to detect the condition of drive component 812 and / or the surrounding environment of vehicle 802. By way of example and not limitation, sensors for drive component 812 may include: one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the wheels of drive component; inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of drive component; cameras or other image sensors; ultrasonic sensors for acoustically detecting objects in the surrounding environment of drive component; lidar sensors; radar sensors, etc. Some sensors (such as wheel encoders) may be unique to drive component 812. In some cases, sensors on drive component 812 may be superimposed on or complement a corresponding system of vehicle 802 (e.g., sensor 806).

[0167] Drive unit 812 may include a number of vehicle systems, including a high-voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery into alternating current for use by other vehicle systems, a steering system including a steering motor and a steering rack (which may be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate traction loss and maintain control, an HVAC system, lighting (e.g., headlights / taillights for illuminating the external surroundings of the vehicle), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC / DC converters, high-voltage connectors, high-voltage cables, charging systems, charging ports, etc.). Additionally, drive unit 812 may include a drive unit controller that can receive and preprocess data from sensors and control the operation of various vehicle systems. In some cases, the drive unit controller may include one or more processors and a memory communicatively coupled to one or more processors. The memory may store one or more components to perform various functionalities of drive unit 812. In addition, the drive unit 812 may also include one or more communication connections, which enable the respective drive unit to communicate with one or more other local or remote computing devices.

[0168] The vehicle computing device 804 may include a processor 814 and a memory 816 communicatively coupled to one or more processors 814. The computing device 832 may also include a processor 834 and / or a memory 836. Processors 814 and / or 834 may be any suitable processor capable of executing instructions to process data and perform operations as described herein. By way of example and not limitation, processors 814 and / or 834 may include one or more central processing units (CPUs), graphics processing units (GPUs), integrated circuits (e.g., application-specific integrated circuits (ASICs)), gate arrays (e.g., field-programmable gate arrays (FPGAs)), and / or any other means or part of means for processing electronic data to convert that electronic data into other electronic data that can be stored in registers and / or memory.

[0169] Memory 816 and / or 836 may be examples of non-transitory computer-readable media. Memory 816 and / or 836 may store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and the functions attributed to various systems. In various implementations, any suitable memory technology may be used to implement the memory, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architectures, systems, and various elements described herein may include many other logical, programming, and physical components; those shown in the figures are merely examples relevant to the discussion herein.

[0170] In some cases, memory 816 and / or memory 836 may store sensing unit 818, positioning unit 820, planning unit 822, map 824, driving log data 826, prediction unit 828 and / or system controller 830, wherein zero or more parts of any of them may be hardware, such as GPU, CPU and / or other processing units.

[0171] The sensing component 818 can detect objects in the environment surrounding the vehicle 802 (e.g., identify the presence of objects), classify objects (e.g., determine the object type associated with the detected objects), segment sensor data and / or other representations of the environment (e.g., identify portions of the sensor data and / or environmental representations as associated with the detected objects and / or object types), determine characteristics associated with the objects (e.g., tracks that identify current, predicted, and / or previous orientation, heading, speed, and / or acceleration associated with the objects), etc. The data determined by the sensing component 818 is referred to as sensing data. The sensing component 818 can be configured to associate boundary regions (or other indications) with the identified objects. The sensing component 818 can be configured to associate a confidence score associated with the classification of the identified objects with the identified objects. In some examples, objects may be colored based on their perceived category when rendered via a display. The object classification determined by the sensing component 818 can distinguish different object types, such as, for example, passenger cars, pedestrians, cyclists, drivers, delivery trucks, semi-trucks, traffic signs, etc. The sensing component 818 can detect objects based on sensor data received from the sensor 806.

[0172] In at least one example, the positioning component 820 may include hardware and / or software to receive data from sensor 806 to determine the orientation, velocity, and / or orientation (e.g., one or more of x-azimuth, y-azimuth, z-azimuth, roll, pitch, or yaw) of vehicle 802. For example, the positioning component 820 may include and / or request / receive a map 824 of the environment and may continuously determine the position, velocity, and / or orientation of the autonomous vehicle 802 within map 824. In some cases, the positioning component 820 may utilize SLAM (Simultaneous Localization and Mapping), CLAMS (Simultaneous Calibration, Localization, and Mapping), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc., to receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc., to accurately determine the position, attitude, and / or velocity of the autonomous vehicle. In some cases, the positioning component 820 may provide data to various components of vehicle 802 to determine the initial orientation of the autonomous vehicle for trajectory generation and / or for determining map data, as discussed herein. In some examples, the positioning component 820 may provide the sensing component 818 with the position and / or orientation of the vehicle 802 relative to the environment and / or associated sensor data.

[0173] The planning component 822 may receive position and / or orientation information of the vehicle 802 from the positioning component 820, and / or receive sensing data from the sensing component 818, and may determine instructions for controlling the operation of the vehicle 802 based at least in part on any such data. In some examples, determining the instructions may include determining the instructions based at least in part on a format associated with the system to which the instructions are associated (e.g., a first instruction for controlling the motion of the autonomous vehicle may be formatted such that the system controller 830 and / or drive component 812 can parse / make implement a first message and / or signal format (e.g., analog, digital, aerodynamic, kinematic), and a second instruction for the transmitter 808 may be formatted according to a second format associated therewith).

[0174] Driving log data 826 may include sensor data, perception data, and / or scenario tags collected / determined by vehicle 802 (e.g., by sensing unit 818), as well as any other messages generated and / or sent by vehicle 802 during operation, including but not limited to control messages, error messages, etc. In some examples, vehicle 802 may transmit driving log data 826 to computing device 832.

[0175] Prediction component 828 can generate one or more probability maps representing predicted probabilities of the possible locations of one or more objects in the environment. For example, prediction component 828 can generate one or more probability maps for vehicles, pedestrians, animals, etc., within a threshold distance from vehicle 802. In some examples, prediction component 828 can measure the tracks of objects and generate discretized prediction probability maps, heatmaps, probability distributions, discretized probability distributions, and / or object trajectories based on observed and predicted behavior. In some examples, one or more probability maps can represent the intentions of one or more objects in the environment. In some examples, planning component 822 can be communicatively coupled to prediction component 828 to generate predicted trajectories of objects in the environment. For example, prediction component 828 can generate one or more predicted trajectories for objects within a threshold distance from vehicle 802. In some examples, prediction component 828 can measure the tracks of objects and generate object trajectories based on observed and predicted behavior. Although prediction component 828 is shown on vehicle 802 in this example, prediction component 828 can also be provided elsewhere, such as in a remote computing device. In some examples, prediction components can be provided at both the vehicle and the remote computing device. These components can be configured to operate according to the same or similar algorithms. The data generated by the prediction component 828 can be provided to the computing device 832 as the perception data. The perception data can be used by the planning component 822 for navigation in the environment.

[0176] Memory 816 and / or 836 may additionally or alternatively store mapping systems, planning systems, riding management systems, etc. Although sensing components 818 and / or planning components 822 are shown as stored in memory 816, sensing components 818 and / or planning components 822 may include processor-executable instructions, machine learning models (e.g., neural networks), and / or hardware.

[0177] As described herein, the localization component 820, the sensing component 818, the planning component 822, and / or other components of the system 800 may include one or more ML models. For example, the localization component 820, the sensing component 818, and / or the planning component 822 may each include different ML model pipelines. In some examples, the ML model may include a neural network. An exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to produce an output. Each layer in the neural network may also include another neural network, or may include any number of layers (whether convolutional or not). As will be understood in the context of this disclosure, neural networks may utilize machine learning, which can refer to a broad class of algorithms in which outputs are generated based on learned parameters.

[0178] Although discussed in the context of neural networks, any type of machine learning consistent with this disclosure can be used. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimation scatter plot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute value convergence and selection operator (LASSO), elastic networks, least angular regression (LARS)), and decision tree algorithms (e.g., classification and regression trees (CART), iterative binary trees). (ID3), Chi-squared Automatic Interaction Detection (CHAD), Decision Stump, Conditional Decision Tree), Bayesian Algorithms (e.g., Naive Bayes, Gaussian Naive Bayes, Multinomial Naive Bayes, Average Single Dependency Estimator (AODE), Bayesian Belief Network (BNN), Bayesian Network), Clustering Algorithms (e.g., k-means, k-median, Expectation-Maximum (EM), Hierarchical Clustering), Association Rule Learning Algorithms (e.g., Perceptron, Backpropagation, Hopfield Network, Radial Basis Function Network (RBFN)), Deep Learning Algorithms (e.g., Deep Boltzmann Machine (DBM), Deep Belief Network (DBN), Convolutional Neural Network (CNN), Stacked Autoencoder), Dimensionality Reduction Algorithms (e.g., Principal Component Analysis (PCA), Principal Component Regression (PCR), Partial Least Squares Regression (PLSR), Sammon Mapping, Multidimensional Scaling (MDS), Projective Pursuit, Linear Discriminant Analysis (LDA), Mixture Discriminant Analysis (MDA), Quadratic Discriminant Analysis (QDA), Flexible Discriminant Analysis (FDA)), Ensemble Algorithms (e.g., Boosting, Bootstrapped) Aggregation (Bagging), AdaBoost, stacked generalization (hybrid), Gradient Boosting Machine (GBM), Gradient Boosting Regression Tree (GBRT), Random Forest, SVM (Support Vector Machine), supervised learning, unsupervised learning, semi-supervised learning, etc. Additional examples of architectures include neural networks such as ResNet-50, ResNet-101, VGG, DenseNet, PointNet, etc. In some examples, the ML models discussed herein may include PointPillars, SECOND, top-down feature layers (e.g., see U.S. Patent Application Serial No. 15 / 963,833, which is incorporated herein in its entirety), and / or VoxelNet. Architecture-delayed optimizations may include MobilenetV2, ShuffleNet, ChannelNet, PeleeNet, etc. In some examples, ML models may include residual blocks, such as Pixor.

[0179] The memory 820 may additionally or alternatively store one or more system controllers 830, which may be configured to control the steering, propulsion, braking, safety, transmitter, communication, and other systems of the vehicle 802. These system controllers 830 may communicate with and / or control corresponding systems of the drive components 812 and / or other components of the vehicle 802.

[0180] It should be noted that, although Figure 9 While shown as a distributed system, in an alternative example, components of vehicle 802 may be associated with computing device 832, and / or components of computing device 832 may be associated with vehicle 802. That is, vehicle 802 may perform one or more of the functions associated with computing device 832, and vice versa.

[0181] Example Terms 1. A system comprising: One or more processors; and One or more non-transitory computer-readable media storing instructions thereon, the instructions, when executed by the one or more processors, causing the one or more processors to perform operations, the operations including: For the corresponding vehicle among multiple vehicles: Determine or receive an urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing a task associated with the vehicle, and based on an contact metric associated with the vehicle, the contact metric being based on the probability that the vehicle comes into contact with one or more objects in the environment surrounding the vehicle; and Determine or receive the duration of time associated with performing the task of the vehicles, the task being performed by one or more operators located remotely from the vehicles and responsible for monitoring and / or controlling the vehicles; and Based on the time length associated with the plurality of vehicles and the urgency metric, a preferred order in which the one or more operators perform the task is determined.

[0182] 2. The system as described in Clause 1, wherein determining the preferred order in which the one or more operators perform the task comprises: Determine multiple different sequences in which the one or more operators can perform the tasks associated with the plurality of vehicles; For a given order among the plurality of different sequences, a sequence cost associated with that sequence is determined, the sequence cost being based on: (i) the urgency metric associated with the plurality of vehicles, (ii) the time length associated with performing the task, and (iii) the sequence in which the task is performed; and Based on the order cost, the preferred order is selected from the plurality of different orders.

[0183] 3. The system as described in Clause 2, wherein after selecting the preferred order, the operation further includes: Determine that additional tasks associated with the additional vehicle need to be performed by the one or more operators; and Determine the execution position of the additional task in the preferred order.

[0184] 4. The system as described in Clause 2 or 3, wherein determining the sequence cost associated with a given sequence among the plurality of different sequences includes: For the corresponding tasks in the aforementioned sequence: The resolution time is determined based on: (i) the length of time associated with performing the task, and (ii) the cumulative length of time associated with performing previous tasks in the sequence; and The task cost is determined based on (i) the resolution time and (ii) the urgency measure associated with the vehicle; The sequence cost associated with the sequence is the sum of the task costs based on the sequence.

[0185] 5. The system as described in Clause 4, wherein the resolution time is further based on at least one task switching time, which indicates the time taken for an operator to switch from one task to another.

[0186] 6. A system as described in any of the preceding clauses, wherein the urgency measure associated with the vehicle is further based on one or more of the following: One or more characteristics of one or more passengers in the vehicle; Traffic conditions; Time of day; The location of the vehicle; or The type of fault associated with the vehicle.

[0187] 7. A method comprising: For the corresponding vehicle among multiple vehicles: Determine an urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing a task associated with the vehicle; and Determine the length of time associated with performing the task of the vehicles, the task being performed by one or more operators located remotely from the vehicles and responsible for monitoring and / or controlling the vehicles; and Based on the time length associated with the plurality of vehicles and the urgency metric, a preferred order in which the one or more operators perform the task is determined.

[0188] 8. The method as described in Clause 7, further comprising: For the corresponding vehicle among the plurality of vehicles: Determine a contact metric associated with the vehicle, the contact metric being based on the probability that the vehicle comes into contact with one or more objects in the environment surrounding the vehicle; Determining the urgency metric associated with the vehicle includes determining the urgency metric based on the contact metric associated with the vehicle.

[0189] 9. The method as described in Clause 7 or 8, wherein determining the urgency metric associated with the vehicle comprises determining the urgency metric based on one or more of the following: One or more characteristics of one or more passengers in the vehicle; Traffic conditions; Time of day; The location of the vehicle; or The type of fault associated with the vehicle.

[0190] 10. The method as described in Clause 9, wherein the location is based on one or more of the following: The position of the vehicle relative to the intersection or other road features; The speed limit of the road where the vehicle is located; The lane in which the vehicle is located; or The number of lanes used for vehicles traveling in the same direction.

[0191] 11. The method of any one of Clauses 7 to 10, wherein determining the preferred order in which the one or more operators perform the task comprises: Determine multiple different sequences in which the one or more operators can perform the tasks associated with the plurality of vehicles; For a given order among the plurality of different sequences, a sequence cost associated with that sequence is determined, the sequence cost being based on: (i) the urgency metric associated with the plurality of vehicles, (ii) the time length associated with performing the task, and (iii) the sequence in which the task is performed; and Based on the order cost, the preferred order is selected from the plurality of different orders.

[0192] 12. The method of claim 11, wherein determining the sequence cost associated with a given sequence among the plurality of different sequences comprises: For the corresponding tasks in the aforementioned sequence: The resolution time is determined based on: (i) the length of time associated with performing the task, and (ii) the cumulative length of time associated with performing previous tasks in the sequence; and The task cost is determined based on (i) the resolution time and (ii) the urgency measure associated with the vehicle; The sequence cost associated with the sequence is the sum of the task costs based on the sequence.

[0193] 13. The method as described in Clause 12, wherein the resolution time is further based on at least one task switching time, which indicates the time spent by an operator switching from one task to another.

[0194] 14. The method as described in clause 11 or 12, wherein after selecting the preferred order, the method further comprises: Determine that additional tasks associated with the additional vehicle need to be performed by the one or more operators; and Determine the execution position of the additional task in the preferred order.

[0195] 15. The method of Clause 14, wherein determining the execution position of the additional task in the preferred order comprises: determining an updated preferred order, the updated preferred order having the additional task at a specific position within the preferred order, and wherein determining the updated preferred order comprises: A plurality of updated preferred orders are determined, wherein the tasks associated with the plurality of vehicles are arranged in the same order as in the preferred order, and the additional tasks associated with the additional vehicles are arranged at different positions in the preferred order; For a given order among the plurality of updated preferred orders, a sequence cost associated with that order is determined, the sequence cost being based on: (i) the urgency metric associated with the plurality of vehicles and the urgency metric associated with the additional vehicle; (ii) the time length associated with performing the task and the time length associated with performing the additional task; and (iii) the execution order of the tasks in the updated preferred order; and Based on the order cost, the updated preferred order is selected from the plurality of updated preferred orders.

[0196] 16. The method of Clause 15, wherein determining the sequence cost associated with a respective order among the plurality of updated preferred orders includes: For the corresponding tasks in the aforementioned sequence: The resolution time is determined based on: (i) the length of time associated with performing the task, and (ii) the cumulative length of time associated with performing previous tasks in the sequence; and The task cost is determined based on (i) the resolution time and (ii) the urgency measure associated with the vehicle; The sequence cost associated with the sequence is the sum of the task costs based on the sequence.

[0197] 17. The method of any one of Clauses 7 to 16, further comprising one of the following: Data is received from a specific vehicle among the plurality of vehicles at a system located remotely from the plurality of vehicles, wherein the urgency measure of the specific vehicle is determined by the remote system based on the data; or The urgency measure associated with the respective vehicle is received at the remote system from the respective vehicle among the plurality of vehicles.

[0198] 18. The method as described in any one of Clauses 7 to 17, wherein: There are N vehicles and N tasks; There are M operators to perform the tasks associated with the plurality of vehicles; and N>M; and The method includes: Assign the first M tasks in the preferred order to the M operators; and Based on the total resolution time of each operator, the remaining tasks in the preferred order are assigned to the M operators, where the total resolution time is based on the cumulative time length associated with executing the task currently assigned to the operator.

[0199] 19. The method of any one of Clauses 7 to 18, further comprising: Receive data from the respective vehicle among the plurality of vehicles at a system located remotely from the plurality of vehicles; and The remote system determines, based on the data, which of the plurality of vehicles requires an operator to perform a task.

[0200] 20. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a system, cause the system to perform operations, the operations comprising: For the corresponding vehicle among multiple vehicles: Determine or receive an urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing a task associated with the vehicle; and Determine or receive the duration of time associated with performing the task of the vehicles, the task being performed by one or more operators located remotely from the vehicles and responsible for monitoring and / or controlling the vehicles; and Based on the time length associated with the plurality of vehicles and the urgency metric, a preferred order in which the one or more operators perform the task is determined.

[0201] 21. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a system, cause the system to perform the method as described in any one of clauses 7 to 19.

[0202] 22. A system comprising: One or more processors; and One or more non-transitory computer-readable media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to perform the method as described in any one of Clauses 7 to 19.

[0203] While the example clauses described above pertain to a particular implementation, it should be understood that, within the context of this document, the content of the example clauses may also be implemented via methods, apparatus, systems, computer-readable media, and / or other implementations. Furthermore, any of example clauses 1 through 22 may be implemented individually or in combination with any other one or more of the example clauses.

[0204] in conclusion While one or more examples of the techniques described herein have been described, various modifications, additions, arrangements, and equivalents thereof are included within the scope of the techniques described herein.

[0205] In the description of the examples, reference is made to the accompanying drawings, which form part of this document, illustrating specific examples of the claimed subject matter by way of illustration. It should be understood that other examples may be used, and variations or modifications, such as structural changes, may be made. Such examples, variations, or modifications do not necessarily deviate from the scope of the established claimed subject matter. While the steps in this document may be presented in a certain order, in some cases the order may be changed so that certain inputs are provided at different times or in a different order without altering the functionality of the described system and method. The disclosed procedures may also be performed in different orders. Furthermore, the various calculations in this document need not be performed in the disclosed order, and other examples using different orders of calculation can be readily implemented. Besides reordering, these calculations may also be decomposed into sub-computations, yielding the same results.

[0206] Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as exemplary forms of implementing the claims.

[0207] The components described herein represent instructions that can be stored in any type of computer-readable medium and can be implemented in software and / or hardware. All the methods and processes described above can be embodied in software code components and / or computer-executable instructions executed by one or more computers or processors, hardware, or some combination thereof, and are fully automated via these software code components and / or computer-executable instructions. Some or all of the methods described may alternatively be embodied in dedicated computer hardware.

[0208] At least some of the processes discussed herein are illustrated as logic flowcharts, where each operation represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, an operation represents computer-executable instructions stored on one or more non-transitory computer-readable storage media, which, when executed by one or more processors, cause a computer or autonomous vehicle to perform the operation. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a specific function or implement a specific abstract data type. The order in which operations are described is not intended to be restrictive, and any number of the operations can be combined in any order and / or in parallel to implement the process.

[0209] Unless otherwise stated, conditional languages ​​such as “can,” “able,” “may,” or “possibly” are understood in context to imply that some examples include certain features, elements, and / or steps that are not included in other examples. Therefore, such conditional languages ​​are generally not intended to imply that certain features, elements, and / or steps are required in any way for one or more examples, or that one or more examples must include logic for determining whether or not certain features, elements, and / or steps are included or will be performed in any particular example, with or without user input or prompts.

[0210] Unless otherwise stated, conjunctive language such as the phrase “at least one of X, Y, or Z” should be understood to mean that an item, term, etc., can be X, Y, or Z, or any combination thereof, including multiples of each element. Unless explicitly stated as singular, “a” means both singular and plural.

[0211] It should be understood that throughout this disclosure, any reference to one aspect being based on the other aspect may imply that the aspect is at least partially based on the other aspect.

[0212] Any routine description, element, or block depicted in the flowcharts described herein and / or in the accompanying drawings should be understood as potentially representing a module, segment, or portion of code comprising one or more computer-executable instructions for implementing a particular logical function or element in the routine. Alternative implementations are included within the scope of the examples described herein, where elements or functions may be removed or performed in a non-shown or non-discussed order, including substantially synchronously, in reverse order, with appended operations, or with omitted operations, depending on the functionality involved, as will be understood by those skilled in the art. Note that the term can substantially indicate scope. For example, substantially simultaneous can indicate that two activities occur within each other's time scope, substantially the same dimension can indicate that two elements have dimensions within each other's scope, etc.

[0213] Many variations and modifications can be made to the above examples, and their elements should be understood as existing in other acceptable examples. All such modifications and variations are intended to be included within the scope of this disclosure and protected by the appended claims.

Claims

1. A system comprising: One or more processors; as well as One or more non-transitory computer-readable media storing instructions thereon, the instructions, when executed by the one or more processors, causing the one or more processors to perform operations, the operations including: For the corresponding vehicle among multiple vehicles: Determine or receive an urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing a task associated with the vehicle, and based on an contact metric associated with the vehicle, the contact metric being based on the probability that the vehicle comes into contact with one or more objects in the environment surrounding the vehicle; and Determine or receive the duration of time associated with performing the task of the vehicles, the task being performed by one or more operators located remotely from the vehicles and responsible for monitoring and / or controlling the vehicles; and Based on the time length associated with the plurality of vehicles and the urgency metric, a preferred order in which the one or more operators perform the task is determined.

2. The system of claim 1, wherein determining the preferred order in which the one or more operators perform the task comprises: Determine multiple different sequences in which the one or more operators can perform the tasks associated with the plurality of vehicles; For a given order among the plurality of different orders, a sequence cost associated with the order is determined, the sequence cost being based on: (i) the urgency metric associated with the plurality of vehicles, (ii) the time length associated with performing the task, and (iii) the order in which the task is performed; as well as Based on the order cost, the preferred order is selected from the plurality of different orders.

3. The system of claim 2, wherein after selecting the preferred order, the operation further includes: It is determined that additional tasks associated with the additional vehicle need to be performed by the one or more operators; as well as Determine the execution position of the additional task in the preferred order.

4. The system of claim 2, wherein determining the sequence cost associated with a given sequence among the plurality of different sequences comprises: For the corresponding tasks in the aforementioned sequence: The resolution time is determined based on (i) the length of time associated with performing the task, and (ii) the cumulative length of time associated with performing the previous tasks in the sequence. as well as The task cost is determined based on (i) the resolution time and (ii) the urgency measure associated with the vehicle; The sequence cost associated with the sequence is the sum of the task costs based on the sequence.

5. The system of claim 4, wherein the resolution time is further based on at least one task switching time, the task switching time indicating the time spent by an operator switching from one task to another.

6. The system of claim 1, wherein the urgency measure associated with the vehicle is further based on one or more of the following: One or more characteristics of one or more passengers in the vehicle; Traffic conditions; Time of day; The location of the vehicle; or The type of fault associated with the vehicle.

7. A method comprising: For the corresponding vehicle among multiple vehicles: Determine an urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing the task associated with the vehicle; as well as Determine the length of time associated with performing the task of the vehicle, the task being performed by one or more operators who are located remotely from the vehicles and are responsible for monitoring and / or controlling the vehicles; as well as Based on the time length associated with the plurality of vehicles and the urgency metric, a preferred order in which the one or more operators perform the task is determined.

8. The method of claim 7, further comprising: For the corresponding vehicle among the plurality of vehicles: Determine a contact metric associated with the vehicle, the contact metric being based on the probability that the vehicle comes into contact with one or more objects in the environment surrounding the vehicle; Determining the urgency metric associated with the vehicle includes determining the urgency metric based on the contact metric associated with the vehicle.

9. The method of claim 7, wherein determining the urgency metric associated with the vehicle comprises determining the urgency metric based on one or more of the following: One or more characteristics of one or more passengers in the vehicle; Traffic conditions; Time of day; The location of the vehicle; or The type of fault associated with the vehicle.

10. The method of claim 9, wherein the position is based on one or more of the following: The position of the vehicle relative to the intersection or other road features; The speed limit of the road where the vehicle is located; The lane in which the vehicle is located; or The number of lanes used for vehicles traveling in the same direction.

11. The method of claim 7, wherein determining the preferred order in which the one or more operators perform the task comprises: Determine multiple different sequences in which the one or more operators can perform the tasks associated with the plurality of vehicles; For a given order among the plurality of different orders, a sequence cost associated with the order is determined, the sequence cost being based on: (i) the urgency metric associated with the plurality of vehicles, (ii) the time length associated with performing the task, and (iii) the order in which the task is performed; as well as Based on the order cost, the preferred order is selected from the plurality of different orders.

12. The method of claim 11, wherein determining the sequence cost associated with a given sequence among the plurality of different sequences comprises: For the corresponding tasks in the aforementioned sequence: The resolution time is determined based on (i) the length of time associated with performing the task, and (ii) the cumulative length of time associated with performing the previous tasks in the sequence. as well as The task cost is determined based on (i) the resolution time and (ii) the urgency measure associated with the vehicle; The sequence cost associated with the sequence is the sum of the task costs based on the sequence.

13. The method of claim 12, wherein the resolution time is further based on at least one task switching time, the task switching time indicating the time spent by an operator switching from one task to another.

14. The method of claim 11, wherein after selecting the preferred order, the method further comprises: It is determined that additional tasks associated with the additional vehicle need to be performed by the one or more operators; as well as Determine the execution position of the additional task in the preferred order.

15. The method of claim 14, wherein determining the execution position of the additional task in the preferred order comprises: Determine an updated preferred order, wherein the updated preferred order has the additional task at a specific position within the preferred order, and wherein determining the updated preferred order includes: A plurality of updated preferred orders are determined, wherein the tasks associated with the plurality of vehicles are arranged in the same order as in the preferred order, and the additional tasks associated with the additional vehicles are arranged at different positions in the preferred order; For a given order among the plurality of updated preferred orders, a sequence cost associated with that order is determined, the sequence cost being based on: (i) the urgency metric associated with the plurality of vehicles and the urgency metric associated with the additional vehicle; (ii) the time length associated with performing the task and the time length associated with performing the additional task; and (iii) the execution order of the tasks in the updated preferred order; and Based on the order cost, the updated preferred order is selected from the plurality of updated preferred orders.

16. The method of claim 15, wherein determining the sequence cost associated with a given sequence among the plurality of updated preferred sequences comprises: For the corresponding tasks in the aforementioned sequence: The resolution time is determined based on (i) the length of time associated with performing the task, and (ii) the cumulative length of time associated with performing the previous tasks in the sequence. as well as The task cost is determined based on (i) the resolution time and (ii) the urgency measure associated with the vehicle; The sequence cost associated with the sequence is the sum of the task costs based on the sequence.

17. The method of claim 7, further comprising one of the following: Data is received from a specific vehicle among the plurality of vehicles at a system located remotely from the plurality of vehicles, wherein the urgency measure of the specific vehicle is determined by the remote system based on the data; or The urgency measure associated with the respective vehicle is received at the remote system from the respective vehicle among the plurality of vehicles.

18. The method of claim 7, wherein: There are N vehicles and N tasks; There are M operators to perform the tasks associated with the plurality of vehicles; and N>M; and The method includes: The first M tasks in the preferred order are assigned to the M operators; as well as Based on the total resolution time of each operator, the remaining tasks in the preferred order are assigned to the M operators, where the total resolution time is based on the cumulative time length associated with executing the task currently assigned to the operator.

19. The method of claim 7, further comprising: Receive data from the respective vehicle among the plurality of vehicles at a system located remotely from the plurality of vehicles; as well as The remote system determines, based on the data, which of the plurality of vehicles requires an operator to perform a task.

20. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors of a system, cause the system to perform operations, the operations comprising: For the corresponding vehicle among multiple vehicles: Determine or receive an urgency metric associated with the vehicle, the urgency metric indicating the urgency of performing a task associated with the vehicle; as well as Determine or receive the duration of time associated with performing the task of the vehicle, the task being performed by one or more operators who are located remotely from the vehicles and are responsible for monitoring and / or controlling the vehicles; as well as Based on the time length associated with the plurality of vehicles and the urgency metric, a preferred order in which the one or more operators perform the task is determined.

Images