OPERATOR MANAGEMENT SYSTEM, OPERATOR MANAGEMENT METHOD, OPERATOR MANAGEMENT PROGRAM

Abstract

To provide an operator management system for managing allocation of a plurality of remote support operators with respect to a plurality of vehicles which transmit a remote support request, and also for appropriately allocating the remote support operator when the number of transmitted remote support requests is greater than the number of the remote support operators.SOLUTION: An operator management system is constituted to execute: processing for setting or updating allocation priority corresponding to contents of a remote support request for each of a plurality of vehicles; and processing for allocating a remote support operator allocated to a second vehicle to a first vehicle when the number of the plurality of vehicles is greater than the number of the plurality of remote support operators and also when the allocation priority of the first vehicle to which the remote support operator is not allocated is higher than the allocation priority of the second vehicle to which the remote support operator is allocated.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present disclosure relates to a technique for managing the allocation of multiple remote assistance operators to multiple vehicles that originate remote assistance requests. [Background technology]

[0002] In an autonomous vehicle that runs according to a driving plan, if the autonomous vehicle encounters or is likely to encounter a driving decision that the autonomous vehicle is not good at, it is considered that the autonomous vehicle will issue a remote support request to a remote support operator in a remote location. By having the remote support operator make appropriate driving decisions, the autonomous vehicle can avoid stopping autonomous driving or getting into dangerous situations and continue autonomous driving smoothly.

[0003] Patent document 1 discloses a control device that includes a generation unit that generates priority information indicating the priority of multiple autonomous driving operations, a control unit that determines one of the multiple autonomous driving operations based on the priority information and performs autonomous driving, and an output unit that outputs a vehicle operation request when the control unit cannot determine one autonomous driving operation.

[0004] Other documents that demonstrate the technical level in the technical field related to the present disclosure include the following Patent Documents 2 to 5. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2018-151908 A [Patent Document 2] JP 2018-142265 A [Patent Document 3] JP 2019-185279 A [Patent Document 4] JP 2019-185451 A [Patent Document 5] JP 2019-021200 A Summary of the Invention [Problem to be solved by the invention]

[0006] When multiple autonomous vehicles are sending remote assistance requests, multiple remote assistance operators are required. On the other hand, from the viewpoint of human costs, the number of remote assistance operators is generally small compared to the number of autonomous vehicles. Therefore, if the number of autonomous vehicles sending remote assistance requests increases, there may be a shortage of remote assistance operators.

[0007] Conventionally, remote support operators are assigned in the order in which remote support requests are made. Therefore, if there is a shortage of remote support operators, autonomous vehicles that make remote support requests later will wait to start remote support regardless of the content of the remote support request. However, depending on the content of the remote support request, it may be more appropriate to first process autonomous vehicles that make remote support requests later, from the perspective of speeding up emergency responses and smoothing traffic flow.

[0008] In consideration of the above-mentioned problems, one object of the present disclosure is to provide technology that manages the allocation of multiple remote support operators to multiple vehicles that issue remote support requests, and that is capable of appropriately allocating remote support operators when the number of remote support requests issued is greater than the number of remote support operators. [Means for solving the problem]

[0009] The first disclosure relates to an operator management system that manages allocation of a plurality of remote assistance operators to a plurality of vehicles that issue remote assistance requests.

[0010] The operator management system according to the first disclosure includes one or more processors, and a memory coupled to the one or more processors and storing a plurality of instructions executable by the one or more processors, wherein the plurality of instructions are configured to cause the one or more processors to execute a process of setting or updating an allocation priority for each of a plurality of vehicles according to the content of a remote support request, and a process of assigning a remote support operator assigned to a second vehicle to a first vehicle when the number of the plurality of vehicles is greater than the number of a plurality of remote support operators and the allocation priority of a first vehicle which is one of the plurality of vehicles to which a plurality of remote support operators has not been assigned is higher than the allocation priority of a second vehicle which is one of the plurality of vehicles to which a plurality of remote support operators has been assigned.

[0011] The second disclosure relates to an operator management system further having the following features in addition to the operator management system according to the first disclosure.

[0012] The process of setting or updating the allocation priority includes setting or updating the allocation priority for each of the multiple vehicles in response to a newly acquired remote assistance request.

[0013] The third disclosure relates to an operator management system further having the following features in addition to the operator management system according to the first or second disclosure.

[0014] The remote assistance request includes one or more assistance request items, and the process of setting or updating the allocation priority includes calculating an assistance priority for each of the one or more assistance request items, and setting or updating the allocation priority based on the assistance priority.

[0015] The fourth disclosure relates to an operator management system further having the following features in addition to the operator management system according to the third disclosure.

[0016] The process of setting or updating the allocation priority includes setting or updating the allocation priority for each of the plurality of vehicles in response to one or a portion of a plurality of assistance request items being processed by a remote assistance operator.

[0017] The fifth disclosure relates to an operator management system further having the following features in addition to the operator management system according to the fourth disclosure.

[0018] Each of the multiple vehicles is an autonomous vehicle that drives according to a driving plan that is a combination of multiple driving decisions, and is configured to transmit the driving plan as a remote assistance request in response to the multiple driving decisions including one or more assistance request items.

[0019] The sixth disclosure relates to an operator management method for managing, by a computer, the allocation of a plurality of remote support operators to a plurality of vehicles that issue remote support requests.

[0020] The operator management method of the sixth disclosure is characterized in that it includes setting or updating an allocation priority for each of a plurality of vehicles according to the content of the remote support request, and when the number of the plurality of vehicles is greater than the number of a plurality of remote support operators and the allocation priority of a first vehicle which is one of the plurality of vehicles to which a plurality of remote support operators has not been assigned is higher than the allocation priority of a second vehicle which is one of the plurality of vehicles to which a plurality of remote support operators has been assigned, assigning the remote support operator assigned to the second vehicle to the first vehicle.

[0021] The seventh disclosure relates to an operator management program that causes a computer to manage the allocation of a plurality of remote support operators to a plurality of vehicles that issue remote support requests.

[0022] The operator management program relating to the seventh disclosure is characterized in that it is configured to cause the computer to execute the following processes: a process of setting or updating an allocation priority for each of a plurality of vehicles according to the content of a remote support request; and a process of assigning a remote support operator assigned to a second vehicle to a first vehicle when the number of the plurality of vehicles is greater than the number of a plurality of remote support operators and the allocation priority of a first vehicle which is one of the plurality of vehicles to which a plurality of remote support operators has not been assigned is higher than the allocation priority of a second vehicle which is one of the plurality of vehicles to which a plurality of remote support operators has been assigned. Effect of the Invention

[0023] According to the present disclosure, an allocation priority is set or updated for each of the multiple vehicles according to the content of the remote support request. When the number of the multiple vehicles is greater than the number of the multiple remote support operators, and when the allocation priority of a first vehicle to which a remote support operator has not been assigned is higher than the allocation priority of a second vehicle to which a remote support operator has been assigned, the remote support operator assigned to the second vehicle is assigned to the first vehicle. This allows the multiple remote support operators to be assigned so as to prioritize vehicles with a high degree of urgency in remote support. In addition, the multiple remote support operators can be appropriately assigned based on the viewpoints of speeding up emergency response and smoothing traffic flow. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a conceptual diagram for explaining an overview of an operator management system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a conceptual diagram showing an example of an operation plan. [Diagram 3] 1 is a conceptual diagram for explaining an overview of an operator management function provided by an operator management system according to an embodiment of the present invention. [Figure 4]1 is a conceptual diagram for explaining an overview of an operator management function provided by an operator management system according to an embodiment of the present invention. [Diagram 5] 1 is a conceptual diagram for explaining an overview of an operator management function provided by an operator management system according to an embodiment of the present invention. [Figure 6] 1 is a block diagram showing a configuration of a vehicle according to an embodiment of the present invention; [Figure 7] 2 is a block diagram showing a configuration of a process executed in the vehicle according to the embodiment; FIG. [Figure 8] FIG. 2 is a block diagram showing a configuration of a management server according to the present embodiment. [Figure 9] 3 is a block diagram showing a configuration of a process executed in a management server according to the embodiment; FIG. [Figure 10] 1 is a block diagram showing a configuration of a remote assistance device according to an embodiment of the present invention. [Figure 11] 2 is a block diagram showing a configuration of a process executed in the remote assistance device according to the present embodiment. FIG. [Figure 12] FIG. 4 is a sequence diagram showing a process executed in the vehicle according to the embodiment. [Figure 13] FIG. 4 is a sequence diagram showing a process executed in the management server according to the embodiment. [Figure 14] FIG. 4 is a sequence diagram showing a process executed in the management server according to the embodiment. [Figure 15] FIG. 4 is a sequence diagram showing a process executed in the remote assistance device according to the embodiment. [Figure 16] FIG. 4 is a sequence diagram showing a process executed in the remote assistance device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] 1. Overview An overview of the operator management system according to this embodiment will be described below. Fig. 1 is a conceptual diagram for explaining the overview of the operator management system 10 according to this embodiment. The operator management system 10 provides a function (hereinafter also referred to as "operator management function") for managing the allocation of multiple remote support operators 2 to multiple vehicles 1 that issue remote support requests.

[0026] Each of the multiple vehicles 1 is typically an autonomous vehicle. In this case, each of the multiple vehicles 1 transmits a remote support request when it encounters a driving decision that the autonomous driving system is not good at or when there is a high possibility of encountering such a driving decision. Each of the multiple vehicles 1 requests a driving decision from the remote support operator 2 by the remote support request. The driving decisions required of the remote support operator 2 (hereinafter also referred to as "support request items") include, for example, the timing of lane changes and merging at a merging point, adjustment of parking position when parking, and subsequent response when running off the road shoulder is confirmed. Note that the remote support request may include multiple support request items.

[0027] Each of the multiple remote support operators 2 corresponds to a remote support device 200. The remote support device 200 connects to and communicates with a vehicle 1 that is a target of remote support. Here, the connection between the remote support device 200 and the vehicle 1 is typically realized via a mobile communication network and the Internet. The remote support device 200 also has a function of notifying the remote support operator 2 of the status of the connected vehicle 1 (contents of a remote support request, driving video, surrounding environment, vehicle state, etc.) and a function of accepting input of the driving judgment of the remote support operator 2. In other words, the remote support operator 2 confirms the status of the vehicle 1 from the notification of the remote support device 200 and inputs a driving judgment to the remote support device 200 to perform remote support of the vehicle 1.

[0028] In the operator management system 10 according to this embodiment, the operator management function is realized by the management server 100. The management server 100 is configured to be able to communicate information with each of the multiple vehicles 1 and the remote support devices 200 corresponding to each of the multiple remote support operators 2. For example, the management server 100 and the remote support devices 200 are configured to be connected to the Internet via a communication cable, and each of the multiple vehicles 1 is configured to be connected to the Internet via a mobile communication network. Alternatively, the management server 100 and the remote support devices 200 may be connected by a local area network.

[0029] The management server 100 determines the allocation of multiple remote support operators 2 to multiple vehicles 1 based on remote support requests acquired from multiple vehicles 1 and the support statuses of the multiple remote support operators 2. Here, the support status is acquired from each remote support device 200, and includes at least information on whether the corresponding remote support operator 2 is providing remote support (whether the corresponding remote support operator 2 is available or not). The support status may further include information on the status of processing of the remote support request. For example, when a remote support request includes multiple support request items, the support status may include information on whether each of the multiple support request items has been processed.

[0030] The management server 100 issues an allocation instruction to the remote support device 200 corresponding to each of the multiple remote support operators 2 based on the determined allocation. Here, the allocation instruction includes at least information specifying the vehicle 1 to be allocated. For example, the allocation instruction includes information of a vehicle ID that identifies the vehicle 1 to be allocated. The remote support device 200 starts communication with the vehicle 1 that is the target of remote support according to the allocation instruction.

[0031] The management server 100 also transmits the allocation status to the multiple vehicles 1. The allocation status includes, for example, information on the allocation result as to whether the allocation of a remote support operator 2 to a remote support request was successful. This allows each of the multiple vehicles 1 to obtain the response status of remote support to the remote support request that was transmitted. Each of the multiple vehicles 1 may be configured to execute a predetermined process according to the allocation status.

[0032] For example, each of the multiple vehicles 1 may be configured to receive an allocation result indicating that the allocation of the remote support operator 2 has been successful and start communication with the remote support device 200. Alternatively, each of the multiple vehicles 1 may be configured to receive an allocation result indicating that the allocation has been put on hold due to a shortage of remote support operators 2 and start driving control to ensure safety and smooth traffic flow in the vicinity until the allocation is made.

[0033] The operator management system 10 according to the present embodiment is configured in this manner. When each of the multiple vehicles 1 is an autonomous vehicle, each of the multiple vehicles 1 typically travels according to a driving plan that is a combination of multiple driving decisions. In this case, each of the multiple vehicles 1 may be configured to transmit the driving plan as a remote support request in response to the inclusion of one or more support request items in the multiple driving decisions related to the driving plan. Furthermore, the vehicle 1 may be configured to transmit the driving plan even in a case where the surrounding traffic conditions or recognition conditions change after transmitting the driving plan as a remote support request and some support request items related to the driving plan are withdrawn. By configuring in this manner, the remote support request and the driving plan can be handled in an integrated manner. In addition, a system with a simple configuration can be realized.

[0034] FIG. 2 is a conceptual diagram showing an example of a driving plan. A driving plan generally specifies driving decisions within a certain distance range from the vehicle's position based on information detected by a sensor or the like. In addition, the driving plan is updated at every predetermined control period. That is, the driving plan specifies driving decisions within a certain distance range from the vehicle's position from moment to moment while the vehicle 1 is traveling. In the example shown in FIG. 2, the driving plan includes three assistance request items as driving decisions in road sections B, D, and F. In this way, the driving plan can be treated as a remote assistance request.

[0035] The operator management function provided by the operator management system 10 of this embodiment is characterized in that it sets or updates an allocation priority for each of the multiple vehicles 1 according to the content of the remote support request, and changes the allocation of multiple remote support operators 2 based on the allocation priority of each of the multiple vehicles 1.

[0036] Here, the allocation priority of each of the multiple vehicles 1 can be calculated as follows by the management server 100. Note that the following describes a case where the allocation priority of one of the multiple vehicles 1 is calculated.

[0037] First, the management server 100 calculates the support priority for the support request items related to the remote support request transmitted by the vehicle 1. The support priority is the degree of urgency of the support request item. The degree of urgency may be indicated by indicators such as safety and smooth traffic flow in the vicinity. The support priority is typically expressed as a numerical value. Note that when the remote support request includes multiple support request items, the management server 100 calculates the support priority for each of the multiple support request items.

[0038] Here, the management server 100 may be configured to calculate a support priority level to be stored in advance for each support request item. For example, the management server 100 stores support priorities for each support request item as shown in Table 1 below. At this time, the management server 100 calculates a numerical value corresponding to the support request item in Table 1 as the support priority level. For example, when the support request item is a lane change, the calculated support priority level is 5. Also, for example, when the vehicle 1 transmits the driving plan shown in FIG. 2 as a remote support request, the calculated support priorities for the three support request items are 5 (lane change), 10 (crossing the sidewalk), and 8 (adjusting parking position).

[0039] [Table 1]

[0040] Next, the management server 100 calculates the allocation priority of the vehicle 1 based on the calculated support priority. When there is one support request item related to the remote support request, the management server 100 may set the calculated support priority as the allocation priority of the vehicle 1. When the remote support request includes multiple support request items, the management server 100 may set the highest support priority among the support priorities calculated for each of the multiple support request items as the allocation priority of the vehicle 1. For example, when the vehicle 1 transmits the driving plan shown in FIG. 2 as a remote support request, the management server 100 may set the allocation priority of the vehicle 1 to 10.

[0041] In this manner, the management server 100 can calculate the allocation priority for each of the multiple vehicles 1. The allocation priority can also be considered as the degree of urgency of the remote assistance for the vehicle 1.

[0042] 3 to 5, an overview of the operator management function provided by the operator management system 10 will be described below. In the following description, the number of remote support operators 2 is assumed to be three.

[0043] First, refer to Fig. 3. Fig. 3 shows a case where a new remote support request is received when multiple remote support operators 2 are available. In Fig. 3, symbols (a, b, c) are added to the reference signs to distinguish each of the multiple remote support operators 2 and the multiple vehicles 1.

[0044] 3, the remote support operator 2c is available and the vehicle 1c newly issues a remote support request, in which case the management server 100 assigns the remote support operator 2c to the vehicle 1c.

[0045] Next, refer to Fig. 4. Fig. 4 shows a case where a new remote support request is acquired after the allocation shown in Fig. 3, when there are no available remote support operators 2. That is, the number of the multiple vehicles 1 is greater than the number of the multiple remote support operators 2. In Fig. 4, vehicle 1d newly transmits a remote support request. However, since there are no available remote support operators 2, allocation cannot be simply performed as shown in Fig. 3.

[0046] Here, the management server 100 sets or updates the allocation priority for each of the multiple vehicles 1, as shown in Fig. 4. Then, when the allocation priority of a vehicle 1 (first vehicle) to which a remote support operator 2 has not been assigned is higher than the allocation priority of a vehicle 1 (second vehicle) to which a remote support operator 2 has been assigned, the management server 100 changes the allocation of the multiple remote support operators 2 so as to assign the remote support operator 2 assigned to the second vehicle to the first vehicle. In the example shown in Fig. 4, since the allocation priority of vehicle 1d is higher than the allocation priority of vehicle 1c, the management server 100 changes the allocation of the multiple remote support operators 2 so as to assign the remote support operator 2c assigned to vehicle 1c to vehicle 1d.

[0047] The second vehicle to which the allocation of the remote support operator 2 is changed may be the second vehicle with the lowest allocation priority. Even if the second vehicle to which the allocation of the remote support operator 2 is changed is not the vehicle 1 with the lowest allocation priority, the same allocation situation will occur if the change in allocation is repeated successively. In addition, when there are multiple first vehicles with the same allocation priority, the management server 100 may be configured to select the first vehicle to which the allocation is changed based on a specific index. Examples of the specific index include the length of time during which no allocation has been performed, the narrowness of the road on which the vehicle 1 is located, the degree of deviation from the operation schedule, and the like. This makes it possible to select a first vehicle that is more in need of remote support depending on the situation of the vehicle 1.

[0048] Next, FIG. 5 is referred to. FIG. 5 shows a case where a part of the support request items is processed for one of the multiple vehicles 1 after the change of allocation shown in FIG. 4. In FIG. 5, the support request item "crossing the sidewalk" for the vehicle 1b is processed. At this time, the management server 100 updates the allocation priority for each of the multiple vehicles 1. As a result, the allocation priority of the vehicle 1b is changed from 10 to 5 as shown in FIG. 5. Then, when the allocation priority of the vehicle 1 (first vehicle) to which the remote support operator 2 is not assigned is higher than the allocation priority of the vehicle 1 (second vehicle) to which the remote support operator 2 is assigned, the management server 100 changes the allocation of the multiple remote support operators 2 so as to assign the remote support operator 2 assigned to the second vehicle to the first vehicle. In FIG. 5, the allocation priority of the vehicle 1c is higher than the allocation priority of the vehicle 1b, so that the management server 100 changes the allocation of the multiple remote support operators 2 so as to assign the remote support operator 2b assigned to the vehicle 1b to the vehicle 1c.

[0049] 2. Configuration Hereinafter, regarding the operator management system 10 according to the present embodiment, configurations of the vehicle 1, the management server 100, and the remote support device 200 will be described. In the following description, it is assumed that the vehicle 1 is an autonomous vehicle that transmits a driving plan as a remote support request.

[0050] 2-1. Self-driving cars First, the configuration of the vehicle 1 (autonomous driving vehicle) will be described. Fig. 6 is a block diagram showing the configuration of the vehicle 1. The vehicle 1 includes one or more sensors 310, one or more communication devices 320, ECUs 330, and one or more actuators 340. The ECUs 330 are configured to be able to transmit information to and from the one or more sensors 310, the one or more communication devices 320, and the one or more actuators 340. Typically, each is connected to an in-vehicle network configured with a CAN (Controller Area Network) or the like.

[0051] The one or more sensors 310 include at least a camera that captures an image of the surroundings of the vehicle 1. Other examples of the one or more sensors 310 include sensors that detect the surrounding environment of the vehicle 1 (preceding vehicle, white lines, obstacles, etc.) and sensors that detect the running state of the vehicle 1 (vehicle speed, acceleration, yaw rate, etc.). Sensors that detect the surrounding environment of the vehicle 1 include, for example, millimeter wave radar, LiDAR (Light Detection And Ranging), etc. Sensors that detect the running state of the vehicle 1 include, for example, a vehicle speed sensor, a G sensor, a gyro sensor, etc. Information detected by the one or more sensors 310 is transmitted to the ECUs 330.

[0052] The one or more communication devices 320 communicate with devices outside the vehicle 1 to transmit and receive information. In particular, the one or more communication devices 320 include a device (typically, a terminal that performs wireless communication with a wireless base station) for communicating with the management server 100 and the remote support device 200. That is, the one or more communication devices 320 acquire the allocation status and the driving judgment by the remote support operator 2, and the like. Other examples of the one or more communication devices 320 include a device that communicates with surrounding vehicles, a GPS receiver, and the like. The information received by the one or more communication devices 320 is transmitted to the ECUs 330. Examples of the information received by the one or more communication devices 320 include the allocation status, the driving judgment by the remote support operator 2, map information, road traffic information, GPS position information, and the like.

[0053] The ECUs 330 are electronic control units (ECUs) that execute processes related to various controls of the vehicle 1 and generate control signals based on acquired information. The ECUs 330 include an autonomous driving ECU 331 and a vehicle control ECU 332.

[0054] The autonomous driving ECU 331 executes processing related to autonomous driving of the vehicle 1. In particular, the autonomous driving ECU 331 executes processing to generate a driving plan based on detection information from one or more sensors 310 and received information from one or more communication devices 320. The vehicle control ECU 332 executes processing to generate control signals for one or more actuators 340 so that the vehicle 1 performs a desired operation. In particular, the vehicle control ECU 332 executes processing to generate control signals for operating the vehicle 1 according to the driving plan generated in the autonomous driving ECU 331. The control signals generated by the vehicle control ECU 332 are transmitted to one or more actuators 340.

[0055] The one or more actuators 340 operate according to control signals transmitted from the ECUs 330. Examples of the one or more actuators 340 include actuators related to the operation of a power plant (internal combustion engine, electric motor, etc.), actuators related to the operation of a brake mechanism, actuators related to the operation of a steering mechanism, etc. The one or more actuators 340 operate according to control signals, thereby realizing automatic driving of the vehicle 1.

[0056] Next, the configuration of the process executed in the vehicle 1 according to this embodiment will be described with reference to Fig. 7. The process executed in the vehicle 1 is configured by a video data transmission processing unit P321, a driving plan transmission processing unit P322, an operator assignment reception processing unit P323, a driving decision reception processing unit P324, a sensor information processing unit P331, an object recognition processing unit P332, a behavior prediction processing unit P333, a driving route generation processing unit P334, and a driving plan generation processing unit P335.

[0057] Here, the sensor information processing unit P331, the object recognition processing unit P332, the behavior prediction processing unit P333, the driving route generation processing unit P334, and the driving plan generation processing unit P335 are realized in the autonomous driving ECU 331. In addition, the video data transmission processing unit P321, the driving plan transmission processing unit P322, the operator assignment reception processing unit P323, and the driving judgment reception processing unit P324 are realized in one or more communication devices 320.

[0058] The sensor information processing unit P331 processes information acquired from one or more sensors 310, and extracts data relating to objects around the vehicle 1.

[0059] The object recognition processing unit P332 integrates information about objects around the vehicle 1 obtained from one or more sensors 310, and determines the type of object around the vehicle 1 (e.g., person, vehicle, traffic light, ground, sky, etc.).

[0060] The behavior prediction processing unit P333 predicts the future movement paths of objects around the vehicle 1.

[0061] The travel route generation processing unit P334 generates a travel route to the destination. Here, the travel route may be information that specifies a travel route.

[0062] The driving plan generation processing unit P335 determines driving decisions (e.g., lane change, starting, stopping, etc.) based on the processing result of the behavior prediction processing unit P333 and the driving route generated by the driving route generation processing unit P334, and generates a driving plan that is a combination of multiple driving decisions. In particular, the driving plan generation processing unit P335 sets driving decisions that are difficult for the autonomous driving system alone as support request items. In addition, when the driving plan generation processing unit P335 obtains the driving decision of the remote support operator 2, it updates the driving plan according to the driving decision of the remote support operator 2.

[0063] The video data transmission processing unit P321 transmits video data acquired from a camera equipped in the vehicle 1 to the remote support device 200. Here, the video data transmission processing unit P321 may incorporate the recognition result of objects around the vehicle 1 into the video data to be transmitted based on the processing result of the object recognition processing unit P332. Alternatively, the video data transmission processing unit P321 may be configured to transmit modeled information such as "a person is at this position" based on the processing result of the object recognition processing unit P332.

[0064] In response to the fact that the driving judgment related to the operation plan includes an assistance request item, the operation plan transmission processing unit P322 transmits the operation plan generated by the operation plan generation processing unit P335.

[0065] The operator allocation reception process unit P323 receives the allocation status of a plurality of remote support operators 2 from the management server 100.

[0066] The driving decision receiving process unit P324 receives the driving decision of the remote support operator 2 from the remote support device 200.

[0067] 2-2. Management Server Next, a description will be given of the configuration of the management server 100. Fig. 8 is a block diagram showing the configuration of the management server 100. The management server 100 is a computer including a memory 110, a processor 120, and a communication device .

[0068] The memory 110 is coupled to the processor 120 and stores a plurality of instructions 112 executable by the processor 120, and various data 113 required for the execution of processing. Here, the plurality of instructions 112 are provided by a computer program 111 (an operator management program).

[0069] The instructions 112 are configured to cause the processor 120 to execute operations related to operator managed functions. In particular, the processor 120 operates in accordance with the instructions 112 to implement the operations related to operator managed functions.

[0070] The communication device 130 communicates with devices external to the management server 100 to transmit and receive information. In particular, the communication device 130 communicates with the vehicle 1 and the remote support device 200. That is, the communication device 130 obtains remote support requests and support status, and transmits allocation status and allocation instructions. The information received by the communication device 130 is stored in the memory 110 as data 113.

[0071] Next, the configuration of the process executed in the management server 100 according to this embodiment will be described with reference to Fig. 9. The process executed in the management server 100 is configured by an allocation priority calculation processing unit P121, a support status management processing unit P122, an operator allocation calculation processing unit P123, an operation plan reception processing unit P131, an operator allocation transmission processing unit P132, and a support status reception processing unit P133.

[0072] Here, the allocation priority calculation processing unit P121, the assistance status management processing unit P122, and the operator allocation calculation processing unit P123 are realized by the processor 120 that operates according to a plurality of instructions 112. In addition, the operation plan reception processing unit P131, the operator allocation transmission processing unit P132, and the assistance status reception processing unit P133 are realized in the communication device 130.

[0073] The allocation priority calculation processing unit P121 calculates the allocation priority for each of the multiple vehicles 1 based on the driving plans of the multiple vehicles 1 and the support status of the multiple remote support operators 2. Here, the calculation of the allocation priority is performed as described above. In addition, the allocation priority calculation processing unit P121 may be configured to calculate the allocation priority when the management server 100 acquires a new remote support request or when some of the support request items have been processed.

[0074] The support status management processing unit P122 manages the support status of the multiple remote support operators 2 in association with the allocation priority of each of the multiple vehicles 1. That is, the support status management processing unit P122 For each of the multiple remote support operators 2, the processor P122 manages what type of remote support request for which vehicle 1 the operator is processing, and the allocation priority of the vehicle 1 that is the target of remote support. For example, the support status management processor P122 manages a support status table such as that shown in Table 2 below as data 113.

[0075] [Table 2]

[0076] The operator allocation calculation processing unit P123 calculates the allocation of the multiple remote support operators 2 based on the support status of the multiple remote support operators 2 and the allocation priority of the multiple vehicles 1. Here, the allocation of the multiple remote support operators 2 is performed as described with reference to Fig. 3 to Fig. 5. The operator allocation calculation processing unit P123 may be configured to calculate the allocation of the multiple remote support operators 2 when the management server 100 acquires a new remote support request or when some of the support request items have been processed.

[0077] The driving plan reception processor P131 receives a driving plan transmitted by the vehicle 1.

[0078] The operator allocation transmission processing unit P132 transmits the allocation result to the vehicle 1 that has issued the remote support request (driving plan). Here, the allocation result may be configured to be acquired from the operator allocation calculation processing unit P123. The operator allocation transmission processing unit P132 also transmits information (vehicle ID) that identifies the vehicle 1 that has been assigned to the remote support device 200. Here, the information that identifies the vehicle 1 may be configured to be acquired from the support status management processing unit P122.

[0079] The support status reception process unit P133 receives the support status from the remote support device 200 corresponding to each of the multiple remote support operators 2.

[0080] 2-3. Remote support device Next, a description will be given of the configuration of the remote support device 200. Fig. 10 is a block diagram showing the configuration of the remote support device 200. The remote support device 200 includes an HMI 210, a processing device 220, and a communication device 230.

[0081] The HMI 210 provides an HMI function for the remote support operator 2. The HMI 210 includes a driving decision input unit 211 and an image display unit 212. The driving decision input unit 211 accepts input of driving decisions of the remote support operator 2. The driving decision input unit 211 is realized by, for example, a mouse, a keyboard, a touch panel, an operation panel, or the like. The image display unit 212 displays information about the vehicle 1 that is the target of remote support. Typically, the image display unit 212 displays image data of the surroundings of the vehicle 1 that is the target of remote support, and information on a remote support request (driving plan) for the vehicle 1 that is the target of remote support. The image display unit 212 is typically a display.

[0082] The remote support operator 2 checks the state of the vehicle 1 from the display on the video display unit 212. Then, the remote support operator 2 operates the driving decision input unit 211 to input a driving decision in response to the remote support request.

[0083] The processing device 220 is a computer that executes various processes based on the acquired information. In particular, the processing device 220 executes processes related to control of the display of the video display unit 212.

[0084] The communication device 230 communicates with devices external to the remote support device 200 to transmit and receive information. In particular, the communication device 230 communicates with the vehicle 1 and the management server 100. That is, the communication device 230 obtains allocation instructions, transmits driving judgments and support situations by the remote support operator 2, and the like. The information received by the communication device 230 is transmitted to the processing device 220.

[0085] Next, referring to Fig. 11, a configuration of the process executed in the remote support device 200 according to the present embodiment will be described. The process executed in the remote support device 200 includes a video generation processing unit P221, a support status transmission processing unit P231, a driving decision transmission processing unit P232, a video data reception processing unit P233, an operation plan reception processing unit P234, and an operator allocation reception processing unit P235. Here, the video generation processing unit P221 is realized in the processing device 220. In addition, the support status transmission processing unit P231, the driving decision transmission processing unit P232, the video data reception processing unit P233, the operation plan reception processing unit P234, and the operator allocation reception processing unit P235 are realized in the communication device 230.

[0086] The video generation processing unit P221 generates a display signal for displaying an image on the video display unit 212 based on the video data acquired from the vehicle 1 that is the target of remote support and the driving plan. The video generation processing unit P221 generates a display signal for displaying an image that combines the acquired video data with information such as the predicted driving trajectory of the vehicle 1, the support request items, and the positions and sizes of surrounding objects. This allows the remote support operator 2 to fully check the situation around the vehicle 1 by checking the video display unit 212.

[0087] The support status transmission processor P231 transmits the support status. Here, the support status transmitted by the support status transmission processor P231 may include information on support request items that have not yet been processed for the vehicle 1 that is the target of remote support.

[0088] The driving decision transmission processing unit P232 transmits the driving decision of the remote support operator 2 that the driving decision input unit 211 has accepted.

[0089] The video data reception processing unit P233 receives video data of the vehicle 1 that is the target of remote support. Here, the video data received by the video data reception processing unit P233 may include recognition information of the surroundings of the vehicle 1, such as the positions and sizes of objects around the vehicle 1.

[0090] The driving plan reception process unit P234 receives a driving plan of the vehicle 1 that is a target of remote support.

[0091] The operator assignment reception processing unit P235 receives information (vehicle ID) that identifies the vehicle 1 that is the target of remote support. When the operator assignment reception processing unit P235 receives the vehicle ID, the remote support device 200 starts remote support of the vehicle 1 identified by the vehicle ID. Conversely, when the operator assignment reception processing unit P235 does not receive a vehicle ID, the remote support operator 2 is available.

[0092] 3. Processing Hereinafter, with regard to the operator management system 10 according to this embodiment, the processes executed in the vehicle 1, the management server 100, and the remote assistance device 200 will be described.

[0093] 2-1. Self-driving cars First, the process executed in the vehicle will be described with reference to FIG 12. FIG 12 is a sequence diagram showing the process executed in the vehicle 1.

[0094] The operation plan generation processing unit P335 periodically updates the operation plan based on the acquired information (T310).

[0095] Then, when a support request item is added to the operation judgment related to the operation plan (T320), the operation plan transmission processing unit P322 transmits the operation plan. After that, the operator assignment reception processing unit P323 waits to receive the assignment result.

[0096] When the operator allocation receiving processing unit P323 receives an allocation result indicating that the allocation of the remote support operator 2 has been successful, the vehicle 1 starts communication with the remote support device 200, and the driving judgment receiving processing unit P324 waits to receive a driving judgment by the remote support operator 2 (not shown in FIG. 12).

[0097] On the other hand, when the operator allocation reception processing unit P323 receives an allocation result indicating that the allocation of the remote support operator 2 has been put on hold, or when the allocation result is not received for a certain period of time and a timeout occurs, the driving plan generation processing unit P335 changes the driving plan. At this time, the driving plan generation processing unit P335 may be configured to change the driving plan to one that can ensure safety and smooth traffic flow in the surrounding area until the allocation is made. For example, the driving plan generation processing unit P335 may change the driving plan to one that includes low-speed driving of the vehicle 1 and stopping of the vehicle 1 on the road shoulder in the driving judgment. Alternatively, the driving plan generation processing unit P335 may be configured to change the policy for generating the driving plan. For example, the driving plan generation processing unit P335 changes the policy to one that generates a driving plan that emphasizes safety.

[0098] 2-2. Management Server Next, the processing executed in the management server 100 will be described. Fig. 13 and Fig. 14 are sequence diagrams showing the processing executed in the management server 100. Here, Fig. 13 and Fig. 14 are connected at "A" and show one sequence diagram. In Fig. 13 and Fig. 14, the support status management processing unit P122 manages the support status of multiple remote support operators 2 using a support status table such as that shown in Table 2.

[0099] The support status management processing unit P122 periodically updates the support status table with respect to the allocation priority (T110). At this time, the support status management processing unit P122 acquires the support status received by the support status reception processing unit P133, and notifies the acquired support status to the allocation priority calculation processing unit P121. Upon receiving the notification of the support status, the allocation priority calculation processing unit P121 calculates an allocation priority for each of the multiple vehicles. Then, the support status management processing unit P122 updates the support status table based on the allocation priority calculated by the allocation priority calculation processing unit P121.

[0100] Upon receiving an operation plan (remote support request), the operation plan reception processing unit P131 notifies the received operation plan to the operator allocation calculation processing unit P123. Upon receiving the operation plan, the operator allocation calculation processing unit P123 acquires the support status from the support status management processing unit P122. Then, the operator allocation calculation processing unit P123 checks whether multiple remote support operators 2 are available.

[0101] When there are vacant remote support operators 2 (T120), the operator allocation calculation processing unit P123 assigns the available remote support operator 2 to the vehicle 1 that newly transmits the driving plan. Then, the operator allocation calculation processing unit P123 notifies the operator allocation transmission processing unit P132 of the allocation result indicating that the allocation of the remote support operator 2 has been successful. The operator allocation transmission processing unit P132 transmits the notified allocation result to the vehicle 1. The operator allocation calculation processing unit P123 also notifies the support status management processing unit P122 of information (support information) about the vehicle 1 that has been assigned. The support information includes the vehicle ID, the support request item, and information about the remote support operator 2 to be assigned. Furthermore, the support information may include information on the allocation priority of the vehicle 1 that has been assigned. In this case, the processing is executed in the allocation priority calculation processing unit P121. The support status management processing unit P122 receives the notification of the support information and updates the support status table based on the support information. Then, the support status management processing unit P122 notifies the operator allocation transmission processing unit P132 of the vehicle ID of the vehicle 1 that has been assigned. The operator allocation transmission process unit P132 transmits the notified vehicle ID to the remote support device 200.

[0102] If there is no availability of remote support operator 2 (T130), the operator allocation calculation processing unit P123 notifies the allocation priority calculation processing unit P121. Upon receiving the notification, the allocation priority calculation processing unit P121 calculates the allocation priority of each of the multiple vehicles 1 including the vehicle 1 that newly transmits the driving plan. Then, the operator allocation calculation processing unit P123 checks the calculated allocation priority of each of the multiple vehicles 1.

[0103] When the allocation priority of the vehicle 1 that newly transmits the driving plan is higher than the allocation priority of the vehicle 1 (second vehicle) to which the remote support operator 2 is assigned (T131), the operator allocation calculation processing unit P123 changes the allocation of the multiple remote support operators 2 so as to assign the remote support operator 2 of the second vehicle to the vehicle 1 that newly transmits the driving plan. After that, the operator allocation calculation processing unit P123 notifies the operator allocation transmission processing unit P132 of the allocation result. Here, the allocation result notified by the operator allocation calculation processing unit P123 indicates that the allocation of the remote support operator 2 has been successful for the vehicle 1 that newly transmits the driving plan, and indicates that the allocation of the remote support operator 2 is removed for the second vehicle. The operator allocation transmission processing unit P132 transmits the notified allocation result to the vehicle 1 that newly transmits the driving plan and the second vehicle. In addition, the operator allocation calculation processing unit P123 notifies the support status management processing unit P122 of the support information for the vehicle 1 whose allocation has been changed. Upon receiving the notification of the support information, the support status management processing unit P122 updates the support status table based on the support information. Then, the support status management processing unit P122 notifies the operator assignment transmission processing unit P132 of the vehicle ID of the assigned vehicle 1. The operator assignment transmission processing unit P132 transmits the notified vehicle ID to the remote support device 200.

[0104] When the allocation priority of the vehicle 1 that newly transmits a driving plan is equal to or lower than the allocation priority of the vehicle 1 to which the remote support operator 2 is assigned (T132), the operator allocation calculation processing unit P123 reserves the allocation of the remote support operator 2 to the vehicle 1 that newly transmits a driving plan. After that, the operator allocation calculation processing unit P123 notifies the operator allocation transmission processing unit P132 of an allocation result indicating that the allocation of the remote support operator 2 has been reserved. The operator allocation transmission processing unit P132 transmits the notified allocation result to the vehicle 1 that newly transmits a driving plan. In addition, the operator allocation calculation processing unit P123 notifies the support status management processing unit P122 of support information about the vehicle 1 that newly transmits a driving plan. The support status management processing unit P122 receives the notification of the support information and updates the support status table based on the support information.

[0105] Thereafter, the support status management processing unit P122 periodically updates the support status table with respect to the allocation priority (T110). Then, when the number of the multiple vehicles 1 is greater than the number of the multiple remote support operators 2 (T130), processing related to T131 or T132 is executed for the vehicle 1 (first vehicle) to which a remote support operator 2 has not been assigned and the vehicle 1 (second vehicle) to which a remote support operator 2 has been assigned. In other words, when the allocation priority of the first vehicle to which a remote support operator 2 has not been assigned is higher than the allocation priority of the second vehicle to which a remote support operator 2 has been assigned, the remote support operator 2 of the second vehicle is assigned to the first vehicle.

[0106] In this manner, the processing is executed in the management server 100. In this manner, an operator management method for managing the allocation of a plurality of remote support operators 2 to a plurality of vehicles 1 that issue remote support requests is realized by the management server 100. Alternatively, the operator management method can be realized as an operator management program that causes a computer to execute the operator management method.

[0107] 3-3. Remote support device Next, a process executed in the remote support device 200 will be described. Fig. 15 and Fig. 16 are sequence diagrams showing the process executed in the remote support device 200. Here, Fig. 15 shows the process when a remote support operator is assigned by the management server 100. Fig. 16 shows the process when the remote support operator 2 inputs a driving decision.

[0108] First, refer to Fig. 15. When the operator allocation reception processing unit P235 receives the vehicle ID, the remote support device 200 starts communication with the vehicle 1 indicated by the received vehicle ID. The operator allocation reception processing unit P235 notifies the operation plan reception processing unit P234 and the video data reception processing unit P233 of the received vehicle ID.

[0109] While the remote support operator 2 is being assigned, the following process is executed (T210). The driving plan reception processing unit P234 receives a driving plan from the vehicle 1 that is the target of remote support. Then, the driving plan reception processing unit P234 notifies the video production processing unit P221 and the support status transmission processing unit P231 of the received driving plan. The video data reception processing unit P233 receives video data from the vehicle 1 that is the target of remote support. Then, the video data reception processing unit P233 notifies the video production processing unit P221 of the received video data. The video production processing unit P221 generates a display signal based on the notified driving plan and video data. Then, the video production processing unit P221 notifies the video display unit 212 of the display signal. The video display unit 212 displays the video according to the notified display signal.

[0110] Upon receiving notification of the driving plan, the support status transmission processing unit P231 checks the support request items related to the driving plan. Here, if the support request items are processed as a result of the remote support operator 2 inputting a driving judgment, the driving plan is updated in the vehicle 1. In other words, by checking the support request items related to the driving plan, it is possible to confirm whether the support request items have been processed. Then, when it is confirmed that the support request items have been processed (T220), the support status transmission processing unit P231 transmits the support status.

[0111] Next, refer to Fig. 16. When the driving judgment input unit 211 accepts the input of the driving judgment of the remote support operator 2, it notifies the driving judgment transmission processing unit P232 of the accepted driving judgment. The driving judgment transmission processing unit P232 transmits the notified driving judgment to the vehicle 1 that is the target of remote support. The processes related to T210 and T220 are the same as those described in Fig. 15.

[0112] 4.Effects As described above, according to this embodiment, the allocation priority is set or updated for each of the multiple vehicles 1 according to the content of the remote support request. When the number of the multiple vehicles 1 is greater than the number of the multiple remote support operators 2, and when the allocation priority of the first vehicle to which the remote support operator 2 is not assigned is higher than the allocation priority of the second vehicle to which the remote support operator 2 is assigned, the remote support operator 2 assigned to the second vehicle is assigned to the first vehicle. This allows the multiple remote support operators 2 to be assigned so that the vehicle 1 with a high degree of urgency of remote support is processed with priority. In addition, the multiple remote support operators 2 can be appropriately assigned based on the viewpoints of speeding up response in an emergency and smoothing traffic flow.

[0113] 5. Variations The operator management system 10 according to this embodiment may employ the following modified aspects.

[0114] The management server 100 may be configured to adjust the allocation priority calculated based on the support priority based on a specific index. For example, the management server 100 may be configured to further multiply the allocation priority calculated based on the support priority by using the length of the period during which the remote support operator 2 has not been assigned as an index. This makes it possible to increase the allocation priority of the first vehicle to which the remote support operator 2 has not been assigned the longer the period during which no assignment has been performed. In turn, it is possible to avoid a situation in which the remote support operator 2 is never assigned.

[0115] Other indicators that may be used include the narrowness of the road on which the vehicle 1 is located, the degree of deviation from the operation schedule, the situation around the vehicle 1, and the like. [Explanation of symbols]

[0116] 1 vehicle 2. Remote Support Operator 10 Operator Management System 100 Management Server 110 Memory 111 Computer Programs (Operator Management Programs) 112 Instructions 120 processors

Claims

1. An operator management system that manages the allocation of a plurality of remote support operators to a plurality of vehicles that transmit a remote support request including one or more support request items, comprising: one or more processors; a memory coupled to the one or more processors and storing a plurality of instructions executable by the one or more processors; Including, The instructions may include instructions for causing the one or more processors to: setting or updating an allocation priority for each of the plurality of vehicles according to the one or more assistance request items of the remote assistance request; When the number of the plurality of vehicles is greater than the number of the plurality of remote support operators, and the allocation priority of a first vehicle which is one of the plurality of vehicles to which the plurality of remote support operators have not been assigned is higher than the allocation priority of a second vehicle which is one of the plurality of vehicles to which the plurality of remote support operators have been assigned, a process of allocating the remote support operator assigned to the second vehicle to the first vehicle; It is configured to run 1. An operator management system comprising:

2. The operator management system according to claim 1 , The process of setting or updating the allocation priority includes: and in response to a newly acquired remote assistance request, setting or updating the allocation priority for each of the plurality of vehicles.

1. An operator management system comprising:

3. The operator management system according to claim 1 or 2, The process of setting or updating the allocation priority includes: Calculating a support priority for each of the one or more support request items; setting or updating the allocation priority based on the assistance priority; Includes 1. An operator management system comprising:

4. The operator management system according to claim 3, The process of setting or updating the allocation priority includes: and setting or updating the allocation priority for each of the plurality of vehicles in response to a portion of the one or more assistance request items being processed by the remote assistance operator.

1. An operator management system comprising:

5. The operator management system according to claim 4, Each of the plurality of vehicles includes: An autonomous vehicle that drives according to a driving plan that is a combination of multiple driving decisions. In response to the plurality of driving decisions including the one or more assistance request items, the driving plan is transmitted as the remote assistance request.

1. An operator management system comprising:

6. An operator management method for managing, by a computer, the allocation of a plurality of remote support operators to a plurality of vehicles that transmit a remote support request including one or a plurality of support request items, comprising: setting or updating an allocation priority for each of the plurality of vehicles according to the one or more assistance request items of the remote assistance request; When the number of the plurality of vehicles is greater than the number of the plurality of remote support operators, and the allocation priority of a first vehicle which is one of the plurality of vehicles to which the plurality of remote support operators have not been assigned is higher than the allocation priority of a second vehicle which is one of the plurality of vehicles to which the plurality of remote support operators have been assigned, assigning the remote support operator assigned to the second vehicle to the first vehicle; Includes 23. An operator management method comprising:

7. An operator management program that causes a computer to manage the allocation of a plurality of remote support operators to a plurality of vehicles that transmit a remote support request including one or more support request items, comprising: setting or updating an allocation priority for each of the plurality of vehicles according to the one or more assistance request items of the remote assistance request; When the number of the plurality of vehicles is greater than the number of the plurality of remote support operators, and the allocation priority of a first vehicle which is one of the plurality of vehicles to which the plurality of remote support operators have not been assigned is higher than the allocation priority of a second vehicle which is one of the plurality of vehicles to which the plurality of remote support operators have been assigned, a process of allocating the remote support operator assigned to the second vehicle to the first vehicle; The computer is configured to execute 23. An operator management program comprising:

Images