Remote support management system, remote support management method, and remote support management program

Abstract

The present disclosure provides a remote support management system, a remote support management method, and a remote support management program. Provided is a technology capable of maintaining smooth traffic and reducing the number of operators required for remote support by a remote support of an autonomous vehicle. The present remote support management system communicates with a plurality of vehicles autonomously traveling, and causes an operator to perform remote support in response to a support request from the vehicles. The present remote support management system predicts occurrence of a future support request for each vehicle in accordance with an operating condition of each vehicle, and calculates a predicted support period for each support request predicted to occur. In a case where a repetitive support request predicted to have a predicted support period overlapping at the same time occurs more than a predetermined number, the present remote support management system instructs a change in driving mode for a target vehicle of which the number of vehicles in which the occurrence of the repetitive support request is predicted exceeds the predetermined number. By changing the driving mode of the target vehicle, the occurrence of the support request is avoided or delayed.

Description

Technical Field

[0001] This disclosure relates to a remote support management system, a remote support management method, and a remote support management program that communicates with multiple autonomously operating vehicles, receives support requests from said vehicles, and enables an operator to provide remote support. Background Technology

[0002] Autonomous vehicles generally operate autonomously and continuously. However, sometimes the autonomous judgment of autonomous vehicles is unreliable, and sometimes more reliable safety judgments are required. Therefore, research is being conducted that does not completely entrust the autonomous judgment of autonomous vehicles, but rather remotely monitors them, and, when necessary, the operator supports the autonomous vehicle's autonomous operation by conveying judgments and remote driving instructions to the vehicle. The prior art related to such a remote support management system is disclosed in Patent Document 1 below.

[0003] Patent Document 1 discloses a prior art method for assigning operators to autonomous vehicles requiring remote support. In this prior art, the processing order is determined based on the operation time and priority of remote support, and remote support tasks are assigned to operators according to this processing order. This avoids vehicles requiring remote support from obstructing traffic, thus achieving smoother traffic flow as a whole for the autonomous driving system.

[0004] Therefore, in a remote support management system, the role of operators who remotely monitor and operate autonomous vehicles is crucial. The more operators available for each autonomous vehicle, the better, provided there is a comprehensive system in place to quickly respond to support requests.

[0005] However, the more operators there are, the higher the labor costs become, making it commercially unsustainable. On the other hand, simply reducing the number of operators not only increases the workload for each operator, but also makes it impossible to cope with support requests from autonomous vehicles that require more personnel.

[0006] The aforementioned prior art assumes a sufficient number of operators to handle support requests. When a large number of support requests occur simultaneously, the prior art may fail to allocate operators to some of these requests. In such cases, there is a possibility that the autonomous vehicle, unable to receive operator judgment or driving instructions, may become stuck on the road, or that problems may arise due to the autonomous vehicle driving based on unreliable information.

[0007] In addition, in addition to the aforementioned Patent Document 1, Patent Document 2 may also be cited as a document indicating the level of technical expertise in the technical field related to this disclosure.

[0008] Existing technical documents

[0009] Patent documents

[0010] Patent Document 1: Japanese Patent Application Publication No. 2019-185279

[0011] Patent Document 2: Japanese Patent Application Publication No. 2020-042764 Summary of the Invention

[0012] This disclosure was made in view of the problems mentioned above, and its purpose is to provide technology that can maintain smooth traffic through remote support of autonomous vehicles and reduce the number of operators required for remote support.

[0013] This disclosure provides a remote support management system for achieving the above-mentioned objectives. The remote support management system of this disclosure is a system that communicates with multiple autonomously operating vehicles, accepts support requests from the vehicles, and enables an operator to provide remote support. This remote support management system includes: at least one memory including at least one program, and at least one processor coupled to the at least one memory. When executing the at least one program, the at least one processor predicts the occurrence of future support requests for each vehicle based on the operating status of each vehicle, and calculates a predicted support period for each predicted support request. If more than a predetermined number of repeated support requests are predicted to overlap at the same time during the predicted support periods, the at least one processor instructs a change of driving mode for more than a predetermined number of vehicles among the vehicles predicted to have repeated support requests. Specifically, the at least one processor instructs a change from a first driving mode, which is the normal driving mode, to a second driving mode for avoiding or delaying the occurrence of support requests.

[0014] According to this remote support management system, future support requests are predicted in advance for each vehicle. Then, if multiple predicted support requests overlap at the same time and the number of repetitions exceeds a predetermined number, a change in driving mode is instructed for more than a predetermined number of vehicles among those predicted to have repeated support requests. By changing the driving mode, the occurrence of support requests is avoided or delayed. Thus, even when support requests do occur, the number of support requests overlapping at the same time is suppressed to below a predetermined number. As a result, the overall workload of operators providing remote support is reduced, smooth traffic can be maintained through remote support from autonomous vehicles, and the number of operators required for remote support is reduced.

[0015] In this remote support management system, at least one processor can calculate an evaluation value for each vehicle for which a recurring support request is predicted. Based on the evaluation values, vehicles designated to switch from driving mode 1 to driving mode 2 are selected in descending order of their evaluation values. The evaluation value is an indicator used to determine which vehicle should be prioritized to avoid or delay support requests. The vehicle for switching driving modes can also be selected randomly. However, by selecting vehicles for switching driving modes based on certain indicators, the number of operators required for remote support can be reduced more reliably.

[0016] As an example of a method for calculating evaluation values, the following method is illustrated.

[0017] In Example 1, for each predicted support request, the probability of its occurrence is calculated. Vehicles with higher predicted probabilities of receiving support requests receive higher evaluation values. Based on Example 1, avoiding or delaying high-probability support requests reduces the overall operator workload.

[0018] In Example 2, for each predicted support request, an impact score representing the magnitude of the impact of the cause of the support request on the surrounding environment is calculated. Vehicles with higher predicted impact scores for support requests receive higher evaluation values. According to Example 2, the occurrence of phenomena with significant impact on the surrounding environment can be avoided or delayed, thus maintaining smooth traffic flow.

[0019] In Example 3, for each predicted support request, the operator skill required to process the support request is calculated. For vehicles where the predicted support request skill level is higher, a higher evaluation value is calculated. Operator utilization costs sometimes depend on the operator's skill level. According to Example 3, avoiding or delaying support requests that require high processing skills can reduce operator utilization costs.

[0020] In the fourth example, for each predicted support request, the processing time required to handle the support request is calculated. For vehicles with support requests predicted to have longer processing times, a higher evaluation value is assigned. Operator utilization costs sometimes depend on the processing time required for support requests. According to the fourth example, it is possible to avoid the occurrence of support requests requiring long processing times or to delay the occurrence of such support requests, thereby reducing operator utilization costs. Furthermore, according to the fourth example, it is possible to prevent operators from being dedicated solely to supporting a single vehicle.

[0021] In Example 5, for each predicted support request, a margin of safety until the support request occurs is calculated. For vehicles with a longer predicted margin of safety, a higher evaluation value is assigned. According to Example 5, by avoiding support requests with long margins or delaying their occurrence, a margin can be added to the response time of a vehicle until a driving mode change is indicated. As a result, the reliability of driving mode changes can be improved.

[0022] In this remote support management system, at least one of the processors can predict the occurrence of support requests at a predetermined update cycle, up to a future time corresponding to a predetermined prediction time longer than the update cycle. By making the prediction time for the occurrence of support requests longer than the update cycle of the prediction results, the accuracy of the prediction can be improved.

[0023] In this remote support management system, at least one processor can also configure operators for vehicles that do not indicate changes in driving mode in vehicles where repeated support requests are predicted, and update the operator configuration for each update cycle. By updating the operator configuration in conjunction with the update cycle that predicts the occurrence of support requests, operators can be configured in a way that allows for rapid response to actual support requests.

[0024] In this remote support management system, at least one memory and at least one processor may be configured on a server that communicates with multiple vehicles. In this case, the server may obtain the operating status of the target vehicle and the operating status of other vehicles besides the target vehicle, and predict the occurrence of future support requests from the target vehicle based on the operating status of the target vehicle and the operating status of other vehicles. By predicting the occurrence of support requests from the target vehicle not only based on the operating status of the target vehicle but also based on the operating status of other vehicles, the prediction accuracy of support request occurrence can be improved. Furthermore, the operating status of the target vehicle and other vehicles can be obtained either from each vehicle or from the operation management server (or a program within the server) that manages / instructs the operation of the vehicles.

[0025] In this remote support management system, the aforementioned at least one memory and at least one processor can be distributed among the on-board computers of multiple vehicles and the server communicating with the on-board computers. In this case, the on-board computer can use the sensors of the target vehicle equipped with the on-board computer to obtain the operating status of the target vehicle, and predict the occurrence of future support requests for the vehicle based on the operating status of the target vehicle. Furthermore, if the occurrence of a support request is predicted, information related to the prediction of the occurrence of the support request can be sent from the on-board computer to the server. By using the sensors of the target vehicle equipped with the on-board computer to obtain the operating status of the target vehicle, the occurrence of support requests in the target vehicle can be predicted with high responsiveness.

[0026] Furthermore, this disclosure provides a remote support management method for achieving the aforementioned objectives. The remote support management method disclosed herein is for multiple vehicles capable of autonomous driving and accepting remote support from an operator. This remote support management method includes: a step of predicting, for each vehicle, the occurrence of future support requests from the vehicle to the operator based on the operating status of each vehicle; and a step of calculating a predicted support period for each predicted support request. Furthermore, this remote support management method includes a step performed when the number of repeated support requests overlapping at the same time during the predicted support period exceeds a predetermined number. In this step, for more than a predetermined number of vehicles among the vehicles predicted to have repeated support requests, a change from a first driving mode, which is the normal driving mode, to a second driving mode for avoiding or delaying the occurrence of support requests is instructed.

[0027] Furthermore, this disclosure provides a remote support management program for achieving the aforementioned objectives. The remote support management program disclosed herein is a program that enables a computer to communicate with multiple autonomously operating vehicles, accept support requests from the vehicles, and enable an operator to provide remote support. This remote support management program enables the computer to: predict the occurrence of future support requests for each vehicle based on the operating status of each vehicle; and calculate the predicted support period for each predicted support request. Furthermore, this remote support management program enables the computer to instruct a change of driving mode when more than a predetermined number of repeated support requests overlapping at the same time during the predicted support period are predicted. Specifically, this remote support management program enables the computer to instruct a change from a first driving mode, which is the normal driving mode, to a second driving mode used to avoid or delay the occurrence of support requests.

[0028] According to the aforementioned remote support management method and procedure, future support requests are predicted in advance for each vehicle. Then, if multiple predicted support requests overlap at the same time and the number of repetitions exceeds a predetermined number, a change in driving mode is instructed for more than a predetermined number of vehicles among those predicted to have repeated support requests. By changing the driving mode, the occurrence of support requests is avoided or delayed. Thus, even when support requests actually occur, the number of support requests overlapping at the same time is suppressed to below a predetermined number. As a result, the overall workload of operators providing remote support is reduced, smooth traffic can be maintained through remote support from autonomous vehicles, and the number of operators required for remote support is reduced.

[0029] As described above, the remote support system, remote support management method, and remote support management procedure disclosed herein can maintain smooth traffic through remote support from autonomous vehicles and reduce the number of operators required for remote support. Attached Figure Description

[0030] Figure 1 This is a structural diagram of a remote monitoring system for autonomous vehicles.

[0031] Figure 2 This is a block diagram illustrating an example of the structure of an autonomous vehicle.

[0032] Figure 3 This is a block diagram illustrating an example of the structure of a monitoring center.

[0033] Figure 4 This is a diagram illustrating an example of operator load in a scenario where support requests have been issued from multiple vehicles.

[0034] Figure 5 This is a diagram illustrating an example of how potential support requests are ordered based on evaluation values.

[0035] Figure 6 This is a diagram illustrating an example of how operator workload can be appropriately managed by avoiding support requests.

[0036] Figure 7 This is a diagram illustrating an example of how operator load can be adjusted by delaying the occurrence of support requests.

[0037] Figure 8 This is a diagram illustrating the first specific example of a change in driving mode.

[0038] Figure 9 This is a diagram illustrating the second specific example of a change in driving mode.

[0039] Figure 10 This is a diagram illustrating the third specific example of a change in driving mode.

[0040] Figure 11 This is a system architecture diagram of the remote support management system according to the first embodiment of this disclosure.

[0041] Figure 12 This is a timing diagram illustrating the flow of information between vehicle A (the vehicle requiring support), vehicle B (the vehicle not requiring support), the remote support management planner, and the operator in the remote support management system according to the first embodiment of this disclosure.

[0042] Figure 13 This is a system architecture diagram of the remote support management system according to the second embodiment of this disclosure.

[0043] Figure 14 This is a timing diagram illustrating the flow of information between vehicle A (the vehicle requiring support), vehicle B (the vehicle not requiring support), the remote support management planner, and the operator in the remote support management system according to the second embodiment of this disclosure.

[0044] (Symbol Explanation)

[0045] 10: Communication network; 20: Autonomous vehicle; 21: Onboard computer; 21a: Processor; 21b: Memory; 21c: Program; 30: Monitoring center; 32: Server (Remote Support Management Planner); 32a: Processor; 32b: Memory; 32c: Program; 34, 37: Operating terminal; 35, 36, 38, 39: Operator; 100: Remote monitoring system. Detailed Implementation

[0046] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, when the number, quantity, amount, range, etc., of each element are mentioned in the embodiments shown below, the technical concept involved in the present disclosure is not limited to the mentioned numbers, unless specifically stated or explicitly determined in principle. In addition, the structures, etc., described in the embodiments shown below are not necessarily necessary in the technical concept involved in the present disclosure, unless specifically stated or explicitly determined in principle.

[0047] 1. Basic Structure of Remote Support Management System

[0048] Figure 1This is a structural diagram of a remote monitoring system for an autonomous vehicle. The remote monitoring system 100 is a system in which remote operators 35, 36, 38, and 39 remotely monitor the autonomous vehicle 20. Hereinafter, remote operators 35, 36, 38, and 39 will be simply referred to as operators 35, 36, 38, and 39. The autonomous driving level of the autonomous vehicle 20, which is the object of remote monitoring, is, for example, assumed to be level 4 or level 5. Hereinafter, the autonomous vehicle 20 will be simply referred to as vehicle 20.

[0049] Operators 35, 36, 38, and 39 include, for example, resident operators 35 and 36 who are stationed at monitoring center 30 to monitor vehicle 20, and home operators 38 and 39 who monitor vehicle 20 from home. A server 32 is installed in monitoring center 30. Operating terminals 34 operated by resident operators 35 and 36 are connected to server 32 via a LAN within monitoring center 30. Operating terminals 37 operated by home operators 38 and 39 are connected to server 32 via a communication network 10 including the Internet. The number of operating terminals 34 and 37 is prepared in proportion to the number of operators 35, 36, 38, and 39.

[0050] One function of the remote monitoring system 100 is the remote support management of the vehicles 20. Furthermore, the system performing remote support management is the remote support management system involved in the various embodiments of this disclosure. In the first embodiment, the server 32 within the monitoring center 30 functions as the remote support management system; in the second embodiment, the remote support management system is composed of the server 32 within the monitoring center 30 and the onboard computer of the vehicle 20. The server 32 is connected to multiple vehicles 20 via a communication network 10, including 4G and 5G networks.

[0051] The remote support management system communicates with multiple autonomous vehicles 20, accepts support requests from the vehicles 20, and enables operators 35, 36, 38, and 39 to provide remote support. In remote support, a portion of the decisions made by operators 35, 36, 38, and 39 regarding the autonomous driving of the vehicles 20 are performed. Basic calculations related to the cognition, judgment, and operation required for driving are performed within the vehicles 20. Operators 35, 36, 38, and 39 determine the actions that the vehicles 20 should take based on information sent from the vehicles 20 and issue commands to the vehicles 20. The remote support commands sent from operators 35, 36, 38, and 39 to the vehicles 20 include commands for the vehicles 20 to move and commands for the vehicles 20 to stop. Additionally, the remote support commands may include commands for avoiding obstacles ahead, instructions for overtaking by other vehicles, and instructions for emergency retreat.

[0052] Furthermore, the skills, i.e., proficiency, of operators 35, 36, 38, and 39 for remote support are uneven. Resident operators 35 and 36 are divided into highly skilled operators 35 and less skilled operators 36. Similarly, home operators 38 and 39 are also divided into highly skilled operators 38 and less skilled operators 39. Generally, the utilization cost (labor cost) of highly skilled operators 35 and 38 is relatively high, while the utilization cost of less skilled operators 36 and 39 is relatively low. The number of operators 35, 36, 38, and 39 is preferably one or more, preferably multiple. In particular, it is preferable to have at least one highly skilled resident operator 35.

[0053] Figure 2 This is a block diagram illustrating an example of the structure of vehicle 20. Vehicle 20 includes an on-board computer 21. The on-board computer 21 is an assembly of multiple ECUs (Electronic Control Units) mounted on vehicle 20. Additionally, vehicle 20 includes external sensors 22, internal sensors 23, actuators 24, and communication devices 25. These are connected to the on-board computer 21 via an in-vehicle network such as CAN (Controller Area Network).

[0054] The vehicle-mounted computer 21 includes one or more processors 21a (hereinafter referred to as processor 21a) and one or more memories 21b (hereinafter referred to as memory 21b) coupled to the processor 21a. The memory 21b stores one or more programs 21c (hereinafter referred to as programs 21c) that can be executed by the processor 21a and various information associated therewith.

[0055] The processor 21a executes program 21c, enabling various processes using the processor 21a. Program 21c includes, for example, programs for implementing autonomous driving and programs for implementing remote support. Furthermore, in the second embodiment, program 21c includes a program that enables the onboard computer 21 to function as part of a remote support management system. The memory 21b includes a main storage device and a secondary storage device. Program 21c can be stored either in the main storage device or in a computer-readable recording medium serving as secondary storage. Additionally, a map database for managing map information used in autonomous driving can be stored in the secondary storage device.

[0056] External sensors 22 include cameras that capture images of the area around the vehicle 20, particularly in front of the vehicle 20. Multiple cameras can be installed, and the cameras can capture images of the sides and rear of the vehicle 20 in addition to the front. Furthermore, the cameras can be shared for both autonomous driving and remote support by operators 35, 36, 38, and 39, or separate cameras can be installed for autonomous driving and remote support.

[0057] External sensors 22 include identification sensors other than cameras. Identification sensors are sensors that sense information for identifying the surrounding conditions of the vehicle 20. Examples of identification sensors other than cameras include LiDAR (Laser Imaging Detection and Ranging) and millimeter-wave radar. Additionally, external sensors 22 include position sensors that detect the position and orientation of the vehicle 20. Examples of position sensors include GPS (Global Positioning System) sensors. The information obtained from external sensors 22 is transmitted to the onboard computer 21. Furthermore, external sensors 22 include microphones that collect sound from the surrounding environment of the vehicle 20.

[0058] The internal sensors 23 include status sensors that acquire information related to the motion of the vehicle 20. Examples of status sensors include wheel speed sensors, acceleration sensors, angular velocity sensors, and rudder angle sensors. Acceleration sensors and angular velocity sensors may also be IMUs (Integrated Mutual Detectors). The information obtained by the internal sensors 23 is sent to the onboard computer 21. Hereinafter, the information obtained by the internal sensors 23 and the information obtained by the external sensors 22 will be collectively referred to as the operating status information of the vehicle 20. However, in addition to the operating status information obtained by the sensors of the vehicle 20, the operating status information also includes operating status information obtained by the operation management server that manages the operation of the vehicle 20.

[0059] Actuator 24 includes a steering mechanism for steering vehicle 20, a drive mechanism for driving vehicle 20, and a braking mechanism for braking vehicle 20. The steering mechanism may include, for example, a power steering system, a steer-by-wire system, and a rear-wheel steering system. The drive mechanism may include, for example, an engine, an EV system, and a hybrid system. The braking mechanism may include, for example, hydraulic braking and regenerative braking. Actuator 24 is operated by control signals sent from onboard computer 21.

[0060] Communication device 25 is a device for controlling wireless communication with the outside of vehicle 20. Communication device 25 communicates with server 32 via communication network 10. Information processed by onboard computer 21 is sent to server 32 using communication device 25. Information processed by server 32 is retrieved into onboard computer 21 using communication device 25. In addition, when vehicle-to-vehicle communication with other vehicles or road-to-road communication with infrastructure is required for autonomous driving, communication with these external devices is also carried out through communication device 25.

[0061] Figure 3 This is a block diagram illustrating an example of the structure of a monitoring center 30. The monitoring center 30 includes a server 32, a communication device 33, and an operating terminal 34. The communication device 33 controls communication with the outside world of the monitoring center 30. The communication device 33 mediates communication between the server 32 and multiple vehicles 20 via the communication network 10. Information processed by the server 32 is sent to the vehicles 20 using the communication device 33. Information processed by the vehicles 20 is retrieved by the communication device 33 and sent to the server 32. Furthermore, the communication device 33 mediates communication between the operating terminal 37, which is located outside the monitoring center 30, and the server 32.

[0062] Server 32 is a single computer or a collection of multiple computers connected by a communication network. Server 32 has one or more processors 32a (hereinafter referred to as processor 32a) and one or more memories 32b (hereinafter referred to as memories 32b) coupled to processor 32a. In memory 32b, one or more programs 32c (hereinafter referred to as programs 32c) that can be executed by processor 32a and various information associated therewith are stored.

[0063] The processor 32a executes program 32c, enabling various processing operations using the processor 32a. In the first embodiment, program 32c includes a program (remote support management program) that enables the server 32 to function as a remote support management system. In the second embodiment, program 32c includes a program that enables the server 32 to function as part of the remote support management system. The memory 32b includes a main memory device and a secondary memory device. Program 32c can be stored either in the main memory device or in a computer-readable recording medium that serves as the secondary memory device. Alternatively, a map database for managing map information used in autonomous driving can be stored in the secondary memory device. The map database can be stored in at least one of the server 32 and the vehicle-mounted computer 21.

[0064] Operating terminals 34 and 37 are equipped with information output units 34a and 37a. Information output units 34a and 37a are devices that output information required for remote support of vehicle 20 to operators 35, 36, 38, and 39. Information output by information output units 34a and 37a is sent from server 32 to each operating terminal 34 and 37. Information output units 34a and 37a include a display for outputting images and a speaker for outputting sound. The display may, for example, show an image of the front of vehicle 20 captured by a camera. The display may have multiple screens and may also display images of the sides and / or rear of vehicle 20. The speaker may, for example, convey the surrounding conditions of vehicle 20, collected by a microphone, to operators 35, 36, 38, and 39 via sound.

[0065] Operating terminals 34 and 37 are equipped with operating input units 34b and 37b. Operating input units 34b and 37b are devices for inputting operations for remote support of operators 35, 36, 38, and 39. Information input via operating input units 34b and 37b is sent from server 32 to vehicle 20. Specific examples of input devices include buttons, joysticks, and touch panels. For example, by tilting the joystick, one can instruct vehicle 20 to move / stop or to move laterally. Lateral movement includes, for example, avoiding obstacles ahead, changing lanes, and overtaking other vehicles.

[0066] 2. Overview of the Remote Support Management System

[0067] The purpose of this disclosed remote support management system is to maintain smooth traffic flow through remote vehicle support and to reduce the number of operators required for remote support. Therefore, firstly, using... Figure 4 This illustrates the load applied to the operator when multiple vehicles are used. Figure 4 This example illustrates the operator load when four vehicles, A, B, C, and D, issue support requests from these vehicles. Furthermore, the operator load referred to here is the number of operators required to process the support request.

[0068] exist Figure 4 In the example shown, support requests are issued sporadically from vehicles A, B, C, and D. The horizontal axis of the graph represents time, and the length of the rectangle corresponding to each support request represents the time required to process that request, i.e., the support time. The required support time varies depending on the content of the support request. Figure 4In the example shown, support requests overlap at the same time. For instance, support request A overlaps with support request B, and also with support request C. Thus, in cases where support requests are repeated at the same time, the number of operators required to execute these repeated support requests increases. Figure 4 In the example shown, the maximum number of operators required is 3. However, assuming there are only 2 operators, one of the support requests A, B, and C cannot be processed, resulting in a failure. The situation where no operator can be assigned to a support request is called an operator failure.

[0069] The remote support management system disclosed herein performs procedures to prevent failures as described above. In general, the remote support management system predicts the occurrence of future support requests for each vehicle based on its operational status, and calculates a predicted support period for each predicted support request. The predicted support period is the predicted duration required for processing the support request. A standard support time is statistically calculated based on the content of the support request, and this statistically calculated support time is used in the calculation of the predicted support period. Hereinafter, future predicted support requests are sometimes referred to as potential support requests. Operational status information used in the prediction of potential support requests includes, for example, map information, vehicle location, travel route, and vehicle speed.

[0070] Examples of situations that predict the occurrence of support requests include the following. The first example is a signalized intersection without vehicle-to-everything (V2I) communication equipment. In such intersections, support requests are predicted based on the signal illumination color and surrounding conditions. By pre-registering information about the presence of V2I equipment in a database, the predicted timing and duration of potential support requests can be calculated based on vehicle location, travel path, and speed information.

[0071] The second example involves areas with a high density of large trucks and large passenger vehicles. When large, tall vehicles are adjacent to each other, the object recognition accuracy using identification sensors decreases, thus affecting the prediction of support requests. By collecting parking data of large vehicles using sensors on each vehicle before and during the operation of this remote support management system, and performing trend analysis based on the collected parking data, locations / time periods with high frequency of encounters with dense large vehicles can be identified. By pre-registering the identified locations / time periods with dense large vehicle traffic in a database, the predicted occurrence time and predicted support time of potential support requests can be calculated based on vehicle location information, travel path information, and speed information.

[0072] The third example is the decrease in recognition accuracy caused by the long-term degradation of LiDAR. LiDAR uses the reflected intensity of the emitted laser light for sensing. Therefore, a decrease in the absolute value of the emitted light intensity means a decrease in recognition performance, which reduces the reliability of autonomous driving. Therefore, in the case of decreased LiDAR recognition accuracy, the occurrence of a support request is predicted. In the semiconductor lasers used in LiDAR, there is rapid degradation due to optical damage caused by heat, overcurrent, etc., and slow degradation due to crystal formation and manufacturing processes. Here, focusing on slow degradation, the monitoring results of the reflected intensity from a reference object are obtained as operational status information. Unlike the first and second examples, in the third example, the occurrence of a support request associated with long-term degradation is predicted by monitoring the monitoring results as operational status information over a long period.

[0073] The remote support management system disclosed herein updates the prediction results of potential support requests at a predetermined update cycle. Then, with each update, potential support requests are predicted up to a future time corresponding to a predetermined prediction time longer than the update cycle. As an example, the update cycle could be set to 1 second, and the prediction time for potential support requests could be set to 1 minute. By making the prediction time for potential support requests longer than the update cycle of the prediction results, the accuracy of the prediction can be improved.

[0074] The remote support management system disclosed herein predicts potential support requests for each vehicle and determines whether the predicted support periods overlap at the same time in the periods of multiple potential support requests. A predicted support period means the period during which an operator is bound by one support request. Therefore, if the predicted support periods overlap in the periods of multiple potential support requests, at least the number of operators required for the number of duplicate potential support requests is needed. Hereinafter, potential support requests whose predicted support periods overlap at the same time are referred to as duplicate support requests.

[0075] In the event of a predicted operator shortage due to a number of repeated support requests exceeding a predetermined number, the remote support management system of this disclosure avoids operator shortage by instructing a change in driving mode for a subset of vehicles. This change in driving mode refers to a change from a first driving mode, which is the normal driving mode, to a second driving mode designed to avoid or delay support requests, thereby preventing repeated support requests. The normal driving mode refers to the most efficient driving mode that allows for comfortable autonomous driving for the occupants. The vehicles for which the driving mode change is instructed are those where a predetermined number of repeated support requests are predicted. Typically, the predetermined number is the number of available waiting operators. Specifically, the vehicle for which the driving mode change is instructed is determined based on an evaluation value calculated for each vehicle for which repeated support requests are predicted.

[0076] The aforementioned evaluation values ​​are used to determine which vehicles should be prioritized for avoiding or delaying support requests. Among vehicles predicted to receive repeated support requests, driving mode changes are prioritized starting with vehicles with higher evaluation values. The evaluation values ​​are calculated using five variables: probability of occurrence, impact, required skills, required processing time, and surplus time. The following formula (1) is an example of an evaluation value calculation using these five variables. Furthermore, the dimensionless coefficient is a predetermined fixed value.

[0077] Evaluation value = Dimensionless coefficient × Probability of occurrence × Required skills × Processing time × Surplus time / Impact

[0078] …Formula (1)

[0079] The probability of occurrence, as the first variable, is the probability of a support request occurring, calculated for each potential support request. According to the calculation formula (1) above, for vehicles with a higher predicted probability of a support request, a higher evaluation value is calculated. By increasing the evaluation value based on the probability of a support request, the occurrence of support requests with a high probability of occurrence can be avoided or delayed, reducing the overall workload of the operator. Furthermore, the following methods can be exemplified as methods for predicting the probability of occurrence. In the first example, the probability of a support request occurring in each time period is predicted by analyzing the location, time period, and frequency of support requests based on past log data. In the second example, the probability of remote support is predicted based on traffic conditions (many trucks, etc.), road conditions (intersections, etc.), and weather conditions at the location being traversed.

[0080] The second variable, impact, is a numerical value representing the magnitude of the influence that a potential support request will have on the surrounding environment. In the following list, hypothetical situations are broadly categorized according to the methodology for considering impact. The numerical value attached to each category is the impact level. Impact is set on a scale of 1 to 6, with a value of 1 representing the situation with the highest impact. The impact decreases as the value increases. Situations related to anomalies / failures have high impact, while situations under normal conditions have low impact.

[0081] Impact level 1. Predicting the occurrence of traffic accidents

[0082] Impact level 2. Predicted difficulty in continuing operation due to malfunction / abnormality.

[0083] Impact level 3. Predicting retreat driving due to malfunction / abnormality

[0084] Impact level 4. A situation where a rear-end collision risk is predicted due to the vehicle's actions.

[0085] Impact level 5. Situations where the risk of disrupting traffic flow is predicted.

[0086] Impact level 6. Situations anticipated to impact operational delays and passenger comfort.

[0087] For each potential support request, the impact level is calculated. According to the calculation formula (1) above, for vehicles that predict the occurrence of support requests with higher impact levels, the evaluation value is calculated to be higher. The higher the impact of the cause of the potential support request on the surrounding area, the higher the evaluation value becomes, which can avoid or delay the occurrence of phenomena that have a large impact on the surrounding area and maintain smooth traffic. The above impact level can also be referred to as the priority used to determine the order of changing driving modes.

[0088] The third variable, required skill, is the operator's skill required to process the predicted potential support requests, calculated for each potential support request. The higher the required skill, the higher the operator's utilization cost. According to the calculation formula (1) above, for vehicles where a higher required skill is predicted, a higher evaluation value is calculated. By making the evaluation value higher with higher required skill, the occurrence of support requests requiring high processing skills can be avoided or delayed, thus reducing the operator's utilization cost.

[0089] The processing time, as the fourth variable, is the predicted processing time for each potential support request. The longer the processing time, the higher the operator utilization cost. According to the above calculation formula (1), the evaluation value is calculated as higher for vehicles where a longer processing time is predicted. By increasing the evaluation value for longer processing times, support requests requiring longer processing times can be avoided or delayed, reducing operator utilization costs. Simultaneously, it can also prevent operators from being dedicated solely to supporting one vehicle. Furthermore, a task level representing the difficulty of a support request can be calculated based on processing time and required skills. Support requests with higher task levels are preferentially assigned to operators with higher skills.

[0090] The margin of safety, as the fifth variable, is the time from the prediction of a potential support request to its actual occurrence, calculated for each potential support request. If there is insufficient margin of safety until the support request occurs, it is difficult to avoid or delay the occurrence of the support request even if the driving mode is changed. According to the calculation formula (1) above, the evaluation value is calculated to be higher for vehicles that predict the occurrence of support requests with longer margin of safety. As a result, it is possible to prioritize avoiding or delaying the occurrence of support requests with longer margin of safety, adding margin to the response time of the vehicle until the driving mode is changed. As a result, the reliability of avoiding or delaying the occurrence of support requests by changing the driving mode can be improved.

[0091] Figure 5This example illustrates the ranking of potential support requests A, B, C, D for each of vehicles A, B, C, and D, based on the evaluation value of each vehicle A, B, C, and D calculated using the method described above. Figure 5 In the example shown, the overlap between potential support requests A, B, and C is the highest. Among vehicles A, B, and C where potential support requests A, B, and C overlap, vehicle C has the highest evaluation value. Therefore, in order to avoid or delay the occurrence of support requests, the driving mode change should be prioritized from vehicle C, which has the highest evaluation value.

[0092] Figure 6 This diagram illustrates an example of how operator workload can be appropriately managed by avoiding support requests. Here, potential support request C is avoided by changing the driving mode of vehicle C. Then, of the remaining potential support requests A, B, and D, potential support requests B and D are assigned to operator A, and potential support request A is assigned to operator B. Operator A, assigned potential support requests B and D, accepts the support requests actually originating from vehicles B and D and performs remote support for vehicles B and D. Operator B, assigned potential support request A, accepts the support request actually originating from vehicle A and performs remote support for vehicle A. Furthermore, by avoiding potential support request C, the occurrence of support requests from vehicle C is prevented.

[0093] Figure 7 This diagram illustrates an example of how delaying the occurrence of support requests can appropriately manage operator workload. Here, by changing the driving mode of vehicle C, potential support request C is delayed, eliminating overlap with potential support requests A and B at the same time. Then, potential support requests B and D are assigned to operator A, while potential support request A and the delayed potential support request C are assigned to operator B. Operator A, assigned potential support requests B and D, accepts support requests actually originating from vehicles B and D and performs remote support for vehicles B and D. Operator B, assigned potential support requests A and C, accepts support requests actually originating from vehicles A and C and performs remote support for vehicles A and C. Because potential support request C is delayed, support requests actually originating from vehicle C are also delayed.

[0094] like Figure 6 as well as Figure 7 In the examples shown, by instructing a change in driving mode for vehicle C (among vehicles A, B, and C where a duplicate support request is predicted), the occurrence of duplicate support requests can be avoided, thus appropriately alleviating operator workload. Specifically, in cases where duplicate support requests cannot be avoided, such as... Figure 4 The example shown requires 3 operators, but by avoiding duplicate support requests, 2 operators are sufficient. Furthermore, Figure 6 as well as Figure 7 In the example shown, operator C is a waiting operator who waits to deal with unforeseen circumstances that differ from expectations. According to the remote support management system of this disclosure, by appropriately managing operator workload, a certain number of such waiting operators can be secured.

[0095] Next, the changes in driving mode will be explained. Changes in driving mode include alterations to the driving plan, changes to speed distribution (including acceleration / deceleration), changes to the driving trajectory, parking instructions, and the activation of lights. The remote support management system disclosed herein selects and executes one or more of these controls based on the content of the potential support request.

[0096] As a specific example, for each situation with the aforementioned level of influence, the controls are selected as follows. However, even if the control categories are the same, the degree of change in speed indication values, etc., varies as the level of influence increases. For example, the changes in speed distribution differ depending on whether a traffic accident is predicted or a situation is predicted to affect running delays or passenger comfort.

[0097] Situation 1: Predicting the occurrence of a traffic accident

[0098] Changes to the driving plan

[0099] Including changes in vehicle speed / acceleration / deceleration speed distribution

[0100] Changes in driving trajectory

[0101] Parking instructions

[0102] Situation 2. A situation where it is anticipated that the vehicle will be unable to continue operating due to a malfunction / abnormality.

[0103] Changes to the driving plan

[0104] Including changes in vehicle speed / acceleration / deceleration speed distribution

[0105] Lights

[0106] Parking instructions

[0107] Situation 3. Anticipation of a reluctance to drive due to a malfunction / abnormality.

[0108] Changes to the driving plan

[0109] Including changes in vehicle speed / acceleration / deceleration speed distribution

[0110] Changes in driving trajectory

[0111] Parking instructions

[0112] Situation 4. A situation where a rear-end collision is anticipated due to the vehicle's actions.

[0113] Changes to the driving plan

[0114] Including changes in vehicle speed / acceleration / deceleration speed distribution

[0115] Changes in driving trajectory

[0116] Situation 5. A situation where the risk of disrupting traffic flow is anticipated.

[0117] Including changes in vehicle speed / acceleration / deceleration speed distribution

[0118] Changes in driving trajectory

[0119] Situation 6. Situations that anticipate impacts on operational delays and passenger comfort.

[0120] Including changes in vehicle speed / acceleration / deceleration speed distribution

[0121] Changes in driving trajectory

[0122] Figures 8 to 10 This diagram illustrates specific examples of changes in driving modes. As is common to all diagrams, white arrows represent vehicle activity in driving mode 1, and black arrows represent vehicle activity in driving mode 2. Additionally, as is common to all diagrams, arrows with shading formed by diagonal lines represent vehicle activity under remote support.

[0123] Figure 8 This illustrates an example of a change in speed distribution, including vehicle speed / acceleration / deceleration. For instance, if a traffic accident is predicted ahead of the vehicle's direction of travel, the section for handling the accident may sometimes require remote assistance to allow the vehicle to pass. If the section cannot be bypassed, assistance requests from the vehicle cannot be avoided, but by changing the speed distribution, the timing of the assistance request from the vehicle can be delayed.

[0124] exist Figure 8 In the example shown, for each of the first and second driving modes, the target position of the vehicle at each time point is represented by a black circle. In this example, the vehicle arrives at the remote support area at time T(i+2) in the first driving mode, but at time T(i+4) in the second driving mode. That is, by reducing the vehicle speed in the second driving mode compared to the first driving mode, the arrival time to the remote support area is delayed. Therefore, in this example, by switching from the first driving mode to the second driving mode, the occurrence of the support request can be delayed.

[0125] Figure 9This illustrates an example of a change in the driving plan. (In the above...) Figure 8 In the example shown, the arrival time to the remote support zone is delayed by changing the speed distribution. However, if there is a bypass road that bypasses the remote support zone, the travel plan can also be changed by altering the way the vehicle travels on that bypass road. Figure 9 In the example shown, a route through the remote support zone is selected in driving mode 1, while a route bypassing the remote support zone is selected in driving mode 2. Therefore, in this example, by switching from driving mode 1 to driving mode 2, the support request can be avoided.

[0126] Figure 10 Other examples illustrating changes in driving plans. For instance, in countries where right-hand traffic is permitted (such as the US and China), vehicles face difficulties making left turns at intersections without traffic lights or dedicated left-turn arrows. Therefore, the probability of a vehicle requesting assistance is high at such intersections. Figure 10 In the example shown, the shortest route to the destination by turning left at the intersection is the route selected in Driving Mode 1. However, the probability of needing remote assistance is high on this route. In contrast, in Driving Mode 2, a route that involves repeatedly turning right without turning left is selected to reach the destination. Unlike turning left, the probability of needing remote assistance is low if turning right. Therefore, in this example, by switching from Driving Mode 1 to Driving Mode 2, it is possible to avoid requests for assistance.

[0127] 3. Structure of the remote support management system according to the first embodiment

[0128] Next, the structure of the remote support management system according to the first embodiment of this disclosure will be described. In the first embodiment, the server 32 functions as a remote support management system by executing a program (remote support management program) 32c stored in the memory 32b of the server 32 by the processor 32a. In the first embodiment, the server 32 that functions as a remote support management system is referred to as a remote support management planner 32.

[0129] Figure 11This is a system structure diagram of the remote support management system, namely the remote support management planner 32, according to the first embodiment. The remote support management planner 32 includes an operation status information acquisition unit 321, a support request occurrence prediction unit 322, an evaluation value calculation unit 323, a driving mode change instruction judgment unit 324, a driving mode change instruction unit 325, a support request priority judgment unit 326, an operator optimal configuration unit 327, an operator HMI function unit 328, and an operator utilization rate monitoring unit 329. When the processor 32a executes the program 32c stored in the memory 32b, they are implemented as the server 32 of the remote support management planner.

[0130] The remote support management planner 32 communicates with multiple vehicles. Here, the vehicles communicated with by the remote support management planner 32 are categorized as vehicles requiring support 20A, vehicles not requiring support 20B, and vehicles not requiring support 20C. Vehicles requiring support 20A are those currently requiring remote support. Vehicles not requiring support 20B are those that do not require remote support at the current time but have potential support requests and high evaluation values. Vehicles not requiring support 20C are those that do not require remote support at the current time but have potential support requests and low evaluation values.

[0131] The operation status information acquisition unit 321 acquires operation status information for all vehicles, including vehicles 20A, 20B, and 20C, in operation. The operation status information includes information acquired from each vehicle and information acquired from the operation management server that manages the operation of the vehicles. If server 32 also functions as an operation management server, the operation status information is delivered from the program that enables server 32 to function as an operation management server to the program that enables server 32 to function as a remote support management planner.

[0132] The support request prediction unit 322 predicts the occurrence of future support requests (potential support requests) for each vehicle based on the operating status information of each vehicle obtained by the operating status information acquisition unit 321. The support request prediction unit 322 predicts potential support requests for the target vehicle not only using information obtained from the vehicle being predicted, but also using information obtained from other vehicles besides the target vehicle, information held by the remote support management planner 32, and information obtained from the operation management server. Specific examples of predicting potential support requests are as described above.

[0133] Furthermore, the support request occurrence prediction unit 322 calculates the predicted support period for each predicted potential support request, that is, the period during which the operator is constrained for processing the potential support request. Then, it determines the temporal overlap between predicted potential support requests and counts the number of overlapping support requests that occur at the same time during the predicted support periods.

[0134] The evaluation value calculation unit 323 calculates an evaluation value for each vehicle for which a recurring support request is predicted, in order to determine which vehicle should prioritize avoiding or delaying the occurrence of the support request. As described above, the evaluation value calculation unit 323 calculates the evaluation value by taking five variables for each potential support request, namely, the probability of occurrence, the degree of impact, the necessary skills, the processing time, and the surplus time, and inputting them into the above calculation formula (1).

[0135] The driving mode change instruction determination unit 324 receives the operator's dedication status from the operator dedication rate monitoring unit 329 (described later). It then determines whether all potential support requests predicted by the support request occurrence prediction unit 322 can be allocated to available waiting operators. If, as a result of this initial determination, a potential support request cannot be allocated, or the predicted operator dedication rate after allocation exceeds a certain limit (e.g., 90%), the driving mode change instruction determination unit 324 determines that the operator has failed. Furthermore, the available waiting operators do not include a fixed number of waiting operators always guaranteed for unforeseen circumstances. Figure 6 as well as Figure 7 Operator C shown).

[0136] In the event of an operator failure, the driving mode change instruction determination unit 324 selects a vehicle for a driving mode change based on the evaluation value obtained from the evaluation value calculation unit 323. Specifically, the driving mode change instruction determination unit 324 selects vehicles for avoidance or delay based on the order of their evaluation values, considering potential support requests that exceed the number of available operators from among multiple potential support requests that generate repeated support requests that are the cause of the operator failure. The driving mode change instruction determination unit 324 performs a final allocation of potential support requests to operators, prioritizing avoiding or delaying the selected potential support requests. Then, the driving mode change instruction determination unit 324 selects the vehicles with potential support requests that have been selected as objects for avoidance or delay as the vehicles for a driving mode change from driving mode 1 to driving mode 2.

[0137] The driving mode change instruction unit 325 instructs the target vehicle selected by the driving mode change instruction determination unit 324 to change the driving mode from the first driving mode to the second driving mode. The target vehicle includes the vehicle 20B that does not require support. The driving mode change instruction unit 325 instructs the target vehicle to select the control as the second driving mode based on the content of the potential support request, especially the degree of impact.

[0138] The target vehicle in vehicle 20B does not require support to change its driving mode according to the instruction from the driving mode change instruction unit 325. By receiving the instruction to change the driving mode, the target vehicle takes actions to avoid or delay potential support requests, preventing repeated support requests exceeding the number of available waiting operators. Even assuming remote support is needed after changing the driving mode, operator failure will not occur immediately because it is ensured for waiting operators in unforeseen circumstances.

[0139] Next, the processing of support requests sent from the vehicle requiring support 20A to the remote support management planner 32 will be explained.

[0140] The support request priority determination unit 326 receives support requests from vehicles 20A requiring support. When support requests received from multiple vehicles 20A overlap in time, the support request priority determination unit 326 prioritizes the support requests according to the following categories. The numerical value attached to each category is the priority. Here, priorities are set to seven levels from 1 to 7, with a value of 1 indicating the highest priority situation. As the value increases, the priority decreases. Regarding priority, situations related to anomalies / malfunctions are set to high priority, while situations under normal conditions are set to low priority. However, even under normal conditions, the priority of situations related to accident risk is increased.

[0141] Priority 1. When a traffic accident occurs

[0142] Priority 2. When it is difficult to continue driving due to malfunction / abnormality.

[0143] Priority 3. Retreating while driving due to malfunction / abnormality

[0144] Priority 4. Situations where there is a risk of this vehicle colliding with other vehicles or pedestrians.

[0145] Priority 5. Situation where there is a risk of rear-end collision due to the actions of this vehicle.

[0146] Priority 6. Situations that pose a risk of disrupting traffic flow.

[0147] Priority 7. Situations affecting operational delays and passenger comfort.

[0148] The support request priority determination unit 326 determines the category based on the vehicle location of the support vehicle 20A, the road characteristics (intersections, merging roads, speed limits, etc.) through which the support vehicle 20A passes, the category of the support request flag from the support vehicle 20A, and the vehicle speed of the support vehicle 20A. According to the above classification, the higher the priority, the more rapid the response and the higher the skill level required for the support request.

[0149] The operator optimal configuration unit 327 receives the operator dedication status from the operator dedication rate monitoring unit 329. Then, according to the priority of each support request determined by the support request priority determination unit 326, available waiting operators are configured. For example, for support requests with relatively high priority, operators 35 and 38 with high skills are configured, and for support requests with relatively low priority, operators 36 and 39 with low skills are configured. If resident operators 35 and 36 and home operators 38 and 39 are available, support requests may be preferentially assigned to resident operators 35 and 36, for example.

[0150] The operator HMI function unit 328 connects the vehicle requiring support 20A and operators 35, 36, 38, and 39 via HMI, based on the combination of support request and waiting operators determined by the operator optimal configuration unit 327. More specifically, it connects the vehicle requiring support 20A and the operation terminals 34 and 37 operated by operators 35, 36, 38, and 39. As a result, images captured by the camera of the vehicle requiring support 20A are displayed on the monitors of the operation terminals 34 and 37, allowing operators 35, 36, 38, and 39 to confirm the status of the vehicle requiring support 20A. After confirming the status of the vehicle requiring support 20A, operators 35, 36, 38, and 39 operate the operation terminals 34 and 37 to perform remote support suitable for the support request from the vehicle requiring support 20A.

[0151] The operator dedicatedness monitoring unit 329 calculates the operator dedicatedness rate based on the connection results of operators 35, 36, 38, and 39 using the operator HMI function unit 328. The operator dedicatedness rate refers to, for example, a parameter indicating how many operators are dedicated to remote support operations within a predetermined time (e.g., 60 seconds) from the current time. The operator dedicatedness monitoring unit 329 calculates the operator dedicatedness rate at a predetermined update cycle and supplies the updated operator dedicatedness rate to the driving mode change instruction determination unit 324 and the operator optimal configuration unit 327.

[0152] Here, use Figure 12 This describes the flow of information implemented through the remote support management system according to the first embodiment constructed as described above. Figure 12 This is a timing diagram illustrating the flow of information between vehicle A (the vehicle requiring support), vehicle B (the vehicle not requiring support), the remote support management planner, and the operator using the remote support management system according to the first embodiment. The timing diagram also illustrates the remote support management method according to the first embodiment of this disclosure.

[0153] exist Figure 12In the example shown, operational status information is sent from each vehicle A and vehicle B to the remote support management planner. Additionally, although not shown in the diagram, operational status information for each of vehicles A and B is also sent from the operational management server to the remote support management planner.

[0154] The remote support management planner predicts future support requests for each of vehicles A and B based on the operational status information obtained. Furthermore, for each predicted support request, i.e., a potential support request, the remote support management planner calculates the predicted support period. Figure 12 In the example shown, it is assumed that vehicles A and B both predict potential support requests.

[0155] If it is predicted that the number of overlapping duplicate support requests occurring at the same time during the predicted support period exceeds the predetermined number that can be utilized, the remote support management planner calculates the above evaluation value for each of vehicles A and B for which duplicate support requests are predicted.

[0156] Next, the remote support management planner determines whether all predicted potential support requests can be assigned to the available waiting operators. In the event of an operator failure, the remote support management planner selects the vehicle that indicates a change from driving mode 1 to driving mode 2, in descending order of evaluation value. Figure 12 In the example shown, vehicle B is selected as the target vehicle.

[0157] Then, the remote support management planner instructs vehicle B, selected as the target vehicle, to change its driving mode from driving mode 1 to driving mode 2. At this point, the remote support management planner will instruct vehicle B to use the control selected based on the content of the potential support request, particularly its impact, as driving mode 2.

[0158] Following instructions from the remote support management planner, vehicle B changes its driving mode from driving mode 1 to driving mode 2. As a result, future support requests that should originate from vehicle B are avoided or delayed.

[0159] On the other hand, vehicle A subsequently becomes required for remote support as predicted, and sends a support request to the remote support management planner.

[0160] The remote support management planner receives a support request from vehicle A and determines the optimal operator configuration. Then, for the operator selected to be responsible for vehicle A, the status of vehicle A, specifically images captured by vehicle A's cameras, are displayed on the monitor.

[0161] The operator confirms the status of vehicle A based on the images displayed on the monitor and performs remote support operations for vehicle A.

[0162] As can be seen from the above description, according to the remote support management system of the first embodiment, when a support request actually occurs, the number of support requests overlapping at the same time during the support period is suppressed to below the number of available waiting operators. As a result, the overall workload of operators providing remote support is reduced, smooth traffic can be maintained through remote support from autonomous vehicles, and the number of operators required for remote support is reduced.

[0163] 4. Structure of the remote support management system according to the second embodiment

[0164] Next, the structure of the remote support management system according to the second embodiment of this disclosure will be described. In the second embodiment, the server 32 functions as part of the remote support management system by executing program 32c stored in memory 32b of the server 32 by processor 32a. Similarly, the vehicle-mounted computer 21 functions as part of the remote support management system by executing program 21c stored in memory 21b of the vehicle-mounted computer 21 by processor 21a. Furthermore, the remote support management system according to the second embodiment is constituted by connecting the server 32 and the vehicle-mounted computers 21 of each of the multiple vehicles via a communication network. Furthermore, the multiple vehicles referred to herein include all vehicles under the monitoring of the monitoring center 30, including vehicles 20A requiring support, vehicles 20B not requiring support, and vehicles 20C not requiring support. In the second embodiment, the server 32, which functions as part of the remote support management system, is referred to as the remote support management planner 32.

[0165] Figure 13 This is a system architecture diagram of the remote support management system according to the second embodiment. In the second embodiment, some of the functions of the remote support management planner 32 according to the first embodiment are transferred to the vehicle-mounted computer 21. The vehicle-mounted computer 21 includes an operation status information acquisition unit 211, a support request occurrence prediction unit 212, and an evaluation value calculation unit 213. These functions are implemented as those of the vehicle-mounted computer 21 when the processor 21a executes the program 21c stored in the memory 21b.

[0166] The operation status information acquisition unit 211 uses the sensors of the target vehicle equipped with the vehicle computer 21 to acquire the operation status information of the target vehicle.

[0167] The support request occurrence prediction unit 212 predicts the occurrence of future support requests (potential support requests) based on the operating status information of the target vehicle obtained by the operating status information acquisition unit 211. Then, it calculates the predicted support period for the predicted potential support requests. The support request occurrence prediction unit 322 of the first embodiment also uses operating status information of other vehicles for prediction, while the support request occurrence prediction unit 212 of the second embodiment only uses the operating status of the target vehicle obtained by the target vehicle's sensors for prediction. The first embodiment has an advantage in terms of the accuracy of predicting the occurrence of support requests, but according to the first embodiment, the occurrence of support requests in the target vehicle can be predicted with high responsiveness.

[0168] The evaluation value calculation unit 213 calculates an evaluation value for potential support requests predicted by the support request occurrence prediction unit 212. The evaluation value calculation unit 213 calculates the evaluation value by calculating five variables, namely the predicted probability of occurrence of potential support requests, impact, necessary skills, processing time, and surplus time, and inputting them into the above calculation formula (1).

[0169] Each vehicle 20A, 20B, and 20C will send the predicted support period and evaluation value of the potential support request calculated by the onboard computer 21 to the remote support management planner 32.

[0170] The remote support management planner 32 according to the second embodiment includes a driving mode change instruction determination unit 324, a driving mode change instruction unit 325, a support request priority determination unit 326, an operator optimal configuration unit 327, an operator HMI function unit 328, and an operator utilization rate monitoring unit 329. When the processor 32a executes the program 32c stored in the memory 32b, they are implemented as a server 32 of the remote support management planner.

[0171] The driving mode change instruction determination unit 324 receives the operator's dedication status from the operator dedication rate monitoring unit 329. It then determines whether all potential support requests obtained from each vehicle 20A, 20B, and 20C can be allocated to available waiting operators. If, as a result of this initial determination, a potential support request cannot be allocated, or the predicted dedication rate of the allocated operator exceeds the upper limit, the driving mode change instruction determination unit 324 determines that the operator has failed. In the case of operator failure, the driving mode change instruction determination unit 324 selects the vehicle to change the driving mode based on the evaluation values ​​obtained from each vehicle 20A, 20B, and 20C along with the potential support requests.

[0172] The driving mode change instruction unit 325 instructs the target vehicle selected by the driving mode change instruction determination unit 324 to change the driving mode from the first driving mode to the second driving mode. The target vehicle includes vehicles 20B that do not require support. The driving mode change instruction unit 325 instructs the target vehicle to select the second driving mode based on the content of the potential support request, particularly its impact. The target vehicles in vehicles 20B that do not require support change their driving mode according to the instruction from the driving mode change instruction unit 325.

[0173] The processing of support requests sent from the vehicle requiring support 20A to the remote support management planner 32 is the same as in the first embodiment. Therefore, descriptions of the functions of the support request priority determination unit 326, the operator optimal configuration unit 327, the operator HMI function unit 328, and the operator utilization rate monitoring unit 329 are omitted.

[0174] Next, use Figure 14 This describes the flow of information implemented through the remote support management system described in the second embodiment as above. Figure 14 This is a timing diagram illustrating the flow of information between vehicle A (the vehicle requiring support), vehicle B (the vehicle not requiring support), the remote support management planner, and the operator using the remote support management system according to the second embodiment. The timing diagram also illustrates the remote support management method according to the second embodiment of this disclosure.

[0175] exist Figure 14 In the example shown, vehicle A uses its sensors to obtain operational status information. Based on this information, it predicts the occurrence of support requests, calculates the predicted support period for each potential support request, and evaluates the predicted potential support request. Similarly, vehicle B uses its sensors to obtain operational status information, predicts the occurrence of support requests, calculates the predicted support period for each potential support request, and evaluates the predicted potential support request. Vehicles A and B independently send the predicted support period and evaluation value of their potential support requests to the remote support management planner.

[0176] The remote support management planner determines whether all potential support requests from vehicle A and vehicle B can be assigned to available waiting operators. In the event of an operator failure, the remote support management planner selects the vehicle that indicates a change from driving mode 1 to driving mode 2, in descending order of evaluation value. Figure 14 In the example shown, vehicle B is selected as the target vehicle.

[0177] Then, the remote support management planner instructs vehicle B, selected as the target vehicle, to change its driving mode from driving mode 1 to driving mode 2. At this point, the remote support management planner will instruct vehicle B to use the control selected based on the content of the potential support request, particularly its impact, as driving mode 2.

[0178] Following instructions from the remote support management planner, vehicle B changed its driving mode from driving mode 1 to driving mode 2. As a result, future support requests that should have originated from vehicle B were avoided or delayed.

[0179] On the other hand, vehicle A subsequently becomes required for remote support as predicted, and sends a support request to the remote support management planner.

[0180] The remote support management planner receives a support request from vehicle A and determines the optimal operator configuration. Then, for the operator selected to serve in vehicle A, the image captured by vehicle A's camera is displayed on the monitor.

[0181] The operator confirms the status of vehicle A based on the images displayed on the monitor and performs remote support operations for vehicle A.

[0182] As can be seen from the above description, similar to the first embodiment, the remote support management system according to the second embodiment also suppresses the number of support requests overlapping at the same time during the support period to below the number of available waiting operators when a support request actually occurs. As a result, the overall workload of operators providing remote support is reduced, smooth traffic can be maintained through remote support from autonomous vehicles, and the number of operators required for remote support is reduced.

Claims

1. A remote support management system that communicates with multiple autonomously operating vehicles, receives support requests from the vehicles, and enables an operator to provide remote support, characterized in that, The remote support management system includes: At least one memory, including at least one program; and At least one processor, coupled to the at least one memory, The at least one program is configured to cause the at least one processor to execute: Based on the operating status of each vehicle, predict the occurrence of future support requests for each vehicle; For each of the predicted support requests, calculate the predicted support period; For each predicted support request, calculate the probability of the support request occurring; and If an operator shortage is predicted when it is predicted that more than a predetermined number of duplicate support requests overlap at the same time during the predicted support period. For more than the predetermined number of vehicles among those predicted to receive repeated support requests, an instruction is given to switch from a first driving mode, which is the normal driving mode, to a second driving mode designed to avoid or delay the occurrence of the support requests. For each vehicle that is predicted to receive the repeated support request, an evaluation value is calculated to determine which vehicle should be prioritized to avoid or delay the occurrence of the support request. Based on the evaluation values ​​from highest to lowest, select the vehicles that indicate a change from the first driving mode to the second driving mode. The evaluation value is calculated using the following formula. Evaluation value = dimensionless coefficient × probability of occurrence × necessary skills × processing time × surplus time / impact. Wherein, the dimensionless coefficient is a predetermined fixed value, the occurrence probability is the probability of the support request occurring, the necessary skill is the operator skill required to process the support request, the processing time is the time required to process the support request, the margin time is the time from predicting the support request to the actual occurrence of the support request, and the influence degree represents the magnitude of the impact of the cause of the support request on the surrounding environment. The second driving mode is at least one of the following driving modes: Compared to the first driving mode, which reduces vehicle speed to delay arrival time in the remote support area and thus delays the occurrence of support requests; Driving mode that selects a route to bypass the remote support area to avoid support requests; and A driving mode that selects a route that repeatedly turns right instead of left to reach the destination in order to avoid support requests.

2. The remote support management system according to claim 1, characterized in that, Predicting the occurrence of such future support requests includes: Predictions are made up to a future time corresponding to a predetermined prediction time that is longer than the predetermined update cycle for predicting the occurrence of the support request.

3. The remote support management system according to claim 2, characterized in that, The at least one program is configured to cause the at least one processor to further execute: For vehicles in which the occurrence of the repeated support request is predicted, the operator is configured for those that do not indicate a change from the first driving mode to the second driving mode; as well as For each update cycle, the operator's configuration is updated.

4. The remote support management system according to claim 1, characterized in that, The at least one memory and the at least one processor are configured in a server that communicates with the plurality of vehicles. The server is configured as follows: Obtain the operating status of the target vehicle and the operating status of other vehicles besides the target vehicle. Based on the operating status of the target vehicle and the operating status of the other vehicles, the occurrence of future support requests for the target vehicle is predicted.

5. The remote support management system according to claim 1, characterized in that, The at least one memory and the at least one processor are distributed among the on-board computers of each of the plurality of vehicles and the servers that communicate with the on-board computers. The on-board computer is configured as follows: The vehicle's operating status is obtained using sensors on the vehicle equipped with the onboard computer. Based on the operating status of the target vehicle, predict the occurrence of future support requests for that vehicle. If the occurrence of the support request is predicted, information related to the prediction of the occurrence of the support request is sent to the server.

6. A remote support management method, which is a remote support management method for multiple vehicles capable of autonomous driving and receiving remote support from an operator, characterized in that, include: Based on the operating status of each vehicle, predict the occurrence of future support requests from the vehicle to the operator for each vehicle; For each of the predicted support requests, calculate the predicted support period; For each of the predicted support requests, calculate the probability of the support request occurring; as well as If an operator shortage is predicted when it is predicted that more than a predetermined number of duplicate support requests overlap at the same time during the predicted support period. For more than the predetermined number of vehicles among those predicted to receive repeated support requests, an instruction is given to switch from a first driving mode, which is the normal driving mode, to a second driving mode designed to avoid or delay the occurrence of the support requests. For each vehicle that is predicted to receive the repeated support request, an evaluation value is calculated to determine which vehicle should be prioritized to avoid or delay the occurrence of the support request. Based on the evaluation values ​​from highest to lowest, select the vehicles that indicate a change from the first driving mode to the second driving mode. The evaluation value is calculated using the following formula. Evaluation value = dimensionless coefficient × probability of occurrence × necessary skills × processing time × surplus time / impact. Wherein, the dimensionless coefficient is a predetermined fixed value, the occurrence probability is the probability of the support request occurring, the necessary skill is the operator skill required to process the support request, the processing time is the time required to process the support request, the margin time is the time from predicting the support request to the actual occurrence of the support request, and the influence degree represents the magnitude of the impact of the cause of the support request on the surrounding environment. The second driving mode is at least one of the following driving modes: Compared to the first driving mode, which reduces vehicle speed to delay arrival time in the remote support area and thus delays the occurrence of support requests; Driving mode that selects a route to bypass the remote support area to avoid support requests; and A driving mode that selects a route that repeatedly turns right instead of left to reach the destination in order to avoid support requests.

7. A computer-readable recording medium recording a remote support management program, said remote support management program being a program that enables a computer to communicate with multiple autonomously driving vehicles, receive support requests from said vehicles, and enable an operator to provide remote support, the computer-readable recording medium being characterized in that the remote support management program is configured to cause the computer to execute: Based on the operating status of each vehicle, predict the occurrence of future support requests for each vehicle; For each of the predicted support requests, calculate the predicted support period; For each of the predicted support requests, calculate the probability of the support request occurring; as well as If an operator shortage is predicted when it is predicted that more than a predetermined number of duplicate support requests overlap at the same time during the predicted support period. For more than the predetermined number of vehicles among those predicted to receive repeated support requests, an instruction is given to switch from a first driving mode, which is the normal driving mode, to a second driving mode designed to avoid or delay the occurrence of the support requests. For each vehicle that is predicted to receive the repeated support request, an evaluation value is calculated to determine which vehicle should be prioritized to avoid or delay the occurrence of the support request. Based on the evaluation values ​​from highest to lowest, select the vehicles that indicate a change from the first driving mode to the second driving mode. in, The evaluation value is calculated using the following formula. Evaluation value = dimensionless coefficient × probability of occurrence × necessary skills × processing time × surplus time / impact. Wherein, the dimensionless coefficient is a predetermined fixed value, the occurrence probability is the probability of the support request occurring, the necessary skill is the operator skill required to process the support request, the processing time is the time required to process the support request, the margin time is the time from predicting the support request to the actual occurrence of the support request, and the influence degree represents the magnitude of the impact of the cause of the support request on the surrounding environment. The second driving mode is at least one of the following driving modes: Compared to the first driving mode, which reduces vehicle speed to delay arrival time in the remote support area and thus delays the occurrence of support requests; Driving mode that selects a route that avoids remote support areas to avoid support requests; as well as A driving mode that selects a route that repeatedly turns right instead of left to reach the destination in order to avoid support requests.

Images