Information processing method and information processing system

Abstract

An information processing method executed on a computer, wherein a first operation performed by a monitor who is monitoring an autonomously driven moving body from a remote site where the moving body cannot be directly monitored is detected (S106), and when the first operation is detected, an execution condition for the autonomous driving of the moving body is relaxed relative to an execution condition for when the first operation is not detected, and the moving body is caused to autonomously drive under the relaxed execution condition (S107).

Description

[Technical Field]

[0001] The present disclosure relates to an information processing method and an information processing system. [Background technology]

[0002] In recent years, remote control systems have been studied in which a remote operator in a remote location indirectly controls an autonomously driven vehicle as needed using wireless communication such as a wireless local area network (LAN) or a mobile phone line. For example, when the vehicle reaches a situation where it cannot drive autonomously, the remote operator controls the vehicle's driving from a remote location by transmitting a control signal related to the vehicle's driving to the vehicle.

[0003] For example, Patent Document 1 discloses a remote control system that displays the predicted movement path of a moving object on an image that shows the moving direction of the moving object, depending on the communication delay time between the moving object and a remote control device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-61346 Summary of the Invention [Problem to be solved by the invention]

[0005] However, a remote control system such as that described in Patent Document 1 places a heavy burden on the remote operator. For example, the field of view on the remote control screen is narrower than that on a mobile object, so the remote operator must operate with more care than when operating on a mobile object.

[0006] Therefore, an object of the present disclosure is to provide an information processing method and an information processing device that can reduce the burden on a remote operator in a remote location. [Means for solving the problem]

[0007] An information processing method according to one embodiment of the present disclosure is an information processing method executed by a computer, which detects a first operation by a monitor who monitors an automatically driving mobile body from a remote location where the monitor cannot directly monitor the mobile body, and when the first operation is detected, relaxes the execution conditions for the automatic driving of the mobile body compared to the execution conditions when the first operation is not detected, and causes the mobile body to automatically drive under the relaxed execution conditions.

[0008] An information processing system according to one embodiment of the present disclosure includes a detection unit that detects a first operation by a monitor who monitors an autonomously driving mobile body from a remote location where the monitor cannot directly monitor the mobile body, and a mode change unit that, when the first operation is detected, relaxes the execution conditions for autonomous driving of the mobile body compared to the execution conditions when the first operation is not detected, and causes the mobile body to autonomously drive under the relaxed execution conditions. [Effects of the Invention]

[0009] According to the information processing method and the like according to one aspect of the present disclosure, it is possible to reduce the burden on a remote operator in a remote location. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a vehicle control system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing a functional configuration of the vehicle control system according to the embodiment. [Figure 3] FIG. 3 is a flowchart showing the operation of the vehicle according to the embodiment. [Figure 4] FIG. 4 is a flowchart showing the operation of the remote control device according to the embodiment. [Figure 5] FIG. 5 is a flowchart showing the operation of notifying the remote operator shown in step S102 of FIG. [Figure 6] FIG. 6 is a diagram illustrating an example of the second vehicle information. [Figure 7]FIG. 7 is a schematic diagram for explaining switching to the ODD-mitigated driving mode. [Figure 8] FIG. 8 is a flowchart showing the operation of the mode search process shown in step S103 of FIG. [Figure 9] FIG. 9 is a diagram illustrating an example of ODD data. [Figure 10] FIG. 10 is a diagram illustrating an example of constraint conditions. [Figure 11] FIG. 11 is a diagram illustrating an example of the first vehicle information. [Figure 12] FIG. 12 is a diagram illustrating an example of the first remote operation information. [Figure 13] FIG. 13 is a diagram illustrating an example of the first system information. [Figure 14] FIG. 14 is a flowchart showing an example of the operation of the switching process shown in step S107 of FIG. [Figure 15] FIG. 15 is a flowchart showing an example of the operation of the determination process shown in step S108 of FIG. [Figure 16] FIG. 16 is a flowchart showing the determination process shown in step S407 of FIG. [Figure 17] FIG. 17 is a diagram illustrating an example of the second remote control information. [Figure 18] FIG. 18 is a diagram illustrating an example of the second system information. [Figure 19] FIG. 19 is a flowchart showing the determination process shown in step S408 of FIG. [Figure 20] FIG. 20 is a diagram illustrating an example of a command. [Figure 21] FIG. 21 is a flowchart showing another example of the operation of the switching process shown in step S107 of FIG. [Figure 22] FIG. 22 is a flowchart showing another example of the operation of the determination process shown in step S108 of FIG. [Figure 23] FIG. 23 is a diagram for explaining the situation of a first application example for mitigating ODD. [Figure 24A]FIG. 24A is a diagram showing various conditions in the automatic driving mode of the first application example. [Figure 24B] FIG. 24B is a diagram showing various conditions in the ODD-mitigating running mode of the first application example. [Figure 25] FIG. 25 is a diagram for explaining the situation of a second application example for mitigating ODD. [Figure 26A] FIG. 26A is a diagram showing various conditions in the automatic driving mode of the second application example. [Figure 26B] FIG. 26B is a diagram showing various conditions in the ODD-mitigating running mode of the second application example. [Figure 27] FIG. 27 is a diagram for explaining the situation of a third application example for mitigating ODD. [Figure 28A] FIG. 28A is a diagram showing various conditions in the automatic driving mode of the third application example. [Figure 28B] FIG. 28B is a diagram showing various conditions in the ODD-mitigating running mode of the third application example. [Figure 29] FIG. 29 is a diagram for explaining the situation of a fourth application example for mitigating ODD. [Figure 30A] FIG. 30A is a diagram showing various conditions in the automatic driving mode of the fourth application example. [Figure 30B] FIG. 30B is a diagram showing various conditions in the ODD-mitigating running mode of the fourth application example. [Figure 31] FIG. 31 is a diagram for explaining the situation of a fifth application example for mitigating ODD. [Figure 32A] FIG. 32A is a diagram showing various conditions in the automatic driving mode of the fifth application example. [Figure 32B] FIG. 32B is a diagram showing various conditions in the first ODD-mitigating running mode of the fifth application example. [Figure 32C] FIG. 32C is a diagram showing various conditions in the second ODD-mitigating running mode of the fifth application example. [Figure 33] FIG. 33 is a block diagram showing a functional configuration of a vehicle control system according to a modified example of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] (Background to this disclosure) Autonomous driving systems are defined by an Operational Design Domain (ODD), which is set from the perspective of safety, and limits the conditions under which the vehicle can drive autonomously. The higher the autonomous driving level, the stricter the ODD conditions become, and at autonomous driving level 4, the driving area and driving conditions are significantly restricted. Therefore, the higher the autonomous driving level, the more likely it is that the ODD conditions will be violated, making autonomous driving impossible. In such cases, remote control of the vehicle by a remote operator in a remote location has been considered. However, if the remote operator has to operate the vehicle remotely every time the ODD conditions are violated, this places a heavy burden on the remote operator. For example, Patent Document 1 improves operability during remote operation by using a joystick or the like for remote control. However, even with an improved user interface, remote operation is difficult in situations where delays in operation and limited field of view occur, and this places a heavy burden on the remote operator.

[0012] Furthermore, during autonomous driving, a minimum level of driving safety must be guaranteed, such as ensuring that the vehicle does not collide with any objects.

[0013] Therefore, the inventors of the present application have conducted extensive research into information processing methods etc. that can reduce the burden on remote operators, and furthermore, information processing methods etc. that can reduce the burden on remote operators while ensuring a minimum level of safety, and have devised the information processing methods etc. described below.

[0014] Note that ODD is an example of an execution condition for autonomous driving.

[0015] An information processing method according to one embodiment of the present disclosure is an information processing method executed by a computer, which detects a first operation by a monitor who monitors an automatically driving mobile body from a remote location where the monitor cannot directly monitor the mobile body, and when the first operation is detected, relaxes the execution conditions for the automatic driving of the mobile body compared to the execution conditions when the first operation is not detected, and causes the mobile body to automatically drive under the relaxed execution conditions.

[0016] As a result, for example, when the moving body is in a state where it cannot travel autonomously, the monitor can perform the first operation, allowing the moving body to travel autonomously under relaxed execution conditions. In other words, even when the moving body is in a state where it cannot travel autonomously, depending on the situation, the moving body can be made to travel without the monitor remotely operating the moving body. As a result, the number of times the monitor remotely operates the moving body can be reduced. Therefore, the burden on the monitor in a remote location can be reduced.

[0017] Furthermore, for example, when the first operation is detected, the restriction on the traveling mode of the autonomous driving may be further strengthened in accordance with the relaxation of the execution condition.

[0018] This strengthens the restrictions on the traveling behavior of the mobile body during the period when automatic driving is being performed under relaxed execution conditions, making it easier to ensure the safety of the mobile body during that period.

[0019] Also, for example, the driving behavior may include at least one of the speed, steering angle, and acceleration of the moving body, and in response to the relaxation of the execution conditions, the restrictions on at least one of the maximum vehicle speed, maximum steering angle, and maximum acceleration during autonomous driving under the relaxed execution conditions may be strengthened.

[0020] This makes it possible to tighten restrictions on at least one of the maximum vehicle speed, maximum steering angle, and maximum acceleration of the moving body during the period when autonomous driving is being performed under relaxed execution conditions, making it easier to ensure the safety of the moving body during that period.

[0021] Furthermore, for example, when the first operation is detected, the monitoring requirement of the monitor with respect to the moving object may be further strengthened in accordance with the relaxation of the execution condition.

[0022] This strengthens the monitoring requirements of the supervisor during the period when autonomous driving is being performed under relaxed execution conditions, making it easier to ensure the safety of the moving body during that period.

[0023] Also, for example, the monitoring requirements may include at least one of a monitoring area, which is an area around the mobile body that requires monitoring by the monitor, and a monitoring object, which is an object that requires monitoring by the monitor, and strengthening the monitoring requirements may include strengthening monitoring of at least one of the monitoring area and the monitoring object in accordance with the relaxation of the execution conditions.

[0024] This makes it possible to strengthen restrictions on at least one of the monitor's monitoring area and the monitored object during the period when autonomous driving is being performed under relaxed execution conditions, making it easier to ensure the safety of the moving body during that period.

[0025] Also, for example, the monitoring requirements may include the operation of the moving body that requires monitoring by the monitor, and strengthening the monitoring requirements may include adding monitoring of the moving body's driving plan or driving control information in response to relaxation of the execution conditions.

[0026] This allows a supervisor to closely monitor the behavior of the moving object while it is being driven automatically under relaxed execution conditions, making it easier to ensure safety during that period.

[0027] Also, for example, the strengthened monitoring requirements may be presented to the monitor.

[0028] This allows the monitor to recognize the strengthened monitoring requirement. Therefore, safety can be ensured by having the monitor monitor according to the monitoring requirement. Furthermore, for example, if the strengthened monitoring requirement is presented before the first operation, the monitor can confirm the strengthened monitoring requirement and then decide whether or not to perform the first operation.

[0029] Also, for example, the mobile object may be automatically driven under the relaxed execution conditions only while the monitor is monitoring under the strengthened monitoring requirements.

[0030] This allows the supervisor to monitor the vehicle during the period when the vehicle is being driven automatically under the relaxed execution conditions, making it easier to ensure the safety of the vehicle during that period. Also, since the supervisor is monitoring the vehicle, the supervisor can smoothly make the decision to switch to remote control mode to control the vehicle's driving, depending on the driving status of the vehicle under the relaxed execution conditions.

[0031] Also, for example, the period during which monitoring is performed under the strengthened monitoring requirements may be a period during which a second operation by the monitor is detected, the second operation indicating that automatic driving will continue under the relaxed execution conditions.

[0032] This allows the mobile body to be autonomously driven under the relaxed execution conditions only during the period when the monitor is performing the second operation. In other words, the mobile body can be autonomously driven under the relaxed execution conditions only during the period when the monitor has determined that autonomous driving under the relaxed execution conditions should be continued. This prevents the autonomous driving under the relaxed execution conditions from being continued unnecessarily, thereby reducing, for example, the monitoring burden on the monitor due to unnecessary monitoring. This further reduces the burden on the monitor in a remote location. Furthermore, because autonomous driving under the relaxed execution conditions may be less safe than normal autonomous driving, safety can be ensured by only executing the autonomous driving under the relaxed execution conditions while the monitor is performing the second operation.

[0033] Furthermore, for example, when the first operation is detected, the requirement for delay in processing or communication related to the monitoring by the monitor may be strengthened in accordance with the relaxation of the execution condition.

[0034] This makes it possible to strengthen requirements for supervisory monitoring processes or communication delays during periods when autonomous driving is being performed under relaxed execution conditions, making it easier to ensure the safety of the moving body during those periods.

[0035] In addition, an information processing system according to one embodiment of the present disclosure includes a detection unit that detects a first operation by a monitor who monitors an automatically driving mobile body from a remote location where the monitor cannot directly monitor the mobile body, and a mode change unit that, when the first operation is detected, relaxes the execution conditions for the automatically driving of the mobile body compared to the execution conditions when the first operation is not detected, and causes the mobile body to automatically drive under the relaxed execution conditions.

[0036] This provides the same effects as the above-described information processing system.

[0037] Furthermore, these comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0038] Specific examples of an information processing method and an information processing system according to one aspect of the present disclosure will be described below with reference to the drawings. Each of the embodiments shown here represents a specific example of the present disclosure. Therefore, the numerical values, components, steps, and order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components that are not recited in independent claims will be described as optional components. Furthermore, the contents of each of the embodiments can be combined.

[0039] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in each figure, substantially the same components are given the same reference numerals, and redundant explanations are omitted or simplified.

[0040] Furthermore, in this specification, numerical values ​​and numerical ranges are not expressions that express only the strict meaning, but are expressions that mean that a substantially equivalent range, for example, a difference of about several percent, is also included.

[0041] (Embodiment) Hereinafter, an information processing method and the like according to this embodiment will be described with reference to FIGS. 1 to 32C.

[0042] [1. Vehicle control system configuration] First, the configuration of a vehicle control system 10 according to this embodiment will be described with reference to Figures 1 and 2. Figure 1 is a diagram showing a schematic configuration of a vehicle control system 10 according to this embodiment.

[0043] As shown in FIG. 1, the vehicle control system 10 includes a remote operation system 100, a network 300, a wireless base station 310, and a vehicle 200. The vehicle control system 10 is a system that communicatively connects the vehicle 200 and the remote operation system 100 (specifically, a remote operation device 130) via the wireless base station 310, such as a wireless LAN or a communication terminal, and the network 300. The wireless base station 310 and the network 300 are an example of a communication network. The vehicle 200 is an example of a moving object that is at least remotely monitored by a remote operator H. The vehicle 200 may also be a vehicle that is remotely monitored and remotely operated by the remote operator H. The vehicle control system 10 is an example of an information processing system. The remote operator H is an example of a monitor who monitors the autonomously driven vehicle 200 from a remote location where the remote operator H cannot directly monitor the vehicle 200. "Not being able to directly monitor" means that the vehicle 200 cannot be seen with the naked eye. That is, the remote operator H remotely monitors and remotely operates the vehicle 200 from a position different from the surroundings of the vehicle 200.

[0044] The remote control system 100 is a system that allows a remote operator H at a remote location to remotely monitor and remotely control the traveling of a vehicle 200 as needed. The remote control system 100 includes a display device 110, an operation input device 120, and a remote control device 130.

[0045] The display device 110 presents various types of information to the remote operator H. The display device 110 is a monitor connected to the remote operation device 130 and displays information related to the vehicle 200. The display device 110 displays information that the remote operator H uses to remotely monitor or remotely operate the traveling of the vehicle 200. The display device 110 displays, for example, an image captured by an imaging unit provided in the vehicle 200. The display device 110 also displays information processed by the remote operation device 130. The display device 110 may also display buttons (images) or the like that the remote operator H uses to switch the traveling mode of the vehicle 200 or to maintain the traveling mode. The display device 110 displays, for example, information such as a monitoring area that is requested to be monitored by the remote operator H. Details of the information displayed by the display device 110 will be described later.

[0046] The display device 110 is realized by, for example, a liquid crystal display.

[0047] The operation input device 120 accepts various operations from the remote operator H. The operation input device 120 accepts input of a driving permission command from the remote operator H when the vehicle 200 is driving in an ODD-mitigated driving mode in which ODD is mitigated more than in the automatic driving mode (automatic operation mode). The driving permission command is a command that permits the vehicle 200 to continue driving in the ODD-mitigated driving mode. The operation input device 120 is realized by, for example, a touch panel, but may also be realized by a hardware key (hardware button), a slide switch, or the like. The operation input device 120 may also accept various inputs based on information such as the voice, gesture, or line of sight of the remote operator H. The operation input device 120 may also accept input by swiping, input of a destination, etc.

[0048] Furthermore, when the vehicle is traveling in the remote control mode, the operation input device 120 accepts remote operation input from the remote operator H. The operation input device 120 is, for example, a steering wheel, foot pedals (for example, an accelerator pedal and a brake pedal), etc., but may also be realized by a joystick or the like.

[0049] The ODD mitigation driving mode and the remote control mode will be described later, but the vehicle 200 is characterized in that it can be driven in the ODD mitigation driving mode.

[0050] When the vehicle 200 is in a situation where it cannot be driven autonomously, the remote control device 130 executes processing to switch the driving mode of the vehicle 200. Even when the vehicle 200 is in a situation where it cannot be driven autonomously, the remote control device 130 reduces the frequency with which the remote operator H remotely controls the vehicle, thereby reducing the burden on the remote operator H. Specifically, the remote control device 130 switches the driving mode to an ODD-mitigated driving mode that is intermediate between autonomous driving level 4, at which the vehicle 200 is driven fully autonomously, and autonomous driving level 1, at which the remote operator H remotely controls the vehicle, and causes the vehicle 200 to continue autonomous driving in the switched ODD-mitigated driving mode. Furthermore, when the remote control device 130 cannot switch the driving mode to the ODD-mitigated driving mode, the remote control device 130 may cause the remote operator H to remotely control the vehicle 200.

[0051] As described above, the remote operation device 130 has an ODD-mitigated driving mode in addition to the conventional driving modes of the automatic driving mode and the remote operation mode. The ODD-mitigated driving mode is a mode in which the vehicle 200 is driven automatically by mitigating the ODD conditions. The ODD-mitigated driving mode can also be considered a mode in which the remote operator H does not remotely operate the vehicle 200. While driving in the ODD-mitigated driving mode, the remote operator H remotely monitors the vehicle 200, for example, based on images acquired from the vehicle 200. In other words, the vehicle 200 continues to drive automatically based on the ODD conditions mitigated under the remote monitoring of the remote operator H. This reduces the frequency with which the remote operator H remotely controls the vehicle at level 1, thereby reducing the burden on the remote operator H. As described above, the remote operation device 130 also functions as a remote monitoring device.

[0052] Furthermore, in this embodiment, the remote control device 130 further imposes constraint conditions on the vehicle 200 while it is traveling in the ODD-mitigated driving mode. It can also be said that the remote control device 130 imposes stricter constraint conditions on the vehicle 200 when it is in the ODD-mitigated driving mode than when it is in the autonomous driving mode. The constraint conditions are criteria for determining whether the vehicle 200 may continue traveling in the ODD-mitigated driving mode. By imparting constraint conditions, the remote control device 130 can make the vehicle 200 travel even more safely. In this way, the remote control device 130 can safely and automatically travel routes that cannot be traveled at autonomous driving level 4, which performs fully autonomous driving, by remote monitoring by the remote operator H and imparting constraint conditions.

[0053] The autonomous driving levels in this specification are defined as follows:

[0054] Autonomous driving level 1 is a level at which one of the operations of the accelerator (acceleration), steering (steering angle), and braking (control) is performed automatically. Autonomous driving level 2 is a level at which two or more of the operations of the accelerator, steering, and braking are performed automatically. Autonomous driving level 3 is a level at which all operations of the accelerator, steering, and braking are performed automatically, with the driver only taking action when necessary. Autonomous driving level 4 is a level at which all operations of the accelerator, steering, and braking are performed automatically, and the driver is not involved in driving. Autonomous driving level 4 is a level at which remote monitoring by a remote operator H is not required, for example. Autonomous driving level 3 is a level at which the driver is required to monitor, for example, and autonomous driving level 4 is a level at which the driver is not required to monitor, for example. Autonomous driving levels 3 and 4 are levels at which autonomous driving to a destination can be performed without the driver needing to perform driving operations. The autonomous driving levels are not limited to the four levels described above, and may be defined as five levels, for example.

[0055] Vehicle 200 is an example of a moving body in which people including a driver ride, and is remotely monitored or remotely operated by remote operator H as necessary. Vehicle 200 is an autonomous vehicle. Vehicle 200 may be an autonomous vehicle that can switch between autonomous driving and manual driving. Note that the autonomous vehicle is not particularly limited as long as it can be ridden by people and is capable of autonomous driving, and may be an autonomous bus, an autonomous taxi, an autonomous private car, an autonomous truck, an autonomous construction vehicle (e.g., a dump truck), etc.

[0056] The configurations of the remote control device 130 and the vehicle 200 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the vehicle control system 10 according to this embodiment. Note that Fig. 2 illustrates only the remote control device 130 and the vehicle 200 out of the various components included in the vehicle control system 10.

[0057] As shown in FIG. 2, the remote control device 130 includes a communication unit 140, a remote control unit 150, and a safety determination unit 160.

[0058] The communication unit 140 communicates with the vehicle 200 via the network 300 and the wireless base station 310. The communication unit 140, for example, acquires vehicle information from the vehicle 200 and outputs information for changing the driving mode to the vehicle 200. The communication unit 140 is realized by, for example, a communication circuit (communication module).

[0059] The remote control unit 150 executes processing related to switching of the driving mode of the vehicle 200. The remote control unit 150 includes a mode changing unit 151, a presenting unit 152, a command receiving unit 153, an operation receiving unit 154, and an operation information acquiring unit 155.

[0060] The mode change unit 151 switches the driving mode of the vehicle 200. The mode change unit 151 outputs information for switching the driving mode of the vehicle 200 to the vehicle 200 via the communication unit 140. The mode change unit 151 switches the driving mode of the vehicle 200 from the current driving mode to one of an autonomous driving mode (for example, a fully autonomous driving mode), an ODD-mitigating driving mode, and a remote control mode. If driving becomes impossible in the autonomous driving mode, the mode change unit 151 switches the driving mode to one of the ODD-mitigating driving mode and the remote control mode. Note that the mode change unit 151 switches the driving mode by, for example, receiving an operation from the remote operator H indicating permission to switch the driving mode.

[0061] The mode change unit 151 changes the ODD conditions of the vehicle 200 according to the driving mode after switching.

[0062] Furthermore, when the mode change unit 151 switches the driving mode of the vehicle 200 to the ODD-mitigated driving mode, the mode change unit 151 may further execute processing to ensure the safety of the vehicle 200 in accordance with the change in ODD. The mode change unit 151 may change settings related to the determination of whether the driving is possible or not by the driving possibility determination unit 231 of the vehicle 200, based on, for example, the constraint conditions of the mitigation possibility determination unit 162 or the remote operation possibility determination unit 163. The settings related to the determination of whether the driving is possible or not include, for example, at least one of a maximum vehicle speed, a maximum steering angle, a maximum acceleration, and the like. When the mode change unit 151 switches from the autonomous driving mode to the ODD-mitigated driving mode, the mode change unit 151 may update (rewrite) the settings related to the determination of whether the driving is possible or not, based on the constraint conditions of the ODD-mitigated driving mode. For example, the mode change unit 151 may reduce at least one of the maximum vehicle speed, a maximum steering angle, a maximum acceleration, and the like used by the driving possibility determination unit 231 to determine whether the driving is possible or not. It can also be said that when switching from the autonomous driving mode to the ODD-mitigated driving mode, mode change unit 151 imposes stricter restrictions than those imposed in the autonomous driving mode on at least one of the maximum vehicle speed, maximum steering angle, maximum acceleration, etc. For example, every time the driving mode of vehicle 200 is switched, mode change unit 151 changes the setting related to the determination of whether or not driving is possible by driving possibility determination unit 231.

[0063] In this way, the mode change unit 151 may change the driving mode of the vehicle 200 to an ODD-mitigated driving mode in which the constraint conditions are limited instead of mitigating the ODD. This allows the remote control device 130 to ensure a minimum level of safety on the vehicle 200 side in the ODD-mitigated driving mode and continue the automatic driving.

[0064] Furthermore, the mode changing unit 151 outputs, for example, the change in ODD and the constraint conditions to the area determining unit 161 and the driving continuation determination unit 164. Furthermore, when the mode changing unit 151 switches to the ODD-mitigated driving mode, for example, the mode changing unit 151 may enable reception of the operation receiving unit 154.

[0065] The speed, steering angle, and acceleration of the vehicle 200 are examples of the driving behavior of the automatic driving. The driving behavior may include at least one of the speed, steering angle, and acceleration of the vehicle 200.

[0066] The presentation unit 152 causes the display device 110 to make a predetermined presentation. For example, the presentation unit 152 generates information to be presented so that the remote operator H can remotely monitor or remotely operate the traveling of the vehicle 200. The presentation unit 152 may also cause the display device 110 to present predetermined information (for example, auxiliary information, which will be described later) superimposed on an image captured by an imaging unit included in the vehicle 200. The presentation unit 152 may also cause the display device 110 to present information processed by the remote operation device 130. The presentation unit 152 may also cause the display device 110 to present a button or the like that the remote operator H uses to switch the traveling mode of the vehicle 200 or to perform an operation to maintain the traveling mode.

[0067] The command receiving unit 153 receives information (e.g., a command) corresponding to an operation input by the remote operator H to the operation input device 120 (e.g., a touch panel, etc.) while driving in the ODD-mitigating driving mode, from the operation input device 120. The command receiving unit 153 is an example of a detection unit.

[0068] The operation receiving unit 154 receives, from the operation input device 120, information corresponding to an operation input by the remote operator H to the operation input device 120 (for example, a steering wheel, a foot pedal, or the like) in the remote operation mode.

[0069] The operation information acquisition unit 155 acquires information about the remote operation system 100, information about the remote operator H, etc. The information about the remote operator H includes the remote operator H's proficiency in remote operation, authority, etc. The operation information acquisition unit 155, for example, reads out information about the remote operator H from the storage unit 167. The information about the remote operator H may also include monitoring results of monitoring the remote operator H's current state. The current state is information that can determine whether the remote operator H is in a state where necessary monitoring can be performed, and includes at least one of, for example, whether the remote operator H is dozing, the level of concentration, and the level of fatigue. The operation information acquisition unit 155 may, for example, acquire an image of the remote operator H and acquire the monitoring results of the remote operator H's current state by image analysis of the image.

[0070] The safety determination unit 160 executes processing related to the driving safety of the vehicle 200 in the vehicle control system 10. The safety determination unit 160 executes processing for ensuring safety when driving in the ODD-mitigated driving mode, for example. For example, while the vehicle 200 is driving in the ODD-mitigated driving mode, the safety determination unit 160 monitors each of the vehicle 200, the remote operator H, and the remote operation system 100, and if a predetermined condition (for example, a constraint condition) is not satisfied, safety will be reduced and therefore the safety determination unit 160 executes processing to stop the vehicle 200.

[0071] The safety determination unit 160 includes an area determination unit 161, a mitigation feasibility determination unit 162, a remote control feasibility determination unit 163, a driving continuation feasibility determination unit 164, a fallback unit 165, a system information acquisition unit 166, and a memory unit 167.

[0072] The area determination unit 161 calculates the area that the remote operator H should monitor when driving in the ODD mitigation driving mode or the remote control mode, based on the monitoring area information from the mode change unit 151 and the vehicle information obtained from the vehicle 200.

[0073] When switching the driving mode, the mitigation feasibility determination unit 162 determines whether or not it is acceptable to switch to the ODD-mitigated driving mode. For example, the mitigation feasibility determination unit 162 calculates constraint conditions when driving in the ODD-mitigated driving mode, and makes the above determination based on whether or not the current situation satisfies the constraint conditions. The constraint conditions include at least one of a maximum vehicle speed, a maximum steering angle, a communication delay, and a required monitoring area. Note that the communication delay is a communication delay between the vehicle 200 and the remote control device 130. The required monitoring area is an area that needs to be monitored by the remote operator H in order for the vehicle 200 to drive autonomously, and is an area around the vehicle 200. The monitoring area monitored by the remote operator H is an example of a monitoring requirement of the monitor. Note that the monitoring requirement may include a remotely monitored object that requires monitoring by the remote operator H. The remotely monitored object is an example of a monitored object. Furthermore, the current situation includes at least one of the situation of the vehicle 200, the situation of the remote operator H, and the situation of the system.

[0074] When switching the driving mode, the remote control feasibility determination unit 163 determines whether or not it is OK to switch to the remote control mode. For example, the remote control feasibility determination unit 163 calculates constraint conditions when driving in the remote control mode, and makes the above determination based on whether or not the current situation satisfies the constraint conditions.

[0075] The continuation of driving possibility determination unit 164 determines whether or not the constraint conditions in the current driving mode are satisfied while the vehicle 200 is driving. When the current driving mode is the ODD alleviated driving mode, the continuation of driving possibility determination unit 164 determines whether or not the constraint conditions defined by the alleviation possibility determination unit 162 are satisfied while the vehicle 200 is driving. Furthermore, when the current driving mode is the remote control mode, the continuation of driving possibility determination unit 164 determines whether or not the constraint conditions defined by the remote control possibility determination unit 163 are satisfied while the vehicle 200 is driving.

[0076] The fallback unit 165 controls the traveling of the vehicle 200 based on the determination result of the traveling continuation determination unit 164. When the traveling continuation determination unit 164 determines that the constraint conditions in the traveling mode are not satisfied, the fallback unit 165 performs a fallback of the traveling of the vehicle 200. The fallback is, for example, stopping traveling, restricting traveling, or forcibly switching the traveling mode. When the traveling continuation determination unit 164 determines that the constraint conditions in the traveling mode are not satisfied, the fallback unit 165 performs one of stopping the traveling of the vehicle 200, further restricting the traveling conditions of the vehicle 200, or forcibly switching the traveling mode. For example, when the traveling continuation determination unit 164 determines that the constraint conditions in the traveling mode are not satisfied, the fallback unit 165 may bring the vehicle 200 to an emergency stop.

[0077] The system information acquisition unit 166 acquires various types of information related to the vehicle control system 10. The system information acquisition unit 166 acquires specification information of the remote operation system 100 (for example, first system information shown in FIG. 13 described later), and information indicating the current health status of the vehicle control system 10, such as the network status, such as communication delay and communication bandwidth, the computational resources of the remote operation device 130, and the operating status of the program (for example, second system information shown in FIG. 18 described later). The system information acquisition unit 166 may store the acquired information in the storage unit 167.

[0078] The storage unit 167 stores data (such as ODDs and constraint conditions, see, for example, FIGS. 9 and 10 described below) related to the vehicle 200 and remote operation for each of the autonomous driving mode, relaxed driving mode, and remote operation mode. The storage unit 167 stores, for example, a set of ODDs and constraint conditions corresponding to each of a plurality of relaxed driving modes. It can also be said that the storage unit 167 stores a set of conditions under which autonomous driving is possible and constraint conditions for those conditions. The storage unit 167 is realized, for example, by a semiconductor memory, but is not limited to this.

[0079] The vehicle 200 includes a communication unit 210, a vehicle information acquisition unit 220, an automatic driving system 230, a vehicle control unit 240, and a command acquisition unit 250.

[0080] The communication unit 210 communicates with the remote control device 130 via the network 300 and the wireless base station 310. The communication unit 210, for example, outputs vehicle information to the remote control device 130 and acquires information for changing the driving mode from the remote control device 130. The communication unit 210 is realized, for example, by a communication circuit (communication module).

[0081] The vehicle information acquisition unit 220 acquires vehicle information of the vehicle 200. The vehicle information includes at least one of the following: specification information of the vehicle 200, current vehicle speed, steering angle, acceleration, driving route information for autonomous driving, driving route information for the ODD mitigation driving mode, and sensing information. The specification information includes at least one of the size, wheelbase, maximum steering angle, maximum vehicle speed, maximum acceleration, maximum deceleration, and obstacle detection performance of the vehicle 200. The sensing information includes at least one of the following: the current position of the vehicle 200, information about surrounding objects, and information indicating the state of the vehicle interior.

[0082] The vehicle information acquisition unit 220 may acquire the vehicle information from an external device (for example, a server device that manages specification information of the vehicle 200), or may acquire the vehicle information based on sensing information from various sensors.

[0083] The automatic driving system 230 executes processing for the automatic driving of the vehicle 200 in accordance with the driving mode of the vehicle 200. The automatic driving system 230 includes a driving possibility determination unit 231 and an automatic driving driving unit 232.

[0084] The driving possibility determination unit 231 determines whether driving is possible in the automatic driving mode and whether driving is possible in the ODD mitigation driving mode. The driving possibility determination unit 231 determines whether driving is possible in the automatic driving mode and whether driving is possible in the ODD mitigation driving mode based on the set ODD and vehicle information. The driving possibility determination unit 231 repeatedly makes the determination at predetermined time intervals while driving in the automatic driving mode and the ODD mitigation driving mode.

[0085] The autonomous driving running unit 232 generates a driving plan for autonomous driving based on the vehicle information. The driving plan includes a driving route, a speed, and the like. Furthermore, when the determination result of the driving continuation feasibility determination unit 164 indicates that driving can be continued, the autonomous driving running unit 232 outputs control information based on the driving plan to the vehicle control unit 240 in order to execute the generated driving plan. The control information based on the driving plan is an example of driving control information.

[0086] The vehicle control unit 240 controls the driving of the vehicle 200 based on a driving plan for automatic driving or a control command from a remote control.

[0087] The command acquisition unit 250 acquires various commands from the remote operation device 130 via the communication unit 210, generates a command for controlling the vehicle 200 according to the acquired command, and outputs the generated command to the vehicle control unit 240. When the vehicle 200 is traveling in the ODD-mitigated traveling mode, the command acquisition unit 250 acquires information (a traveling continuation command) from the remote operation device 130 indicating that continuing traveling in the ODD-mitigated traveling mode is permitted. The command acquisition unit 250 acquires the information periodically, for example. Upon acquiring the information, the command acquisition unit 250 outputs a command indicating that the ODD-mitigated traveling mode is to be continued to the vehicle control unit 240. Note that the command acquisition unit 250 may stop traveling of the vehicle 200 if it does not acquire a traveling continuation command for a predetermined period of time, if there is a large delay in the traveling continuation command (for example, a large delay in the communication network), or if it acquires information indicating that the ODD-mitigated traveling mode is to be terminated.

[0088] In addition, when the vehicle 200 is traveling in remote control mode, the command acquisition unit 250 acquires a remote control command generated by the remote operator H operating the operation reception unit 154, generates a command for driving control of the vehicle 200 based on the acquired remote control command, and outputs the generated command to the vehicle control unit 240.

[0089] [2. Vehicle Control System Operation] Next, the operation of the vehicle control system 10 described above will be described with reference to Figures 3 to 22. Figure 3 is a flowchart showing the operation of the vehicle 200 according to this embodiment. Figure 3 shows the operation when the vehicle 200 is traveling in an autonomous driving mode.

[0090] 3, the driving possibility determination unit 231 determines whether or not autonomous driving can be continued (S11). The driving possibility determination unit 231 determines whether or not driving in the autonomous driving mode can be continued, for example, based on the ODD corresponding to the autonomous driving mode and vehicle information.

[0091] If it is possible to continue the autonomous driving (Yes in S11), the driving feasibility determination unit 231 continues the autonomous driving (S12). Specifically, the driving feasibility determination unit 231 outputs information indicating that the autonomous driving will be continued to the autonomous driving unit 232.

[0092] The traveling possibility determination unit 231 determines whether the vehicle 200 has arrived at the destination (S13). The traveling possibility determination unit 231 may determine whether the vehicle 200 has arrived at the destination, for example, based on the current position of the vehicle 200. The traveling possibility determination unit 231 may acquire the current position of the vehicle 200 from an external source via the communication unit 210, or may acquire the current position of the vehicle 200 based on a GPS (Global Positioning System) signal (i.e., a radio wave transmitted from a satellite). The traveling possibility determination unit 231 is configured to include a GPS module that acquires the current position by measuring the current position of the vehicle 200 based on the GPS signal. The destination is acquired in advance.

[0093] If the vehicle has arrived at the destination (Yes in S13), the traveling possibility determination unit 231 ends the traveling of the vehicle 200. If the vehicle has not arrived at the destination (No in S13), the traveling possibility determination unit 231 returns to step S11 and continues traveling.

[0094] If the automatic driving cannot be continued (No in S11), the driving possibility determination unit 231 stops the driving of the vehicle 200 (S14). Then, the driving possibility determination unit 231 outputs information indicating that the vehicle 200 cannot drive (automatic driving cannot be continued) to the remote operation device 130 via the communication unit 210 (S15).

[0095] Next, when the remote control device 130 acquires information indicating that driving is not possible from the vehicle 200, it executes a process of switching the driving mode. The process of switching the driving mode by the remote control device 130 will be described later. When the post-switching driving mode is determined, the remote control device 130 outputs information indicating that the driving mode will be switched to the vehicle 200. Hereinafter, an example will be described in which the post-switching driving mode is the ODD-mitigating driving mode.

[0096] Next, when the automatic driving traveling unit 232 acquires a command to switch the traveling mode to the ODD-mitigated traveling mode (Yes in S16), it switches the traveling mode to the ODD-mitigated traveling mode and starts traveling (S17). Specifically, the automatic driving traveling unit 232 generates a traveling plan for the ODD-mitigated traveling mode and outputs a command to the vehicle control unit 240 to execute the generated traveling plan.

[0097] Next, the command acquisition unit 250 determines whether or not an instruction to stop driving has been acquired from the remote operation device 130 (S18). In this embodiment, while driving in the ODD-mitigated driving mode, the command acquisition unit 250 periodically acquires, for example, a command that permits continuation of driving in the ODD-mitigated driving mode (hereinafter also referred to as a continue driving command). For example, if the command acquisition unit 250 does not acquire a continue driving command for a predetermined period of time, if there is a large delay in the continue driving command, or if it acquires information indicating that the ODD-mitigated driving mode will be terminated, the command acquisition unit 250 may determine Yes in step S18. Not acquiring a continue driving command for a predetermined period of time and information indicating that the ODD-mitigated driving mode will be terminated are examples of an instruction to stop driving.

[0098] If the command acquisition unit 250 acquires an instruction to stop driving from the remote operation device 130 (Yes in S18), the vehicle control unit 240 returns to step S14 and continues the subsequent processing. If the command acquisition unit 250 does not acquire an instruction to stop driving from the remote operation device 130 (No in S18), the vehicle control unit 240 continues driving in the ODD-mitigating driving mode (S19).

[0099] Next, the traveling possibility determination unit 231 determines whether or not the destination has been reached (S20). The determination process in step S20 is the same as that in step S13, and therefore a description thereof will be omitted. If the result in step S20 is No, the process returns to step S18 and continues.

[0100] Next, a description will be given of the operation of the remote control device 130. Fig. 4 is a flowchart showing the operation of the remote control device 130 according to this embodiment.

[0101] 4, when the presentation unit 152 acquires information indicating that driving is not possible (automated driving is not possible) from the vehicle 200 via the communication unit 140 (Yes in S101), it notifies the remote operator H (S102). The presentation unit 152 notifies the remote operator H by, for example, having the display device 110 present (display in this embodiment) the information acquired in step S101. Note that the presentation manner in which the presentation unit 152 presents the information to the remote operator H is not particularly limited.

[0102] Here, the process of notifying the remote operator H will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the operation of notifying the remote operator H shown in step S102 of Fig. 4.

[0103] As shown in Fig. 5, first, the presentation unit 152 acquires ODD deviation information (S201). The presentation unit 152 may acquire the ODD deviation information, for example, by acquiring second vehicle information including the ODD deviation information from the vehicle 200. Fig. 6 is a diagram showing an example of the second vehicle information. The remote control device 130 sequentially acquires the second vehicle information from the vehicle 200.

[0104] As shown in FIG. 6, the second vehicle information is information indicating the current state of the vehicle 200, and is output from the vehicle 200 to the remote control device 130 at predetermined time intervals. The second vehicle information includes the current vehicle speed, the current steering angle, the driving mode, the driving state, and ODD deviation information. The second vehicle information may also include the current acceleration. The driving mode is an autonomous driving mode, but the driving state is stopped. The ODD deviation information indicates that driving in the oncoming lane is necessary. In other words, the vehicle 200 is in a situation where an obstacle or the like is present ahead and driving in the oncoming lane is necessary, but driving in the oncoming lane would result in deviation from the ODD, and therefore autonomous driving cannot continue.

[0105] The presentation unit 152 may acquire the ODD deviation information separately from the second vehicle information.

[0106] Fig. 7 is a schematic diagram for explaining switching to the ODD mitigation driving mode. Fig. 7(a) shows a state in which vehicle 200 has stopped due to ODD deviation. Note that the arrow from vehicle 200 indicates the driving route. Fig. 7 also shows an overhead image, but the image presented by presentation unit 152 is not limited to an overhead image. If vehicle 200 is equipped with a surround view system, presentation unit 152 can cause display device 110 to present an overhead image.

[0107] FIG. 7(a) shows a state in which a vehicle 400 is stopped ahead of a vehicle 200 traveling on a road L, and an image showing the vehicle 200 stopped is presented on the display device 110. The current driving mode of the vehicle 200 and the result of a determination as to whether the vehicle 200 can continue autonomous driving may also be presented on the image. The driving mode is autonomous driving, and the result of the determination as to whether the vehicle 200 can continue autonomous driving is STOP (autonomous driving is not possible). The vehicle 400 is blocking the road L, and the ODD of the vehicle 200 includes the fact that, of driving in the vehicle's own lane and driving in the oncoming lane, only driving in the vehicle's own lane is possible. Therefore, the vehicle 200 cannot pass by avoiding the vehicle 400 by driving in the oncoming lane. The state shown in FIG. 7(a) may be presented to the remote operator H by the display device 110.

[0108] 5 again, the presentation unit 152 presents an ODD deviation alert (S202). That is, the presentation unit 152 presents information indicating that the vehicle 200 has deviated from the ODD lane on the display device 110. Next, the presentation unit 152 presents the deviation ODD information (S203). Steps S202 and S203 allow the remote operator H to know the ODD deviation alert and the deviation ODD information, which allows the remote operator H to smoothly remotely monitor or remotely operate the vehicle 200.

[0109] FIG. 7(b) shows an image showing a state in which an ODD deviation alert and deviation ODD information are presented on the presentation unit 152. The image includes information that the vehicle 400 is the cause of the vehicle 200 stopping, the deviation ODD information, the current driving mode, the result of a determination as to whether or not to continue automatic driving, and a switch button for switching the driving mode. In this way, by presenting the cause of the vehicle 200 becoming unable to drive automatically to the remote operator H, the remote operator H can be made to appropriately recognize the situation of the vehicle 200. Note that the ODD deviation alert presents the cause of the stop, the deviation ODD information, the result of a determination as to whether or not to continue automatic driving, etc.

[0110] The switching buttons include, for example, a button 112 indicating an "oncoming traffic lane driving mode" and a button 113 (a so-called software button) indicating "remote control." Here, the oncoming traffic lane driving mode is an example of an ODD-mitigated driving mode, which is a driving mode in which ODD is mitigated and restrictions such as restrictive conditions are strengthened. Also, remote control means a remote control mode. Note that if the switching buttons are hardware buttons, the switching buttons do not need to be presented on the display device 110.

[0111] The remote operator H can easily recognize that the vehicle 200 is stopped just by looking at the image shown in FIG. 7(b).

[0112] 4 again, next, the mode change unit 151 executes a mode search process (S103). Since driving in the autonomous driving mode is not possible, the mode change unit 151 can be said to determine whether or not there is a drive mode to which the vehicle can be switched. The mode change unit 151 determines whether or not there is an ODD-mitigating drive mode or a remote control mode to which the vehicle can be switched.

[0113] The mode search process will now be described with reference to Figures 8 to 13. Figure 8 is a flowchart showing the operation of the mode search process shown in step S103 of Figure 4.

[0114] The mode change unit 151 acquires ODD data and constraint conditions (S301). The mode change unit 151 acquires the ODD data and constraint conditions for the ODD-reduced driving mode from the storage unit 167. In the example of FIG. 7, the mode change unit 151 acquires the ODD data and constraint conditions for the oncoming lane driving mode. The oncoming lane driving mode is, for example, a driving mode in which an oncoming lane is included in the ODD drivable area of ​​the vehicle 200. For example, when multiple ODD-reduced driving modes are stored in the storage unit 167, the mode change unit 151 may extract one or more ODD-reduced driving modes (one example of a switchable ODD-reduced driving mode) in which an oncoming lane is included in the ODD drivable area.

[0115] Here, an example of ODD data and constraint conditions will be described with reference to Fig. 9 and Fig. 10. Fig. 9 is a diagram showing an example of ODD data. Fig. 9 shows ODD data in the oncoming lane driving mode (ODD-reduced driving mode). Fig. 10 is a diagram showing an example of constraint conditions. The constraint conditions shown in Fig. 10 are constraint conditions corresponding to the ODD data shown in Fig. 9.

[0116] FIG. 9(a) shows the drivable area, and in the oncoming lane driving mode, the drivable area is the current vehicle lane and the oncoming lane. FIG. 9(b) shows objects to be overtaken, and in the oncoming lane driving mode, the objects to be overtaken are a vehicle (pulling over to the shoulder), a color cone (registered trademark), and a vehicle (vehicle ahead). Note that in the ODD data in the automatic driving mode, for example, the driving area is only the current vehicle lane, and the objects to be overtaken are only a vehicle (pulling over to the shoulder) and a color cone (registered trademark). In other words, in the ODD data shown in FIG. 9, the oncoming lane is added to the drivable area of ​​the ODD data in the automatic driving mode, and a vehicle (vehicle ahead) is added to the objects to be overtaken. This is an example of ODD being relaxed.

[0117] FIG. 10(a) shows the vehicle conditions, in which in the oncoming lane driving mode, the vehicle conditions are a maximum vehicle speed of 10 km / h, a maximum steering angle of 45 degrees, a maximum acceleration of 0.3 G, and a system response time of 0.8 seconds.

[0118] (b) of FIG. 10 shows the conditions for safety monitoring, and in the oncoming lane driving mode, the areas to be monitored are the vehicle's own lane and the oncoming lane, and remote operator H can take over monitoring (however, this is limited to those with a low level of fatigue). This indicates that the vehicle 200 itself may monitor the areas to be monitored (for example, by detecting objects), or the remote operator H may remotely monitor at least a part of the areas to be monitored. Note that the level of fatigue of the remote operator H can be acquired, for example, from an image captured of the remote operator H. Furthermore, the areas to be monitored by the remote operator H may be areas where the vehicle 200 itself cannot detect objects or where the reliability of object detection is low.

[0119] (c) in FIG. 10 shows the system conditions, where in the oncoming lane driving mode, the communication delay is 0.2 seconds (maximum 0.2 seconds) and the system state is normal.

[0120] Referring again to FIG. 8 , next, mode change unit 151 acquires first vehicle information (S302), acquires first remote operation information related to remote operation (S303), and acquires first system information (S304). There are no particular limitations on where the first vehicle information, first remote operation information, and first system information are acquired from. Taking the first vehicle information as an example, mode change unit 151 may acquire the first vehicle information from vehicle 200, or if the first vehicle information is stored in storage unit 167, read the first vehicle information from storage unit 167, or acquire the first system information from a server device that manages the vehicle information of vehicle 200. Furthermore, the first system information may be acquired via system information acquisition unit 166. It is sufficient that the first vehicle information, the first remote operation information, and the first system information are acquired at least once. If the first vehicle information, the first remote operation information, and the first system information have already been acquired, the processes of steps S302 to S304 may be omitted.

[0121] The first vehicle information, the first remote control information, and the first system information will be described below with reference to Fig. 11 to Fig. 13. Fig. 11 is a diagram showing an example of the first vehicle information. Fig. 12 is a diagram showing an example of the first remote control information. Fig. 13 is a diagram showing an example of the first system information.

[0122] 11, the first vehicle information is information indicating the specifications of vehicle 200, and includes maximum vehicle speed, maximum steering angle, maximum acceleration, system response time, and monitorable area. Since the monitorable area is the vehicle's own lane, vehicle 200 itself can only monitor the vehicle's own lane.

[0123] As shown in (a) and (b) of FIG. 12, the first remote operation information includes information about the remote operation device 130 and information (specifications) about the remote operator H. The information about the remote operation device 130 includes information about the capabilities of the remote operation device 130, for example, information about a monitorable area that can be monitored by the remote operation device 130. The example of (a) of FIG. 12 indicates that the remote operation device 130 has the capability to monitor oncoming lanes. The information about the remote operator H includes the authority of the remote operator H to perform remote operation. The example of (b) of FIG. 12 indicates that the remote operator H has been granted the authority to perform remote operation. The contents described in (a) and (b) of FIG. 12 are set in advance.

[0124] As shown in Fig. 13, the first system information is information indicating the specifications of the vehicle control system 10, and includes the minimum communication delay. In the example of Fig. 13, the minimum communication delay is 0.1 seconds.

[0125] 8 again, the mode change unit 151 searches for a possible driving mode based on the various information acquired in steps S301 to S304 (S305). When a constraint condition is satisfied by each of the first vehicle information, the first remote-operation information, and the first system information, the mode change unit 151 searches for the constraint condition and the ODD corresponding to the constraint condition as a possible driving mode. The mode change unit 151 may search for, for example, a plurality of possible driving modes.

[0126] In step S305, the mode change unit 151 may determine whether or not it is possible to switch to the remote operation mode. The mode change unit 151 may determine whether or not it is possible to switch to the remote operation mode based on whether or not a constraint condition corresponding to the remote operation mode is satisfied. The constraint condition corresponding to the remote operation mode includes, for example, a communication delay.

[0127] 4 again, next, if there is an available driving mode (Yes in S104), the presentation unit 152 presents the driving mode to the remote operator H (S105). The presentation unit 152 may present a plurality of driving modes to the remote operator H. For example, as shown in (b) of FIG. 7, the presentation unit 152 may present buttons 112 and 113 by superimposing them on the image.

[0128] When the mode change unit 151 has searched for a plurality of driving modes, the presentation unit 152 may present a driving mode with less restrictive conditions with priority.

[0129] Furthermore, when presenting the ODD-mitigated driving mode to the remote operator H, the presentation unit 152 may further superimpose and present an area that the remote operator H must remotely monitor in the ODD-mitigated driving mode (for example, the area requiring monitoring shown in (c) of FIG. 7, which is an example of an enhanced monitoring requirement). In this way, the area that the remote operator H must remotely monitor in the ODD-mitigated driving mode may be presented on the display device 110 before the remote operator H operates the button 112.

[0130] This allows the remote operator H to check the area that he or she needs to monitor and then determine whether or not to switch to the ODD-mitigating driving mode. The notification unit 152 acquires the area that needs to be monitored from the area determination unit 161.

[0131] Next, the command receiving unit 153 receives a selection of a driving mode from the remote operator H (S106). The command receiving unit 153 detects, for example, an operation (for example, a touch operation) on the button 112 or 113. It can also be said that the command receiving unit 153 receives an operation from the remote operator H permitting switching of the driving mode. In this way, for example, when the remote operation device 130 acquires an operation from the remote operator H permitting switching of the driving mode, the remote operation device 130 executes a process for switching the driving mode. For example, when the remote operation device 130 is capable of driving in automatic driving with ODD mitigation and the remote operator H permits switching to the ODD-mitigated driving mode, the remote operation device 130 switches to the ODD-mitigated driving mode.

[0132] When the command receiving unit 153 receives a selection of a driving mode from the remote operator, the mode change unit 151 executes a driving mode switching process to switch to the received driving mode (S107). When an operation on the button 112 is detected, the mode change unit 151 executes a process to switch the vehicle 200 to the ODD-mitigating driving mode, and when an operation on the button 113 is detected, the mode change unit 151 executes a process to switch the vehicle 200 to the remote control mode. Note that the operation on the button 112 is an example of a first operation.

[0133] Here, the operation when the command receiving unit 153 receives a selection of the oncoming lane driving mode will be described with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the operation of the switching process shown in step S107 of Fig. 4.

[0134] As shown in FIG. 14, the mode change unit 151 changes the ODD of the vehicle 200 (S401). The mode change unit 151 changes the ODD condition of the vehicle 200 to the ODD in the post-switching driving mode (see, for example, FIG. 9). The mode change unit 151 outputs, to the vehicle 200 via the communication unit 140, information indicating that the ODD will be changed to the ODD in the post-switching driving mode. When the mode change unit 151 detects an operation on the button 112 by the remote operator H, it changes the ODD of the vehicle 200 from the ODD of the automatic driving mode to the ODD of the ODD-reduced driving mode. When the mode change unit 151 detects an operation on the button 112 by the remote operator H, it can also be said that the mode change unit 151 reduces the ODD of the vehicle 200 to be lower than the ODD in the automatic driving mode when the operation is not detected.

[0135] Next, the mode change unit 151 changes the setting of the vehicle 200 (S402). The mode change unit 151 changes the criterion used by the travel possibility determination unit 231 of the vehicle 200 to determine whether or not the vehicle is travelable. The mode change unit 151 changes the criterion to the vehicle conditions in the travel mode after the switch (for example, see (a) of FIG. 10). The mode change unit 151 outputs, via the communication unit 140, information indicating that the criterion will be changed to the vehicle conditions in the travel mode after the switch, to the vehicle 200. Furthermore, the mode change unit 151 outputs the constraint conditions for the ODD after the switch (for example, see FIG. 10) to the area determination unit 161.

[0136] The autonomous driving unit 232 generates a driving plan for autonomous driving in the ODD-mitigated driving mode based on the mitigated ODD and the changed settings of the vehicle 200. It can also be said that the autonomous driving unit 232 updates the driving plan in accordance with the mitigation of the ODD. Furthermore, the autonomous driving unit 232 outputs control information based on the driving plan to the vehicle control unit 240 in order to execute the driving plan.

[0137] The remote operator H may monitor the driving plan or control information of the vehicle 200 in the ODD-mitigated driving mode. Information indicating the driving plan or the control information is an example of auxiliary information, and may be presented on the display device 110.

[0138] In this manner, the monitoring requirements include operations of the vehicle 200 that require monitoring by the remote operator H, and strengthening the monitoring requirements may include adding monitoring of the vehicle 200's driving plan or driving control information in response to mitigation of the ODD.

[0139] Next, the area determination unit 161 sets a monitoring area for the remote operator H (S403). The area determination unit 161 sets the monitoring area for the remote operator H so as to satisfy the constraint conditions for the ODD after switching. The area determination unit 161 sets the monitoring area for the remote operator H based on the constraint conditions, the first vehicle information, and the first remote operation information. For example, the area determination unit 161 sets an area that cannot be monitored by the vehicle 200 among the areas set as monitoring areas for the safety monitoring conditions as the monitoring area for the remote operator H.

[0140] As shown in (b) of Fig. 10, the areas to be monitored are the current lane and the oncoming lane. Also, as shown in Fig. 11, the monitorable area of ​​vehicle 200 is the current lane. In this case, since vehicle 200 cannot monitor the oncoming lane, the oncoming lane is set as the monitoring area of ​​remote operator H.

[0141] The mitigation possibility determination unit 162 may determine whether the remote operation device 130 has the capability to monitor the monitoring area set by the area determination unit 161 based on the first remote operation information, and if the remote operation device 130 does not have the capability, may output information indicating that the monitoring-required area indicated by the constraint condition cannot be monitored to the mode change unit 151. In this case, the mode change unit 151 may search again for another driving mode (for example, another ODD-mitigating driving mode).

[0142] Referring again to FIG. 14 , next, the driving continuation determination unit 164 sets a driving continuation condition for the ODD-mitigated driving mode after the switch (S404). The driving continuation condition here is a criterion for determining whether or not to continue driving in the ODD-mitigated driving mode after the switch. The driving continuation determination unit 164 may set, for example, a constraint condition for the ODD-mitigated driving mode after the switch as the driving continuation condition. The driving continuation determination unit 164 may determine that driving cannot be continued and may stop the vehicle 200 when the constraint condition corresponding to the switched driving mode is no longer satisfied while driving in the switched driving mode, for example, when the vehicle speed exceeds 10 km / h, when the remote operator H is no longer able to monitor the oncoming lane, or when the communication delay exceeds 0.2 seconds. The driving continuation determination unit 164 performs the determination in step S404 while the vehicle 200 is driving in the ODD-mitigated driving mode, but may also perform the determination in step S404 while the vehicle 200 is driving in the remote control mode.

[0143] Next, the presentation unit 152 presents the monitoring-required area and auxiliary information to the remote operator H (S405). The monitoring-required area is an area that needs to be monitored by the remote operator H. The auxiliary information is information that is presented auxiliary during remote monitoring, and includes, for example, at least one of the vehicle speed, the driving route during automatic driving, the current driving mode, and the determination result of the driving continuation determination unit 164.

[0144] Fig. 7(c) is a diagram showing a state in which the ODD mitigation driving mode has been selected. Fig. 7(c) shows a state after the oncoming lane driving mode has been selected in Fig. 7(b). Fig. 7(c) shows that the vehicle 410 is traveling in the oncoming lane. Fig. 7(c) also shows monitoring areas that require monitoring by the remote operator H with dotted hatching.

[0145] As shown in (c) of FIG. 7, the monitoring area that requires monitoring by the remote operator H in response to the alleviation of the ODD and auxiliary information regarding the travel of the vehicle 200 are presented. The presentation unit 152, for example, causes the display device 110 to present the monitoring area and auxiliary information by superimposing them on an image from the vehicle 200. In this way, the monitoring area that requires monitoring in response to the alleviation of the ODD is presented, so that the remote operator H can easily know the area that he or she should monitor. The monitoring area that requires monitoring in response to the alleviation of the ODD is an example of a strengthened monitoring requirement.

[0146] Furthermore, the presentation unit 152 may cause the display device 110 to present the driving route, the current driving mode ("ODD-mitigated driving" shown in (c) of FIG. 7), the determination result of the driving continuation determination unit 164 ("RUN" shown in (c) of FIG. 7), and the like. This allows the remote operator H to remotely monitor the vehicle 200 with reference to the auxiliary information. Furthermore, since the vehicle 410 is present on the driving route, the presentation unit 152 may cause the display device 110 to present information calling attention, such as "watch ahead," as an alert for an approaching object. The information calling attention is also included in the auxiliary information.

[0147] Also, in (c) of FIG. 7, a button 114 indicating "normal driving" is presented as a driving mode switching button. Normal driving refers to an automatic driving mode, and when the button 114 is operated, the vehicle 200 switches to the automatic driving mode. In other words, when the button 114 is operated, the mode change unit 151 switches from the current driving mode (for example, the ODD-mitigated driving mode) to the automatic driving mode. Operating the button 114 means that the ODD-mitigated driving mode is to be ended.

[0148] 7(c), a button 115 indicating "continue driving" is further presented. The button 115 is operated by the remote operator H when the ODD-mitigated driving mode is to be continued. The button 115 is presented when the driving mode is switched to the ODD-mitigated driving mode. The operation of the button 115 by the remote operator H is an example of a second operation.

[0149] Referring again to FIG. 14, the mode change unit 151 causes the vehicle 200 to start autonomous driving in the ODD-mitigated driving mode (S406). The mode change unit 151, for example, outputs information indicating that autonomous driving in the ODD-mitigated driving mode will be started to the vehicle 200. For example, after presenting the image shown in (c) of FIG. 7 to the remote operator H, the mode change unit 151 may cause the vehicle 200 to start autonomous driving in the ODD-mitigated driving mode when detecting an operation from the remote operator H indicating that autonomous driving in the ODD-mitigated driving mode will be started (for example, an operation of touching the button 115). Note that operating the button 115 to start the ODD-mitigated driving mode may be the first operation.

[0150] In this way, the mode change unit 151 causes the vehicle 200 to automatically travel (drive) with relaxed ODD.

[0151] When the autonomous driving unit 232 of the vehicle 200 acquires information indicating that autonomous driving in the ODD-mitigated driving mode is to be started, the autonomous driving unit 232 generates a command for starting the autonomous driving and outputs the command to the vehicle control unit 240. As a result, the vehicle 200 starts autonomous driving in the ODD-mitigated driving mode.

[0152] In this way, when the selection of the driving mode is accepted in step S106, for example, the switching process in step S107 is executed once.

[0153] 4 again, next, the driving continuation possibility determination unit 164 executes a driving continuation determination process for the vehicle 200 that is automatically driving in the ODD-mitigated driving mode (S108). FIG. 15 is a flowchart showing an example of the operation of the determination process shown in step S108 of FIG.

[0154] 15, when the current driving mode is the ODD-mitigated driving mode or the remote control mode, the driving continuation determination unit 164 determines whether driving in the current driving mode can be continued. The driving continuation determination unit 164 makes this determination based on whether the current state of the vehicle control system 10 satisfies a driving continuation condition corresponding to the current driving mode. The driving continuation determination unit 164 makes this determination repeatedly at predetermined time intervals, for example. For example, when the current driving mode is the ODD-mitigated driving mode, the driving continuation determination unit 164 determines whether automated driving in the ODD-mitigated driving mode can be continued (S407).

[0155] The determination process in step S407 will now be described with reference to Fig. 16. Fig. 16 is a flowchart showing the determination process shown in step S407 in Fig. 15.

[0156] 16, the traveling continuation possibility determination unit 164 acquires second vehicle information from the vehicle 200 (S501). The second vehicle information (see FIG. 6) is acquired sequentially from the vehicle 200 regardless of the traveling mode.

[0157] Next, the traveling continuation determination unit 164 acquires second remote operation information (S502). The second remote operation information indicates the current state of the remote operator H. Fig. 17 is a diagram showing an example of the second remote operation information.

[0158] 17, the second remote operation information includes the fatigue level of the remote operator H as the current state of the remote operator H. It can also be said that the continuation of traveling determination unit 164 acquires the fatigue level of the remote operator H as the current state of the remote operator H. The fatigue level indicates the current degree of fatigue of the remote operator H, and is expressed as a level or a numerical value such as "high," "medium," or "low." In the example of FIG. 17, the continuation of traveling determination unit 164 acquires that the fatigue level of the remote operator H is "low."

[0159] The second remote operation information may also include information other than the degree of fatigue. The second remote operation information may further include information related to remote monitoring of the remote operator H. The second remote monitoring information may include, for example, a determination result as to whether the remote operator H is looking at the display device 110 or information related to the line of sight of the remote operator H. In other words, the second remote operation information may include a determination result as to whether the remote operator H is performing necessary remote monitoring. Note that the second remote operation information may include at least one of the degree of fatigue and information related to remote monitoring.

[0160] Referring again to FIG. 16, next, the continuing driving possibility determination unit 164 acquires second system information (S503). The second system information indicates the current state of the vehicle control system 10. The second system information indicates, for example, the network state such as communication delay and communication bandwidth, and the current health state of the vehicle control system 10 such as the computational resources of the remote operation device 130 and the operating state of the program. The continuing driving possibility determination unit 164 may acquire the second system information via, for example, the system information acquisition unit 166. FIG. 18 is a diagram showing an example of the second system information.

[0161] As shown in FIG. 18 , the second system information includes, as the current state of the vehicle control system 10, a communication delay in the vehicle control system 10 and a system state of the vehicle control system 10. It can also be said that the driving continuation determination unit 164 acquires the current communication delay and system state as the current state of the vehicle control system 10. In the example of FIG. 18 , the driving continuation determination unit 164 acquires, as the criteria for determining whether or not driving is possible, that the communication delay is 0.2 seconds and that the system state is normal (healthy). Note that the method for acquiring the second system information is not particularly limited. For example, when acquiring information from the vehicle 200, the driving continuation determination unit 164 may calculate the current communication delay based on the time when the information was acquired and the timestamp included in the information. Furthermore, the driving continuation determination unit 164 may acquire the current system state from, for example, a monitoring device (not shown) that monitors the system state of the vehicle control system 10.

[0162] 16 again, the continuation of driving determination unit 164 determines whether or not a continuation of driving condition corresponding to the current driving mode is satisfied based on the information acquired in steps S501 to S503 (S504). It can also be said that the continuation of driving determination unit 164 determines whether or not the current state of the vehicle control system 10 satisfies the continuation of driving condition.

[0163] If all of the second vehicle information, the second remote operation information, and the second system information satisfy the driving continuation condition (Yes in S504), the driving continuation determination unit 164 determines that the automatic driving in the ODD-mitigated driving mode can be continued (S505), and proceeds to step S408 in Fig. 15. Furthermore, if the second remote monitoring information includes a determination result of whether the remote operator H is looking at the display device 110 or information regarding the line of sight of the remote operator H, the driving continuation determination unit 164 may determine Yes in step S504 if the remote operator H is looking at the display device 110 or if the line of sight of the remote operator H is directed toward the display device 110 (for example, directed toward a monitoring-required area of ​​the display device 110). In other words, the driving continuation determination unit 164 may determine Yes in step S504 if the remote operator H is monitoring an enhanced monitoring requirement. As a result, only when the remote operator H is performing monitoring with the strengthened monitoring requirements, the vehicle 200 can be made to automatically travel with relaxed ODD.

[0164] In addition, if at least one of the second vehicle information, the second remote control information, and the second system information does not satisfy the driving continuation conditions (No in S504), the driving continuation determination unit 164 determines that automatic driving in the ODD mitigation driving mode cannot be continued (S506), and proceeds to step S410 in Figure 15.

[0165] 15 again, when the driving continuation possibility determination unit 164 determines that the automatic driving in the ODD-mitigated driving mode can be continued, that is, the driving continuation condition is satisfied (Yes in S407), the driving continuation possibility determination unit 164 determines whether or not the command receiving unit 153 has received a command to continue driving from the remote operator H (S408). On the other hand, when the driving continuation possibility determination unit 164 determines that the automatic driving in the ODD-mitigated driving mode cannot be continued, that is, the driving continuation condition is not satisfied (No in S407), the driving continuation possibility determination unit 164 stops the driving of the vehicle 200 (S410), proceeds to step S102 shown in FIG. 4, and notifies the remote operator H. Note that when the result in step S407 is No, the driving of the vehicle 200 is not limited to being stopped, and a process of switching the driving mode may be performed. In this case, at least one of the ODD-mitigated driving mode and the remote operation mode to which the switch can be made is notified to the remote operator H. The switchable ODD-mitigated driving mode here is a driving mode for which the determination in step S407 is Yes, and is, for example, a driving mode in which the monitoring burden on the remote operator H increases or driving restrictions increase compared to the current ODD-mitigated driving mode. In this case, the driving of the vehicle 200 does not need to be stopped.

[0166] Fig. 7(d) is a diagram showing a state in which the vehicle is traveling in the ODD-mitigated driving mode. Fig. 7(d) shows a state in which the vehicle has traveled in the oncoming lane in the ODD-mitigated driving mode from the state in Fig. 7(c) and has overtaken vehicle 400.

[0167] As shown in FIG. 7(d), the continue driving determination unit 164 makes a determination in step S408 based on whether or not the remote operator H has operated the button 115 indicating "continue driving."

[0168] Note that while the remote operator H continues to operate (for example, continues to touch) the button 115, traveling in the ODD-reduced traveling mode may continue. In this case, traveling in the ODD-reduced traveling mode is performed only while the remote operator H is operating the button 115. Detection of the operation of the button 115 by the remote operator H corresponds to, for example, monitoring by the remote operator H. Monitoring may be, for example, a period during which the operation of the button 115 is detected. This allows the vehicle 200 to be automatically driven with relaxed ODD only while the remote operator H is monitoring with strengthened monitoring requirements (only while the button 115 is being operated).

[0169] Furthermore, driving in the ODD-mitigated driving mode may be performed for a certain time, a certain section, or a certain distance each time the remote operator H operates the button 115. The remote operator H can continue automatic driving in the ODD-mitigated driving mode by continuing to operate the button 115 at predetermined time intervals.

[0170] The process of step S408 will now be described with reference to Fig. 19 and Fig. 20. Fig. 19 is a flowchart showing the determination process shown in step S408 of Fig. 15. Fig. 20 is a diagram showing an example of a command.

[0171] 19, the continuing traveling possibility determination unit 164 receives a command from the remote operator H from the command receiving unit 153 (S601). For example, the continuing traveling possibility determination unit 164 receives information indicating that the remote operator H has operated the button 115 from the command receiving unit 153.

[0172] Next, the continuation of traveling determination unit 164 checks the timestamp included in the command (S602). That is, the continuation of traveling determination unit 164 checks whether the received command is a command issued when the remote operator H operated the button 115.

[0173] The continuing traveling possibility determination unit 164 may check the timestamp to determine whether the command is a command acquired by an operation of the remote operator H within a predetermined time period or whether it is the most recent command. If the command is a command acquired by an operation of the remote operator H within a predetermined time period or is the most recent command, the continuing traveling possibility determination unit 164 may proceed to step S603, and if the command is a command acquired by an operation of the remote operator H outside the predetermined time period or is an old command, the unit may proceed to step S411.

[0174] 20, the travel permission command is a command generated when remote operator H operates button 115, and includes a timestamp and a command. The timestamp indicates, for example, the date and time when remote operator H operates button 115. The command is information corresponding to the operated button, and indicates that travel should continue when button 115 is operated.

[0175] 19 again, next, the continuing driving determination unit 164 determines whether the communication delay is equal to or less than a certain value (S603). The continuing driving determination unit 164, for example, acquires the current communication delay time, and if the acquired communication delay satisfies the continuing driving condition, determines that the communication delay is equal to or less than a certain value. This makes it possible to determine whether to continue driving in the ODD-mitigating driving mode in accordance with the real-time operation of the button 115 by the remote operator H.

[0176] If the communication delay is equal to or less than a certain level (Yes in S603) and the command acquired in step S601 is a command indicating that driving should be continued (Yes in S604), the continuation of driving determination unit 164 determines to proceed to step S409 shown in Fig. 15. If the communication delay is not equal to or less than a certain level (No in S603) or the command acquired in step S601 is not a command indicating that driving should be continued (No in S604), the continuation of driving determination unit 164 determines to proceed to step S411 shown in Fig. 15.

[0177] 15 again, if the answer is Yes in step S408, the driving continuation determination unit 164 determines to continue automatic driving in the ODD-mitigated driving mode (S409) and proceed to step S109 in Fig. 4. As a result, automatic driving in the ODD-mitigated driving mode is continued only when the remote operator H permits the continuation of automatic driving in the ODD-mitigated driving mode. Therefore, the safety of automatic driving in the ODD-mitigated driving mode can be further ensured.

[0178] Furthermore, if the command to continue driving has not been received (No in S408), the driving continuation determination unit 164 stops driving of the vehicle 200 (S411) and proceeds to step S408. The stopping of driving here is a temporary stop, and if a command to continue driving is received again after driving has stopped (Yes in step S408), the vehicle 200 may resume driving in the ODD-mitigating driving mode.

[0179] In this way, when continuing driving in the ODD-reduced driving mode, the remote operator H can continue the ODD-reduced driving mode simply by operating (for example, touching) the button 115. Therefore, even when continuing the ODD-reduced driving mode, the burden on the remote operator H can be reduced. In addition, the determination process shown in FIG. 15 is repeatedly executed while driving in the ODD-reduced driving mode.

[0180] If the current driving mode is the remote control mode, the determination process of step S408 is not performed (see FIG. 22). In other words, if the current driving mode is the remote control mode, and the driving continuation possibility determination unit 164 determines that driving in the remote control mode is possible, driving in the remote control mode is continued.

[0181] 4 again, when the mode change unit 151 acquires an operation to switch to the automatic driving mode while traveling in the ODD-mitigated driving mode or the remote control mode (Yes in S109), it switches the driving mode to the automatic driving mode (S111), returns to step S101, and continues the subsequent processing. Furthermore, when the mode change unit 151 does not acquire an operation to switch to the automatic driving mode while traveling in the ODD-mitigated driving mode or the remote control mode (No in S109), it determines whether or not the destination has been reached (S110). If the destination has been reached (Yes in S110), the remote monitoring and remote operation of the vehicle 200 by the remote operator H is terminated. Furthermore, if the destination has not been reached (No in S110), the mode change unit 151 proceeds to step S108, and continues the subsequent processing.

[0182] Furthermore, if there is no driving mode (No in S104), the presentation unit 152 causes the display device 110 to present that driving is not possible, and ends the driving of the vehicle 200 (S113).

[0183] The processes of steps S103 to S105, S112 and S113 shown in FIG. 4 can also be said to be processes for determining whether or not mode switching is possible.

[0184] Next, the operation when the command receiving unit 153 receives the selection of the remote operation mode, for example, when the remote operator H operates the button 113 in the image of Fig. 7(b), will be described with reference to Fig. 21. Fig. 21 is a flowchart showing another example of the operation for controlling traveling shown in step S107 of Fig. 4.

[0185] 21, the area determination unit 161 sets a monitoring area for the remote operator H (S701). The area determination unit 161 sets the monitoring area for the remote operator H so as to satisfy the constraint conditions for the remote operation mode. For example, the area determination unit 161 sets an area that is set as a monitoring area in the safety monitoring conditions as the monitoring area for the remote operator H. The monitoring area in the safety monitoring conditions may be the entire area around the vehicle 200.

[0186] The remote control possibility determination unit 163 may determine whether the remote control device 130 has the capability to monitor the monitoring area set by the area determination unit 161 based on the first remote control information, and if the remote control device 130 does not have the capability, may output information indicating that the monitoring-required area indicated by the constraint condition cannot be monitored to the mode change unit 151. In this case, the mode change unit 151 may search again for another driving mode (for example, the ODD-mitigated driving mode) or may stop the vehicle 200 from traveling.

[0187] Next, the traveling continuation determination unit 164 sets a traveling continuation condition in the remote control mode (S702). The traveling continuation condition here is a criterion for determining whether or not to continue traveling in the remote control mode. The traveling continuation determination unit 164 may set, for example, a constraint condition of the remote control mode as the traveling continuation condition. The constraint condition of the remote control mode includes, for example, at least some of the conditions shown in (a) to (c) of FIG. 10.

[0188] Next, the presentation unit 152 presents the monitoring area and auxiliary information to the remote operator H (S703). The monitoring area is an area that needs to be monitored by the remote operator H. The auxiliary information is information that is presented as an auxiliary information during remote operation, and includes at least one of the vehicle speed, the current driving mode, and the determination result of the driving continuation determination unit 164, for example.

[0189] Next, the mode change unit 151 causes the vehicle 200 to start autonomous traveling in the remote control mode (S704). For example, when the operation receiving unit 154 acquires an operation accepted by the operation input device 120, the operation receiving unit 154 generates control information based on the acquired operation and outputs it to the vehicle 200, thereby causing the vehicle 200 to start traveling in the remote control mode.

[0190] In this way, when the selection of the driving mode is accepted in step S106, for example, the switching process in step S107 is executed once.

[0191] 4 again, next, the traveling continuation possibility determination unit 164 executes a traveling continuation determination process for the vehicle 200 traveling in the remote control mode (S108). FIG. 22 is a flowchart showing another example of the operation of the determination process shown in step S108 of FIG.

[0192] 22, next, the traveling continuation determination unit 164 determines whether traveling in the remote control mode can be continued (S705). If the traveling continuation determination unit 164 determines that traveling in the remote control mode can be continued, that is, that the traveling continuation condition is satisfied (Yes in S705), the traveling continuation determination unit 164 continues traveling in the remote control mode (S706) and proceeds to step S109 in FIG. 4. On the other hand, if the traveling continuation determination unit 164 determines that traveling in the remote control mode cannot be continued, that is, that the traveling continuation condition is not satisfied (No in S705), the traveling of the vehicle 200 is stopped (S707), and the process proceeds to step S102 in FIG. 4, where a notification is sent to the remote operator H.

[0193] If the result of step S705 is No, the process is not limited to stopping the vehicle 200 from traveling, and a process of switching the traveling mode may be performed. For example, if the vehicle 200 is a manually operable vehicle, a process of switching to the manual driving mode may be performed. In this case, information indicating switching to the manual driving mode is notified to the remote operator H. Also, in this case, the traveling of the vehicle 200 does not have to be stopped. If selectable, the vehicle 200 may be switched to the ODD mitigation mode.

[0194] The determination process shown in FIG. 22 is repeatedly executed while the vehicle is traveling in the remote control mode.

[0195] [3. Application Examples] 23 to 32C, application examples when switching to the ODD-reduced driving mode under various circumstances will be described. Note that the application examples described below are merely examples, and the ODD and constraint conditions corresponding to the circumstances of the vehicle 200 are not limited to those described below. For convenience, the following will describe a case where switching from the automatic driving mode to the ODD-reduced driving mode is performed.

[0196] First, the first application example will be described with reference to Figs. 23 to 24B. Fig. 23 is a diagram for explaining the situation of the first application example in which ODD is alleviated. Fig. 23 is, for example, an image displayed on the display device 110, and is, for example, an image being checked by the remote operator H. Fig. 24A is a diagram showing various conditions (ODD and constraint conditions) in the automatic driving mode of the first application example. Fig. 24B is a diagram showing various conditions (ODD and constraint conditions) in the ODD-alleviated driving mode of the first application example.

[0197] As a first application example, a case where the types of objects that are subject to overtaking by ODD will be relaxed will be described. Specifically, a case where a vehicle 420 that is not pulled over to the shoulder is stopped in front of the vehicle 200, and the object that is subject to overtaking by ODD in the automatic driving mode does not include the vehicle that is not pulled over to the shoulder will be described. Note that a frame surrounding the vehicle 420 is illustrated to indicate that the vehicle 200 has detected the vehicle 420 by sensing.

[0198] As shown in Fig. 23, vehicle 200 is stopped in front of vehicle 420 (vehicle in front) that has not pulled over to the shoulder, but since vehicle 420 that has not pulled over to the shoulder is not included in the objects that can be overtaken by ODD, vehicle 200 cannot overtake in the automatic driving mode. Note that an x ​​mark in Fig. 23 indicates that vehicle 200 cannot be driven. The x mark may be included in the auxiliary information.

[0199] 24A, in the automatic driving mode, the objects to be overtaken by the ODD include a vehicle (pulling over to the shoulder) and a traffic cone (registered trademark). In other words, in the automatic driving mode, the vehicle 200 cannot overtake a vehicle 420 that is not pulling over to the shoulder.

[0200] Therefore, the remote control device 130 switches from the automatic driving mode to the ODD-mitigated driving mode. Specifically, the remote control device 130 switches to the ODD-mitigated driving mode, which allows overtaking of vehicles that have not pulled over to the shoulder. In step S103 of FIG. 4, the remote control device 130 searches for an ODD-mitigated driving mode, which allows overtaking of vehicles that have not pulled over to the shoulder, from the multiple ODD-mitigated driving modes stored in the storage unit 167. FIG. 24B shows, for example, various conditions in the ODD-mitigated driving mode searched for in step S103.

[0201] As shown in Fig. 24B, a vehicle (vehicle ahead) and a pedestrian (stationary) are added to the objects that can be overtaken by ODD in the ODD-reduced driving mode. That is, in the ODD-reduced driving mode, the vehicle 200 can overtake a vehicle 420 that is not pulling over to the shoulder. Furthermore, instead of relaxing the ODD conditions, the constraints in the ODD-reduced driving mode are stricter than the constraints in the automatic driving mode. For example, the vehicle conditions limit the maximum vehicle speed from 20 km / h to 5 km / h.

[0202] In this way, the restriction on the driving mode of the autonomous driving is strengthened in response to the relaxation of the ODD. For example, in response to the relaxation of the ODD, the restriction on at least one of the maximum vehicle speed, maximum steering angle, and maximum acceleration during autonomous driving with the relaxed ODD is strengthened. Specifically, the value of at least one of the maximum vehicle speed, maximum steering angle, and maximum acceleration is changed to a smaller value. In the example of FIG. 24B, the restriction on the maximum vehicle speed is strengthened.

[0203] Furthermore, as a condition for safety monitoring, a preceding vehicle is added to the list of objects requiring remote monitoring. In other words, to drive in the ODD mitigation driving mode, remote operator H needs to monitor the preceding vehicle.

[0204] In this way, the monitoring requirements of the remote operator H for the vehicle 200 are strengthened in response to the relaxation of the ODD. Strengthening the monitoring requirements includes, for example, strengthening the monitoring of at least one of the monitoring area and the monitoring object that the remote operator H monitors during autonomous driving with the relaxed ODD in response to the relaxation of the ODD. It can also be said that the monitoring area and the monitoring object that the remote operator H monitors are changed to the monitoring area and the monitoring object that are monitored during autonomous driving when the ODD is relaxed. Specifically, this includes widening the monitoring area or increasing the number of monitoring objects. In the example of FIG. 24B, monitoring of the monitoring object is strengthened.

[0205] If the constraint conditions shown in FIG. 24B are satisfied, the vehicle 200, under the supervision of the remote operator H, will be able to overtake the vehicle 420 that is not pulling over to the shoulder during automatic traveling.

[0206] Next, a second application example will be described with reference to Fig. 25 to Fig. 26B. Fig. 25 is a diagram for explaining the situation of the second application example in which ODD is alleviated. Fig. 26A is a diagram showing various conditions in the automatic driving mode of the second application example. Fig. 26B is a diagram showing various conditions in the ODD-alleviated driving mode of the second application example.

[0207] As a second application example, a case where the ODD drivable area is relaxed will be described. Specifically, the case where vehicle 430 that is not pulling over to the shoulder is parked in front of vehicle 200 and the drivable area in the automatic driving mode does not include an oncoming lane will be described.

[0208] As shown in Figure 25, vehicle 200 is facing vehicle 430 that is stopped in front of it and has not pulled over to the shoulder. To overtake vehicle 430, vehicle 200 must travel in the oncoming lane. However, since the oncoming lane is not included in the ODD's driving area, overtaking cannot be performed in the automatic driving mode.

[0209] 26A, in the autonomous driving mode, the ODD driving area includes only the vehicle's own lane. In other words, in the autonomous driving mode, vehicle 200 cannot overtake vehicle 430 by driving in the oncoming lane.

[0210] Therefore, the remote control device 130 switches from the automatic driving mode to the ODD-mitigated driving mode. Specifically, the remote control device 130 switches to the ODD-mitigated driving mode in which driving in the oncoming lane is also possible. In step S103 of FIG. 4, the remote control device 130 searches for an ODD-mitigated driving mode in which driving in the oncoming lane is possible from the multiple ODD-mitigated driving modes stored in the storage unit 167. FIG. 26B shows, for example, various conditions in the ODD-mitigated driving mode searched for in step S103.

[0211] As shown in FIG. 26B, an oncoming lane is added to the drivable area in the ODD-reduced driving mode. That is, in the ODD-reduced driving mode, vehicle 200 can travel in the oncoming lane and overtake vehicle 430. Moreover, instead of relaxing the ODD conditions, the constraint conditions in the ODD-reduced driving mode are stricter than the constraint conditions in the automatic driving mode. The vehicle conditions are such that the maximum vehicle speed is limited from 20 km / h to 5 km / h. In this way, in the example of FIG. 26B, the limit on the maximum vehicle speed is tightened.

[0212] Furthermore, as a condition for safety monitoring, the oncoming lane is added to the monitoring area. In other words, to drive in the ODD mitigation driving mode, the remote operator H needs to monitor the oncoming lane. In this way, in the example of Fig. 26B, the monitoring area is expanded.

[0213] If the constraint conditions shown in FIG. 26B are satisfied, the vehicle 200 will be able to travel in the oncoming lane and overtake the vehicle 430 in the autonomous driving mode under the supervision of the remote operator H.

[0214] Next, a third application example will be described with reference to Figs. 27 to 28B. Fig. 27 is a diagram for explaining the situation of the third application example in which ODD is alleviated. Fig. 28A is a diagram showing various conditions in the automatic driving mode of the third application example. Fig. 28B is a diagram showing various conditions in the ODD-alleviated driving mode of the third application example.

[0215] As a third application example, a case where the driving prohibition conditions of the ODD are relaxed will be described. The driving prohibition conditions are conditions that prohibit the vehicle 200 from driving. Specifically, a case where an earthquake occurs while the vehicle is driving in the autonomous driving mode and the driving prohibition conditions in the autonomous driving mode include an earthquake (seismic intensity: 3 or higher) will be described. Note that the vehicle 200 acquires, for example, information related to weather or disasters such as the occurrence of an earthquake from an external device via communication.

[0216] As shown in FIG. 27, when an earthquake with a seismic intensity of 3 or more occurs, the vehicle 200 stops because the occurrence of an earthquake (seismic intensity: 3 or more) is included in the conditions for disabling travel of the ODD.

[0217] 28A, in the autonomous driving mode, the ODD driving disable conditions include weather and disaster conditions. Specifically, the ODD driving disable conditions include the occurrence of an earthquake (seismic intensity: 3 or higher), heavy rain (precipitation: 200 mm or higher), dense fog (visibility: 60 m or lower), and strong winds (wind speed: 15 m / s or higher). In other words, in the autonomous driving mode, the vehicle 200 cannot travel if an earthquake with a seismic intensity of 3 or higher occurs.

[0218] Therefore, the remote control device 130 switches from the automatic driving mode to the ODD-reduced driving mode. Specifically, the remote control device 130 switches to the ODD-reduced driving mode, which allows driving even when an earthquake of seismic intensity 3 or higher occurs. In step S103 of FIG. 4, the remote control device 130 searches for an ODD-reduced driving mode, which allows driving even when an earthquake of seismic intensity 3 or higher occurs, from the multiple ODD-reduced driving modes stored in the storage unit 167. FIG. 28B shows, for example, various conditions for the ODD-reduced driving mode searched for in step S103.

[0219] As shown in FIG. 28B, earthquakes have been removed from the driving prohibition conditions in the ODD-mitigated driving mode. That is, in the ODD-mitigated driving mode, the vehicle 200 can drive even if an earthquake occurs. In addition, instead of mitigating the ODD conditions, the constraint conditions in the ODD-mitigated driving mode are stricter than the constraint conditions in the automatic driving mode. The vehicle conditions limit the maximum vehicle speed from 20 km / h to 10 km / h. In the example of FIG. 28B, the limit on the maximum vehicle speed is tightened.

[0220] Furthermore, as a condition for safety monitoring, a double check of the monitoring area by the vehicle 200 and the remote operator H is added. In other words, to travel in the ODD-mitigated driving mode, the remote operator H also needs to monitor the vehicle's own lane. In the example of FIG. 28B, the restriction on the monitoring area is strengthened.

[0221] 28B is satisfied, vehicle 200 can automatically travel even when an earthquake occurs under the supervision of remote operator H. This allows vehicle 200 to evacuate to a safe place when an earthquake occurs, for example.

[0222] Next, a fourth application example will be described with reference to Figs. 29 to 30B. Fig. 29 is a diagram for explaining the situation of the fourth application example in which ODD is alleviated. Fig. 30A is a diagram showing various conditions in the automatic driving mode of the fourth application example. Fig. 30B is a diagram showing various conditions in the ODD-alleviated driving mode of the fourth application example. Note that in Fig. 29, areas where the road surface of road L is frozen are indicated by dotted hatching.

[0223] As a fourth application example, a case where the ODD impassable area state is alleviated will be described. The impassable area state is a road surface state that prohibits the vehicle 200 from traveling. Specifically, a case where the road surface is frozen and the impassable area state in the automatic traveling mode includes freezing will be described.

[0224] As shown in FIG. 29, if the road surface in the traveling direction is frozen, the vehicle 200 stops because the ODD impassable area status includes frozen road surface.

[0225] 30A, in the autonomous driving mode, the ODD non-travelable area states include frozen roads and flooded roads (10 cm or more). In other words, in the autonomous driving mode, the vehicle 200 cannot travel if the road is frozen.

[0226] Therefore, the remote control device 130 switches from the automatic driving mode to the ODD-reduced driving mode. Specifically, the remote control device 130 switches to the ODD-reduced driving mode in which driving is possible even on icy roads. In step S103 of Fig. 4, the remote control device 130 searches for an ODD-reduced driving mode in which driving is possible even on icy roads from the multiple ODD-reduced driving modes stored in the storage unit 167. Fig. 30B shows, for example, various conditions in the ODD-reduced driving mode searched for in step S103.

[0227] As shown in FIG. 30B, the "frozen road" condition is deleted from the non-travelable area state in the ODD-reduced driving mode. That is, in the ODD-reduced driving mode, the vehicle 200 can travel even if the road is frozen. In addition, instead of relaxing the ODD conditions, the constraints in the ODD-reduced driving mode are stricter than the constraints in the automatic driving mode. The vehicle conditions are limited to a maximum vehicle speed of 5 km / h from 20 km / h and a maximum acceleration of 0.1 G from 0.3 G. In the example of FIG. 30B, the constraints on the maximum vehicle speed and maximum acceleration are tightened.

[0228] Furthermore, as a condition for safety monitoring, a double check of the monitoring area by the vehicle 200 and the remote operator H is added. In other words, to travel in the ODD-mitigated driving mode, the remote operator H also needs to monitor the vehicle's own lane. In the example of FIG. 30B, the restrictions on the monitoring area are strengthened.

[0229] If the constraints shown in FIG. 30B are satisfied, the vehicle 200 will be able to automatically travel at a low speed under the supervision of the remote operator H even if the road surface is frozen.

[0230] Next, a fifth application example will be described with reference to Figs. 31 to 32C. Fig. 31 is a diagram for explaining the situation of the fifth application example in which ODD is alleviated. Fig. 32A is a diagram showing various conditions in the automatic driving mode of the fifth application example. Fig. 32B is a diagram showing various conditions in the first ODD-alleviated driving mode of the fifth application example. Fig. 32C is a diagram showing various conditions in the second ODD-alleviated driving mode of the fifth application example. Note that Fig. 31(a) is the same as Fig. 27.

[0231] As a fifth application example, a case where ODD is further alleviated while driving in the ODD-mitigated driving mode will be described. That is, as the fifth application example, a case where ODD is alleviated in multiple stages will be described. Specifically, since an earthquake occurs while driving in the automatic driving mode, the driving mode is switched to an ODD-mitigated driving mode (first ODD-mitigated driving mode) that allows driving even in an earthquake, and since the vehicle 440 is stopped by pulling over to the shoulder while driving in the ODD-mitigated driving mode, the driving mode is further switched to an ODD-mitigated driving mode (second ODD-mitigated driving mode) that allows driving even in a situation where an earthquake occurs and the vehicle 440 is stopped by pulling over to the shoulder.

[0232] First, we will explain what happens when an earthquake occurs while driving in automatic driving mode.

[0233] As shown in (a) of Figure 31, when an earthquake of seismic intensity 3 or higher occurs, the vehicle 200 stops because the occurrence of an earthquake (seismic intensity: 3 or higher) is included in the ODD driving disable conditions. At this time, the vehicle 200 can be driven by switching to an ODD mitigation driving mode that allows driving even when an earthquake of seismic intensity 3 or higher occurs.

[0234] As shown in Fig. 32A, in the automatic driving mode, the conditions under which the ODD cannot travel include the occurrence of an earthquake (seismic intensity: 3 or higher), heavy rain (precipitation: 200 mm or higher), dense fog (visibility: 60 m or lower), and strong winds (wind speed: 15 m / s or higher). In other words, in the automatic driving mode, the vehicle 200 cannot travel if an earthquake with a seismic intensity of 3 or higher occurs. Note that, in the automatic driving mode, no object is set as an object for the ODD to overtake.

[0235] Therefore, the remote control device 130 switches from the automatic driving mode to the ODD-mitigating driving mode. This driving mode switching is similar to that in the third application example, and the explanation will be simplified. It is assumed that the remote control device 130 searches for the first ODD-mitigating driving mode shown in FIG. 32B in step S103 in FIG. 4.

[0236] As shown in Figure 32B, earthquakes are excluded from the driving prohibition conditions in the first ODD-reduced driving mode, so driving is possible even if an earthquake occurs. Furthermore, the constraints in the first ODD-reduced driving mode are stricter than those in the automatic driving mode. In the example of Figure 32B, the maximum vehicle speed and monitoring area are restricted more strictly.

[0237] If the constraints shown in FIG. 32B are satisfied, the vehicle 200 will be able to travel automatically under the supervision of the remote operator H even if an earthquake occurs.

[0238] Next, a case will be described in which a vehicle 440 pulling over to the shoulder appears ahead while the vehicle is traveling in the first ODD mitigation traveling mode.

[0239] As shown in (b) of Figure 31, vehicle 200 is facing vehicle 440 (parked vehicle) parked in front of it, pulled over to the shoulder of the road. However, since vehicle 440 pulled over to the shoulder of the road is not included in the objects that can be overtaken by ODD, vehicle 200 cannot overtake vehicle 440 in the first ODD-mitigated driving mode.

[0240] 32B, in the first ODD-reduced traveling mode, the objects to be overtaken in the ODD mode do not include the vehicle 440 that is pulling over to the shoulder. In other words, in the first ODD-reduced traveling mode, the vehicle 200 cannot overtake the vehicle 440 that is pulling over to the shoulder.

[0241] Therefore, the remote control device 130 switches from the first ODD-mitigated driving mode to the second ODD-mitigated driving mode. Specifically, the remote control device 130 switches to the ODD-mitigated driving mode that allows the vehicle to overtake a vehicle that has pulled over to the shoulder. In step S103 of FIG. 4, the remote control device 130 searches, from the multiple ODD-mitigated driving modes stored in the storage unit 167, for an ODD-mitigated driving mode that allows the vehicle to travel even when an earthquake has occurred and that allows the vehicle to overtake a vehicle that has pulled over to the shoulder. FIG. 32C shows, for example, various conditions for the ODD-mitigated driving mode searched for in step S103.

[0242] 32C, a vehicle (pulling over to the shoulder) is added to the objects to be overtaken by ODD in the second ODD-mitigated driving mode. That is, in the second ODD-mitigated driving mode, vehicle 200 can overtake vehicle 440 that is pulling over to the shoulder in a situation where an earthquake is occurring.

[0243] In addition, instead of relaxing the ODD conditions, the constraints in the second ODD-reduced driving mode are stricter than the constraints in the first ODD-reduced driving mode. The vehicle conditions are such that the maximum vehicle speed is limited from 10 km / h to 5 km / h. In the example of Fig. 32C, the maximum vehicle speed is further restricted.

[0244] Furthermore, a preceding vehicle has been added to the list of objects requiring remote monitoring as a condition for safety monitoring. In other words, to drive in the second ODD-mitigated driving mode, remote operator H must monitor the preceding vehicle. In the example of Fig. 32C, the restrictions on objects requiring remote monitoring are tightened.

[0245] If the constraints shown in FIG. 32C are satisfied, vehicle 200, under the supervision of remote operator H, will be able to automatically overtake vehicle 440 that has pulled over to the side of the road in the event of an earthquake.

[0246] In this way, the higher the ODD relaxation level, the stricter the constraints become.

[0247] For example, the remote operation device 130 may switch from the second ODD-reduced driving mode to a third ODD-reduced driving mode that further reduces ODD, depending on the circumstances around the vehicle 200. In this way, the remote operation device 130 may change the ODD mitigation level in multiple stages depending on the driving environment of the vehicle 200, etc. Even when changing the ODD mitigation level, the processing of step S106 is executed. That is, in each of the cases of switching from the automatic driving mode to the first ODD-reduced driving mode and switching from the first ODD-reduced driving mode to the second ODD-reduced driving mode, an operation indicating permission to switch the driving mode from the remote operator H is accepted.

[0248] In the above application example, an example has been described in which the restrictions on the system conditions are not strengthened, but the present invention is not limited to this. The restrictions on the system conditions may be strengthened in response to the relaxation of the ODD. For example, in response to the relaxation of the ODD, the restrictions on the communication delay during autonomous driving with the relaxed ODD may be strengthened. Specifically, the value of the allowable communication delay may be changed to a smaller value.

[0249] Furthermore, restrictions on the processing load of the remote operation system 100 may be strengthened in response to the relaxation of the ODD. For example, a process may be performed to lower a threshold (e.g., an upper limit) of the processing load of the remote operation system 100 in response to the relaxation of the ODD. The processing load includes, for example, at least one of the processing amount and processing delay (e.g., the processing time of the remote operation device 130) in the remote operation system 100. Strengthening the restrictions on the processing load of the remote operation system 100 is an example of strengthening restrictions on processing related to monitoring by the remote operator H.

[0250] The ODD may be relaxed by relaxing at least one of the vehicle conditions. For example, the ODD may be relaxed by relaxing the maximum steering angle, i.e., by enabling sudden steering. The ODD may also be relaxed by relaxing the weather that is a driving prohibition condition of the ODD. For example, the ODD may be relaxed by lifting the driving restriction in rainy weather or by increasing the precipitation amount that determines that driving in rainy weather is not possible.

[0251] In addition, in the automatic driving in the ODD-mitigated driving mode, the driving range in the automatic driving may be limited. For example, the system conditions may set a time, section, or distance for which driving continues in the ODD-mitigated driving mode when the remote operator H operates the button 115. Furthermore, the restriction on the driving range may be strengthened by changing the time, section, or distance to a smaller value depending on the mitigation of ODD.

[0252] (Modification of the embodiment) The vehicle control system according to this modification will be described below with reference to FIG. 33. FIG. 33 is a block diagram showing the functional configuration of a vehicle control system 10a according to this modification. The vehicle control system 10a according to this modification differs from the vehicle control system 10 according to the embodiment mainly in that the vehicle 200a has multiple autonomous driving systems. Hereinafter, the vehicle control system 10a according to this modification will be described, focusing on the differences from the vehicle control system 10 according to the embodiment. Furthermore, configurations that are the same as or similar to those of the vehicle control system 10 according to the embodiment will be assigned the same reference numerals as those of the vehicle control system 10 according to the embodiment, and descriptions thereof will be omitted or simplified.

[0253] As shown in FIG. 33, the vehicle control system 10a includes a remote control device 130a and a vehicle 200a.

[0254] The vehicle 200a has an automatic driving system 260 instead of the automatic driving system 230 that the vehicle 200 of the embodiment has. The automatic driving system 260 is configured to include a plurality of automatic driving systems.

[0255] The automated driving system 260 according to this modification includes a first automated driving system 230a and a second automated driving system 230b.

[0256] The first automatic driving system 230a is a system that operates when the vehicle is traveling in an automatic driving mode, and includes a first traveling feasibility determination unit 231a and a first automatic driving traveling unit 232a.

[0257] The first driving possibility determination unit 231a executes a determination as to whether driving in the autonomous driving mode is possible. The first driving possibility determination unit 231a stores constraint conditions in the autonomous driving mode, and determines whether driving in the autonomous driving mode is possible based on the constraint conditions and vehicle information.

[0258] The first automatic driving travel unit 232a generates a travel plan for the automatic driving mode based on the vehicle information. The travel plan includes a travel route, a speed, etc. Furthermore, when the determination result of the travel continuation possibility determination unit 164 indicates that travel can be continued, the first automatic driving travel unit 232a outputs control information based on the travel plan to the vehicle control unit 240 in order to execute the generated travel plan.

[0259] The second automatic driving system 230b is a system that operates when driving in the ODD-mitigated driving mode, and includes a second driving feasibility determination unit 231b and a second automatic driving unit 232b.

[0260] The second driving possibility determination unit 231b determines whether driving in the ODD-mitigated driving mode is possible. The second driving possibility determination unit 231b stores constraint conditions in the ODD-mitigated driving mode, and determines whether driving in the ODD-mitigated driving mode is possible based on the constraint conditions and vehicle information.

[0261] The second automatic driving unit 232b generates a driving plan for the ODD mitigation driving mode based on the vehicle information. The driving plan includes a driving route, a speed, etc. Furthermore, when the driving continuation feasibility determination unit 164 determines that driving can be continued, the second automatic driving unit 232b outputs control information based on the driving plan to the vehicle control unit 240 in order to execute the generated driving plan.

[0262] In addition, in FIG. 33, an example has been described in which the automatic driving system 260 has two automatic driving systems, but the number of automatic driving systems is not particularly limited, and three or more automatic driving systems may be included.

[0263] The mode change unit 151 outputs to the vehicle 200a a signal for operating an automatic driving system corresponding to the driving mode to be switched to. For example, when driving in the automatic driving mode, the mode change unit 151 outputs to the vehicle 200a a signal for operating only the first automatic driving system 230a out of the multiple automatic driving systems. Furthermore, for example, when driving in the ODD-mitigation driving mode, the mode change unit 151 outputs to the vehicle 200a a signal for operating only the second automatic driving system 230b out of the multiple automatic driving systems. In this way, the mode change unit 151 operates only the automatic driving system corresponding to the switched driving mode out of the multiple automatic driving systems possessed by the automatic driving system 260.

[0264] Furthermore, the safety determination unit 160a of the remote operator H does not necessarily have to have a storage unit that stores the ODD and constraint conditions of the ODD-mitigated driving mode.

[0265] The remote control device 130a may have a storage unit (not shown) that stores information indicating which driving mode each of the plurality of automatic driving systems possessed by the vehicle 200a corresponds to.

[0266] As described above, the vehicle 200a may be configured to have an automated driving system for each driving mode. For example, if there are multiple ODD-mitigating driving modes, one automated driving system is provided for each of the multiple ODD-mitigating driving modes.

[0267] As a result, the mode change unit 151 only needs to transmit information about the autonomous driving system to be operated to the vehicle 200a, thereby reducing the amount of communication between the vehicle 200a and the remote control device 130a. Also, since the process of changing the settings related to the determination of whether or not the vehicle 200a is capable of driving by the driving possibility determination unit can be omitted, the time required to switch the driving mode can be shortened. Therefore, the safety of the vehicle 200a when switching the driving mode can be improved compared to when switching requires a long time.

[0268] (Other embodiments) The present disclosure has been described above based on the embodiments and modifications (hereinafter also referred to as embodiments, etc.), but the present disclosure is not limited to the above-described embodiments, etc. As long as it does not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art may make to the present embodiments, etc., or forms constructed by combining components of different embodiments, etc., may also be included within the scope of one or more aspects of the present disclosure.

[0269] For example, in the above-described embodiment, the constraints are strengthened when the ODD is relaxed, but the present invention is not limited to this. Depending on the level of relaxation of the ODD, the constraints may not be strengthened if a remote operator is performing remote monitoring.

[0270] Furthermore, in the above-described embodiments, when constraint conditions are strengthened in response to the relaxation of ODD, it is sufficient that at least one of the vehicle conditions, safety monitoring conditions, and system conditions included in the constraint conditions is strengthened.

[0271] Furthermore, in the above-described embodiment and the like, an example has been described in which driving in the ODD-reduced driving mode is continued by the remote operator operating a button to continue driving in the ODD-reduced driving mode, but this is not limiting. For example, whether to continue driving in the ODD-reduced driving mode may be determined based on the remote operator's attitude toward monitoring while driving in the ODD-reduced driving mode. The driving continuation determination unit may determine whether to continue driving in the ODD-reduced driving mode based on the remote operator's line of sight, for example. As a result, if the remote operator is not gazing at the screen of the display device while driving in the ODD-reduced driving mode, the safety monitoring condition of the constraint condition is not satisfied, and therefore driving in the ODD-reduced driving mode can be automatically stopped.

[0272] Furthermore, in the above-described embodiments, an example has been described in which the remote operator remotely monitors when the vehicle is traveling in the ODD-mitigated driving mode, but the present invention is not limited to this. The remote operator may perform some of the driving operations when traveling in the ODD-mitigated driving mode. The remote operator may control the speed of the vehicle in the ODD-mitigated driving mode, for example, by operating the accelerator pedal and the brake pedal. Even in this case, the burden on the remote operator can be reduced compared to when traveling in the remote control mode.

[0273] In the above-described embodiments, the display device presents an overhead view image, but the present invention is not limited to this. The display device may present, for example, a 360-degree image, or may recombine images to present a third-person perspective image. The display device may also present object information detected by a sensor (e.g., an object detection sensor) mounted on the vehicle. The object information includes at least one of the object's position, size, speed, etc.

[0274] The order of the processes described in the above embodiments is merely an example. The order of the processes may be changed, the processes may be executed in parallel, or some of the processes may not be executed.

[0275] Furthermore, each component described in the above embodiments may be implemented as software or, typically, as an LSI, an integrated circuit. These components may be integrated individually on a single chip, or some or all of them may be integrated on a single chip. While LSI is used here, it may also be referred to as an IC, system LSI, super LSI, or ultra LSI depending on the level of integration. Furthermore, the integration method is not limited to LSI; dedicated circuits or general-purpose processors may also be used. Field programmable gate arrays (FPGAs), which can be programmed after LSI fabrication, or reconfigurable processors, which allow the connection or settings of circuit cells within an LSI to be reconfigured, may also be used. Furthermore, if an integrated circuit technology that replaces LSI emerges due to advances in semiconductor technology or other derivative technologies, that technology may naturally be used to integrate the components.

[0276] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or time-shared by a single piece of hardware or software.

[0277] Furthermore, the remote control device provided in the vehicle control system may be realized as a single device or may be realized by multiple devices. For example, each processing unit of the remote control device may be realized by two or more devices. For example, the remote control unit and the safety determination unit may be realized by different devices (e.g., a server device). When the remote control system is realized by multiple devices, the components provided in the remote control system may be allocated to the multiple devices in any manner. Furthermore, there is no particular limitation on the communication method between the multiple devices.

[0278] Furthermore, the technology of the present disclosure may be the above-mentioned program, or a non-transitory computer-readable recording medium on which the above-mentioned program is recorded. It goes without saying that the above-mentioned program can be distributed via a transmission medium such as the Internet. For example, the above-mentioned program and a digital signal comprising the above-mentioned program may be transmitted via a telecommunications line, a wireless or wired communication line, a network such as the Internet, data broadcasting, etc. Furthermore, the above-mentioned program and a digital signal comprising the above-mentioned program may be executed by another independent computer system by being recorded on a recording medium and transferred, or by being transferred via a network, etc.

[0279] In each embodiment, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for that component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. [Industrial Applicability]

[0280] The present disclosure is widely applicable to systems that operate autonomously driven mobile bodies. [Explanation of symbols]

[0281] 10, 10a Vehicle control system (information processing system) 100 Remote Control System 110 Display device 112, 113, 114, 115 buttons 120 Operation input device 130, 130a Remote control device 140, 210 Communications Department 150 Remote control unit 151 Mode change section 152 Presentation section 153 Command reception unit (detection unit) 154 Operation reception section 155 Operation information acquisition section 160, 160a Safety judgment section 161 Area determination unit 162 Mitigation possibility determination unit 163 Remote control feasibility determination unit 164 Determination unit for determining whether or not driving can continue 165 Fallback Section 166 System Information Acquisition Unit 167 Memory section 200, 200a, 400, 410, 420, 430, 440 vehicles 220 Vehicle Information Acquisition Unit 230, 260 Autonomous Driving System 230a First Autonomous Driving System 230b Second Autonomous Driving System 231 Driving permission determination unit 231a First driving decision unit 231b Second traveling possibility determination unit 232 Autonomous Driving Unit 232a First Autonomous Driving Division 232b Second Autonomous Driving Unit 240 Vehicle control unit 250 Command acquisition section 300 Network 310 Wireless Base Station H Remote Operator L road

Claims

1. An information processing method executed by an information processing device, A mobile object that travels automatically based on a first operational design domain (ODD) that indicates an execution condition of the automatic driving is connected to the information processing device via a network, If the autonomous driving of the moving body cannot be continued, acquiring driving impossibility information indicating that autonomous driving is not possible from the moving body; When the travel impossibility information is acquired, a first command is output to the mobile body to operate the mobile body in a first mode in which the mobile body is automatically driven based on a second ODD whose execution conditions for automatic driving are relaxed compared to those of the first ODD, in a state in which a remote operator of the mobile body does not remotely operate the mobile body but remotely monitors the mobile body. Information processing methods.

2. When the moving body is operated in the first mode, the restriction on the moving mode of the automatic driving is strengthened in accordance with the second ODD. The information processing method according to claim 1 .

3. the movement mode includes at least one of a speed, a steering angle, and an acceleration of the moving body; Strengthening the limit on at least one of the maximum speed, the maximum steering angle, and the maximum acceleration during automatic driving in accordance with the second ODD. The information processing method according to claim 2 .

4. a first requirement indicating a requirement for the remote operator to monitor the moving object when the moving object is automatically traveling is set; When the moving body is operated in the first mode, a second requirement that is stronger than the first requirement is set in accordance with the second ODD. The information processing method according to claim 1 .

5. the requirements for monitoring include at least one of a monitoring area, which is an area around the mobile object that needs to be monitored by the remote operator, and a monitoring target, which is an object that needs to be monitored by the remote operator; Strengthening the requirements for monitoring includes strengthening monitoring of at least one of the monitoring area and the monitoring object in response to the second ODD. The information processing method according to claim 4.

6. the monitoring requirements include the operation of the mobile object that requires monitoring by the remote operator; Strengthening the requirements for monitoring includes adding monitoring of a travel plan or travel control information of the moving object in accordance with the second ODD. The information processing method according to claim 4.

7. displaying the enhanced monitoring requirements on a display device used by the remote operator; The mobile object is automatically driven based on the second ODD only during monitoring by the remote operator under the enhanced monitoring requirements. The information processing method according to any one of claims 4 to 6.

8. During said monitoring, the requirements for said enhanced monitoring are: a period during which an operation by the remote operator indicating that automatic driving based on the second ODD is to be continued is detected; The information processing method according to claim 7.

9. When the mobile unit is operated in the first mode, requirements for processing or communication delay related to monitoring by the remote operator are increased in response to the second ODD. The information processing method according to claim 1 .

10. An information processing system including an information processing device, A mobile object that travels automatically based on a first operational design domain (ODD) that indicates an execution condition of the automatic driving is connected to the information processing device via a network, an acquisition unit that acquires, when the autonomous driving of the moving body cannot be continued, driving impossibility information indicating that the autonomous driving is not possible from the moving body; an output unit that, when the acquisition unit acquires the travel impossibility information, outputs to the moving body a first command for operating the moving body in a first mode in which the moving body is automatically driven based on a second ODD whose execution conditions for automatic driving are relaxed compared to those of the first ODD, in a state in which a remote operator of the moving body does not remotely operate the moving body but remotely monitors the moving body. Information processing system.

11. When an operation permitting remote operation of the mobile body is obtained from the remote operator in addition to obtaining the driving prohibition information, a second command is output to operate the mobile body in a second mode in which the remote operator remotely operates the mobile body. The information processing method according to any one of claims 1 to 9.

12. When the driving-disabled information is acquired, driving mode information for switching the operation of the moving body to the first mode or the second mode is displayed on a display device used by the remote operator, When an operation permitting switching of the operation of the moving object to the first mode is received from the remote operator, the first command is output. The information processing method according to claim 11.

13. When the moving body is automatically driven based on the second ODD, a display device used by the remote operator displays an image in which a traveling area of ​​the moving body and an area in the traveling area that needs to be monitored by the remote operator are superimposed in a visible manner. The information processing method according to any one of claims 1 to 9.

14. Furthermore, the display device is caused to display at least one of information regarding the moving route of the moving object and a determination result as to whether or not the moving object is movable in the current operation mode. The information processing method according to claim 13.

15. The travel impossibility information is determined based on the first ODD and information related to the state or specifications of the moving body. The information processing method according to any one of claims 1 to 9 and 11 to 14.

Images