Remote vehicle driving system and remote vehicle driving method

Abstract

To provide a vehicle remote operation system that enables a vehicle to continue running even if there is an abnormality on a remote side.SOLUTION: A vehicle remote operation system that controls a remotely operated vehicle which is capable of remote operation by a remote operator and an in-vehicle operation through manipulation of an in-vehicle operation member provided in the vehicle, comprises: a remote-side abnormality determination unit 1421 that sequentially determines whether or not there is an abnormality in a remote-side determination target, the remote-side determination target being at least one of a remote operator and a remote-side system operated by the remote operator; and an in-vehicle operation permission unit 1422 that permits in-vehicle operation when it is determined that there is an abnormality in the remote-side determination target by the remote-side abnormality determination unit 1421.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a system and method for remotely driving a vehicle. [Background technology]

[0002] As disclosed in Patent Document 1, a technology is known in which a remote driver remotely drives a vehicle. Patent Document 2 discloses a system for remotely monitoring an autonomous vehicle. In the remote monitoring system disclosed in Patent Document 2, the autonomous vehicle is automatically stopped when communication with a remote monitoring center is interrupted. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-168893 A [Patent Document 2] JP 2019-087015 A Summary of the Invention [Problem to be solved by the invention]

[0004] In a system in which a remote driver remotely drives a vehicle as disclosed in Patent Document 1, if an abnormality occurs on the remote side, it is conceivable to stop the vehicle as disclosed in Patent Document 2. However, it is preferable for the vehicle to be able to continue traveling even if an abnormality occurs on the remote side.

[0005] The present disclosure has been made based on this situation, and its purpose is to provide a vehicle remote driving system and a vehicle remote driving method that allow the vehicle to continue driving even if an abnormality occurs on the remote side. [Means for solving the problem]

[0006] The above object is achieved by the combination of features as recited in the independent claims, and the subclaims define further advantageous specific examples. The reference characters in parentheses in the claims indicate a correspondence with the specific aspects described in the embodiments below as one aspect, and do not limit the disclosed technical scope. Furthermore, claims not expressly stated in the claims may be combined together unless there is a particular problem with the combination.

[0007] One disclosure of a vehicle remote driving system for achieving the above object is: The vehicle can be remotely driven by a remote driver (21) or driven inside the vehicle by operating an in-vehicle operating member (13) located inside the vehicle. For mobility services A vehicle remote driving system (5) for controlling a remote driving vehicle (10), a remote side abnormality determination unit (1421, 1426) for sequentially determining whether or not there is an abnormality in a remote side determination target, which is at least one of a remote driver and a remote side system (30) operated by the remote driver; an in-vehicle driving permission unit (1422, 1427) that does not permit driving in the vehicle when the remote side abnormality judgment unit does not judge that there is an abnormality in the remote side judgment object, and that permits driving in the vehicle when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment object; A driving control unit (150) that controls driving of the remote-operated vehicle; Equipped with 、 The driving control unit, when the in-vehicle driving permission unit permits in-vehicle driving, limits the driving speed of the remotely driven vehicle more than when the vehicle is driven by remote driving. It is a remote vehicle driving system. One disclosure relating to a vehicle remote driving system is: The vehicle can be remotely driven by a remote driver (21) or driven inside the vehicle by operating an in-vehicle operating member (13) located inside the vehicle. For mobility services A vehicle remote driving system (5) for controlling a remote driving vehicle (10), a remote side abnormality determination unit (1421, 1426) for sequentially determining whether or not there is an abnormality in a remote side determination target, which is at least one of a remote driver and a remote side system (30) operated by the remote driver; When the remote side abnormality determination unit does not determine that there is an abnormality in the remote side determination target, driving in the vehicle is not permitted; When the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment target for and an in-vehicle driving permission unit (1422, 1427) for permitting driving in the vehicle. 、 When the in-vehicle driving permission unit permits the in-vehicle driving, the in-vehicle driving permission unit further determines whether or not to permit the specific occupant (14s) to drive in the vehicle based on occupant information regarding the specific occupant (14s) who is a passenger of the remote-operated vehicle and attempts to operate the in-vehicle operating member. It is a remote vehicle driving system. One disclosure relating to a vehicle remote driving system is: A vehicle remote driving system (5) for controlling a remote driving vehicle (10) for a mobility service that can be remotely driven by a remote driver (21) and in-vehicle driven by operating an in-vehicle operating member (13) in the vehicle, a remote side abnormality determination unit (1421, 1426) for sequentially determining whether or not there is an abnormality in a remote side determination target, which is at least one of a remote driver and a remote side system (30) operated by the remote driver; and an in-vehicle driving permission unit (1422, 1427) that, when the remote side abnormality judgment unit does not judge that there is an abnormality in the remote side judgment object, does not permit in-vehicle driving, and that, when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment object, permits in-vehicle driving, The in-vehicle driving permission unit is a vehicle remote driving system that determines whether or not there is a replacement driver for the remote driver performing remote driving, and if there is no replacement driver, restricts some of the driving functions of the remotely driven vehicle when remotely driven.

[0008] One disclosure of a vehicle remote driving method for achieving the above object is: The vehicle can be remotely driven by a remote driver (21) or driven inside the vehicle by operating an in-vehicle operating member (13) located inside the vehicle. For mobility services A vehicle remote driving method for controlling a remotely driven vehicle (10), comprising: Sequentially judging whether or not there is an abnormality in at least one of the remote driver and the remote side system (30) operated by the remote driver; If it is not determined that there is an abnormality in the remote judgment target, driving in the vehicle is not permitted, and if it is determined that there is an abnormality in the remote judgment target, driving in the vehicle is permitted. death , When in-vehicle driving is permitted, the speed of the remotely driven vehicle is limited more than when the vehicle is driven remotely. do, This is included in the processing executed by the processing unit. A method for remotely driving a vehicle. One disclosure relating to a vehicle remote driving method is: The vehicle can be remotely driven by a remote driver (21) or driven inside the vehicle by operating an in-vehicle operating member (13) located inside the vehicle. For mobility services A vehicle remote driving method for controlling a remotely driven vehicle (10), comprising: Sequentially judging whether or not there is an abnormality in at least one of the remote driver and the remote side system (30) operated by the remote driver; If it is not determined that there is an abnormality in the remote judgment target, driving in the vehicle is not permitted. When it is determined that there is an abnormality in the remote judgment target for , Driving allowed in the car death , When driving in the vehicle is permitted, it is further determined whether or not to permit the specific occupant (14s) to drive in the vehicle based on occupant information regarding the specific occupant (14s) who is a passenger of the remote driving vehicle and attempts to operate the in-vehicle operating member. do, This is included in the processing executed by the processing unit. A method for remotely driving a vehicle. One disclosure relating to a vehicle remote driving method is: A vehicle remote driving method for controlling a remote-driving vehicle (10) for a mobility service that can be remotely driven by a remote driver (21) and in-vehicle driven by operating an in-vehicle operation member (13) in the vehicle, comprising: Sequentially judging whether or not there is an abnormality in at least one of the remote driver and the remote side system (30) operated by the remote driver; When it is determined that there is no abnormality in the remote judgment object, driving in the vehicle is not permitted, and when it is determined that there is an abnormality in the remote judgment object, driving in the vehicle is permitted; Determine whether there is a replacement driver for the remote driver who is performing remote driving, and if there is no replacement driver, restrict some of the driving functions of the remotely driven vehicle when remotely driven. The above is a method for remotely driving a vehicle, the method including the steps of:

[0009] According to the remote vehicle driving system and the remote vehicle driving method, in-vehicle driving is permitted when an abnormality is detected in the remote-side determination object. Therefore, even if an abnormality is detected in the remote-side determination object, the remote-driven vehicle can continue to run by in-vehicle driving in which the occupant operates the in-vehicle operating member. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing a configuration of a remote vehicle driving system 5 according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing the configuration of a remote system 30. [Diagram 3] FIG. 1 shows a configuration of a vehicle-side system 100. [Figure 4] FIG. 2 is a diagram showing functions executed by an autonomous driving ECU 142. [Diagram 5] 4 is a flowchart showing a process executed by a remote control device 36. [Figure 6] 4 is a flowchart showing a process executed by the vehicle-side system 100. [Figure 7] 7 is a flowchart showing the process executed subsequent to FIG. 6. [Figure 8] FIG. 13 is a diagram showing the configuration of a remote vehicle driving system 5 according to a second embodiment. [Figure 9] FIG. 2 is a diagram showing the configuration of a remote system 30. [Figure 10] FIG. 1 shows a configuration of a vehicle-side system 100. [Figure 11] 5 is a flowchart showing the processing executed by the automatic driving ECU 142 to respond to an abnormality on the remote side. [Figure 12] 6 is a flowchart showing a process for switching the display on the vehicle exterior display unit 135. [Figure 13]5 is a flowchart showing a process executed by the vehicle-side system 100 when a remote driver 21 is changed. [Figure 14] 6 is a flowchart showing a process for notifying a remote-controlled driver 21 or the like that an abnormality has occurred in the remote-controlled vehicle 10. [Figure 15] 4 is a flowchart showing the process executed by the remote control device 36 when an abnormality occurs in the remote-controlled vehicles 10 that are traveling in a convoy. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, a number of embodiments will be described with reference to the drawings. In addition, by assigning the same reference numerals to corresponding components in each embodiment, duplicated descriptions may be omitted. When only a part of the configuration is described in each embodiment, the configuration of the other embodiment described above can be applied to the other parts of the configuration. In addition to the combination of configurations explicitly stated in the description of each embodiment, configurations of multiple embodiments can be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.

[0012] (First embodiment) [Overall structure] 1 shows the configuration of a remote vehicle driving system 5 of the first embodiment. The remote vehicle driving system 5 includes a vehicle-side system 100 mounted on a remotely driven vehicle 10, and a remote-side system 30 installed in a remote driving facility 20. The remotely driven vehicle 10 can be remotely driven by a remote driver 21 operating the remote-side system 30. In addition, the remotely driven vehicle 10 can also be driven by an in-vehicle driver 12 sitting in a driver's seat 11 of the remotely driven vehicle 10 operating an in-vehicle operating member 13. Driving the remotely driven vehicle 10 by operating the in-vehicle operating member 13 is referred to as in-vehicle driving.

[0013] The remotely driven vehicle 10 only needs to be capable of remote driving and in-vehicle driving. Various types of vehicles can be applied to the remotely driven vehicle 10. For example, the remotely driven vehicle 10 includes passenger cars, buses, and trucks. The remotely driven vehicle 10 may also be a vehicle that runs on a dedicated road. The remotely driven vehicle 10 may also be a vehicle that is not driven by an occupant under normal circumstances, and is driven by an occupant only in an emergency. In-vehicle driving means operating the in-vehicle operating member 13 to control the movement of the remotely driven vehicle 10. Driving also includes only decelerating and stopping the remotely driven vehicle 10 without accelerating or steering.

[0014] The remotely driven vehicle 10 is capable of manual driving and automatic driving both during remote driving and in-vehicle driving. The remotely driven vehicle 10 of this embodiment is capable of automatic driving level 3 and lower levels of automatic driving. At automatic driving level 3, the vehicle-side system 100 performs all driving operations in the operational design domain. The operational design domain is hereinafter referred to as ODD. Automatic driving at levels 1 and 2 is driving in which the driver is primarily responsible for operation and surrounding monitoring. Note that when simply referred to as a driver, it means a driver who may be either the remote driver 21 or the in-vehicle driver 12.

[0015] Specific examples of the in-vehicle operation member 13 are a steering wheel, an accelerator pedal, and a brake pedal. However, since the remotely driven vehicle 10 may be a vehicle that is driven by a passenger only in an emergency, the in-vehicle operation member 13 does not have to include a steering wheel or a member with the same function as a steering wheel. For example, the in-vehicle operation member 13 may also include a button for decelerating and stopping the remotely driven vehicle 10.

[0016] The remotely driven vehicle 10 can switch between remote driving and in-vehicle driving, and both types of driving are not performed at the same time. In order to prevent the running of the remotely driven vehicle 10 from changing even if the in-vehicle operation member 13 is operated during remote driving, the in-vehicle operation member 13 and the actuator operated by the in-vehicle operation member 13 (hereinafter, the corresponding actuator) are electrically connected. During remote driving, the connection between the in-vehicle operation member 13 and the corresponding actuator is cut off.

[0017] [Configuration of remote system 30] 2 shows the configuration of the remote side system 30. The remote side system 30 includes a remote operation member 31, a remote driver monitoring device 32, a remote communication device 33, a manager notification device , a remote driver notification device 35, and a remote control device .

[0018] The remote control member 31 is a member that the remote driver 21 operates when remotely driving the vehicle. The remote control member 31 can be, for example, a steering wheel, an accelerator pedal, or a brake pedal, similar to the in-vehicle operation member 13. Also, a rod-shaped operation body can be used instead of the steering wheel. Buttons can be used instead of the accelerator pedal and the brake pedal.

[0019] The remote driver monitoring device 32 continuously monitors whether the remote driver 21 is in a state suitable for remote driving. Whether the remote driver 21 is in a state suitable for remote driving is determined from the alertness level and driving posture of the remote driver 21. The remote driver monitoring device 32 includes, for example, a near-infrared light source and a near-infrared camera, and a control unit that controls them. The remote driver monitoring device 32 uses the near-infrared camera to capture an image of the head of the remote driver 21 illuminated with near-infrared light by the near-infrared light source. The image captured by the near-infrared camera is subjected to image analysis by the control unit. The control unit continuously determines the alertness level of the driver based on the image analysis results.

[0020] In addition, when the remote-driven vehicle 10 is being remotely driven at autonomous driving level 3, the remote driver monitoring device 32 continuously monitors whether the remote driver 21 is in a state where he or she can take over driving in response to a request from the system. If the alertness level of the remote driver 21 is sufficiently high and the remote driver 21 is in a position where he or she can immediately operate the remote control member 31, the remote driver 21 is deemed to be in a state where he or she can take over driving.

[0021] The remote communication device 33 is a communication device that wirelessly communicates with the in-vehicle communication device 180 provided in the vehicle-side system 100. The remote communication device 33 may communicate with the in-vehicle communication device 180 via a public communication line network, or may communicate with the in-vehicle communication device 180 directly.

[0022] The manager notification device 34 is a device that issues various notifications to a manager (see FIG. 8, remote manager 25) who manages the remote driver 21. The manager is a person different from the remote driver 21, but is able to operate the remote side system 30. The manager notification device 34 is equipped with a display 341 and a speaker 342. The display 341 and the speaker 342 are installed in positions that are easy for the manager to recognize.

[0023] The remote driver notification device 35 is a device that issues various notifications to the remote driver 21. The remote driver notification device 35 is equipped with a display 351 and a speaker 352. The display 351 and the speaker 352 are installed near the seat where the remote driver 21 sits. If there is a manager near the remote driver 21, the remote driver notification device 35 may also function as the manager notification device 34.

[0024] The remote control device 36 can be realized by a configuration including at least one processor. For example, the remote control device 36 can be realized by a computer including a processor, a non-volatile memory, a RAM, an I / O, and a bus line connecting these components. The non-volatile memory stores a program for operating a general-purpose computer as the remote control device 36. The processor executes the program stored in the non-volatile memory while utilizing the temporary storage function of the RAM, so that the remote control device 36 operates as a driving information acquisition unit 361, a remote driver state notification unit 362, and a driving instruction value acquisition unit 363. The execution of these operations means that a method corresponding to the program is executed.

[0025] The driving information acquisition unit 361 acquires driving information from the remote communication device 33, and outputs the acquired driving information from the display 351. The driving information is information necessary for remotely driving the remotely driven vehicle 10, and is transmitted from the remotely driven vehicle 10. The driving information includes information indicating an image of the surroundings of the remotely driven vehicle 10, the vehicle speed of the remotely driven vehicle 10, and the steering angle.

[0026] The remote driver status notification unit 362 acquires the monitoring results that the remote driver monitoring device 32 judges successively from the remote driver monitoring device 32. Then, the acquired monitoring results are transmitted from the remote communication device 33 to the remote driver vehicle 10. The monitoring results include whether the remote driver 21 is in a state suitable for driving and the alertness level of the remote driver 21. If the acquired monitoring results indicate that the remote driver 21 is not in a state suitable for remote driving, the remote driver status notification unit 362 outputs an alarm from the remote driver notification device 35. If the judgment result of the remote driver monitoring device 32 does not change even if an alarm is output from the remote driver notification device 35, the remote driver status notification unit 362 outputs from the administrator notification device 34 that the remote driver 21 is abnormal.

[0027] The driving instruction value acquisition unit 363 acquires driving instruction values ​​from the remote operation member 31, and transmits the acquired driving instruction values ​​from the remote communication device 33 to the remotely operated vehicle 10. The driving instruction values ​​are values ​​that instruct how to drive the remotely operated vehicle 10. The driving instruction values ​​are values ​​that instruct, for example, the accelerator operation amount, the brake operation amount, and the steering operation amount.

[0028] [Configuration of vehicle-side system 100] Next, the configuration of the vehicle-side system 100 will be described with reference to Fig. 3. The vehicle-side system 100 includes an in-vehicle driver monitoring device 120, an in-vehicle notification unit 130, an automatic driving system 140, a driving control ECU 150, a surroundings monitoring sensor 160, a locator 170, an in-vehicle communication device 180, and a communication bus 190 that communicatively connects these components.

[0029] When the in-vehicle driver 12 is driving, the in-vehicle driver monitoring device 120 continuously monitors whether the in-vehicle driver 12 is in a state suitable for driving. The configuration of the in-vehicle driver monitoring device 120 may be the same as that of the remote driver monitoring device 32. Whether the in-vehicle driver 12 is in a state suitable for driving can be determined from the posture and alertness level of the in-vehicle driver 12. In addition, when the remotely driven vehicle 10 is remotely driven at autonomous driving level 3, the in-vehicle driver monitoring device 120 continuously monitors whether the in-vehicle driver 12 is in a state capable of taking over driving.

[0030] The in-vehicle notification unit 130 provides various notifications to the driver. The in-vehicle notification unit 130 includes a speaker 131 and a display 132. The speaker 131 outputs sound toward the cabin of the remote-operated vehicle 10. One or more displays 132 are provided. At least one display 132 is disposed in a position visible to the driver. For example, one display 132 is disposed on the instrument panel.

[0031] The autonomous driving system 140 performs autonomous driving control on the remotely driven vehicle 10 when the remotely driven vehicle 10 is autonomously driven at least at autonomous driving level 3. The autonomous driving system 140 may also perform autonomous driving control on the remotely driven vehicle 10 at an autonomous driving level of autonomous driving level 4 or higher.

[0032] In autonomous driving level 3, the driver is not required to monitor the surroundings. However, in autonomous driving level 3, the driver is required to be in a state where he / she can take over driving when requested by the system. In this embodiment, the driver is either the remote driver 21 or the in-vehicle driver 12.

[0033] Autonomous driving level 3 requires that an ODD be satisfied. If the autonomous driving system 140 determines that the ODD will not be satisfied while performing autonomous driving at autonomous driving level 3, it requests the driver to take over driving. The ODD can be set in various ways. One or more of road conditions, geographical conditions, environmental conditions, and other conditions can be set in the ODD. One example of an ODD is to limit the road type to expressways.

[0034] In autonomous driving level 2, the driver takes the lead in driving the remotely driven vehicle 10, and the autonomous driving system 140 assists the driver in driving. In autonomous driving level 2, the driver naturally has the responsibility to monitor the surroundings.

[0035] The autonomous driving system 140 includes a driving assistance ECU 141 and an autonomous driving ECU 142. The driving assistance ECU 141 is an ECU (Electronic Control Unit) that sequentially determines vehicle control when the autonomous driving system 140 executes autonomous driving equivalent to driving assistance. The autonomous driving ECU 142 is an ECU that sequentially determines vehicle control when the autonomous driving system 140 executes autonomous driving at a level where the driver is not required to monitor the surroundings.

[0036] The autonomous driving ECU 142 is a computer including a processing unit 143, a RAM 144, a storage unit 145, an input / output interface 146, and a bus connecting these units. The driving assistance ECU 141 is also a computer including a processing unit, a RAM, a storage unit, an input / output interface, and a bus connecting these units.

[0037] The processing unit 143 includes at least one processor. The processing unit 143 executes various processes for implementing the vehicle remote driving method of the present disclosure by accessing the RAM 144. The storage unit 145 stores various programs (such as a vehicle remote driving program) executed by the processing unit 143. By executing the programs by the processing unit 143, the autonomous driving ECU 142 realizes the functions of each unit shown in Fig. 4. The contents of Fig. 4 will be described later.

[0038] The driving control ECU 150, which is a driving control unit, is an electronic control device mainly including a microcontroller. The driving control ECU 150 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The driving control ECU 150 continuously performs brake force control, output control of the in-vehicle power source, and steering angle control based on one of a driving instruction value based on the driving operation of the in-vehicle driver 12 or the remote driver 21, and a control instruction value of the driving assistance ECU 141 or the automatic driving ECU 142.

[0039] The driving instruction value based on the driving operation of the driver 12 in the vehicle is a detection value detected by the operation amount detection sensor 151. The operation amount detection sensor 151 is a sensor that detects the operation amount of the in-vehicle operation member 13. Therefore, the detection value detected by the operation amount detection sensor 151 indicates the operation amount by which the in-vehicle operation member 13 is operated.

[0040] The driving instruction values ​​based on the driving operation of the remote driver 21 are transmitted from the remote side system 30 via the in-vehicle communication device 180. During remote driving, the driving control ECU 150 performs driving control based on the driving instruction values ​​based on the driving operation of the remote driver 21. However, during remote driving, in other words, even when the in-vehicle driving permission unit 1422 does not permit in-vehicle driving, the driving control ECU 150 sequentially acquires detection values ​​detected by the operation amount detection sensor 151. This is to promptly perform driving control that reflects the driving operation of the in-vehicle driver 12 when switching to in-vehicle driving.

[0041] In addition, the driving control ECU 150 generates vehicle speed information indicating the current driving speed of the remote-driven vehicle 10 based on the detection signals of wheel speed sensors installed on the hub portions of each wheel, and outputs the generated vehicle speed information to the communication bus 190 sequentially.

[0042] The periphery monitoring sensor 160 is an autonomous sensor that monitors the environment surrounding the remotely driven vehicle 10. The periphery monitoring sensor 160 can detect moving objects and stationary objects within a detection range around the vehicle. The periphery monitoring sensor 160 can detect at least a vehicle ahead, a vehicle behind, a vehicle on either side, and the like that are traveling around the remotely driven vehicle 10. The periphery monitoring sensor 160 provides detection information of objects around the vehicle, i.e., the remotely driven vehicle 10, to the driving assistance ECU 141, the autonomous driving ECU 142, and the like.

[0043] The periphery monitoring sensor 160 includes, for example, one or more of a camera unit 161, a millimeter wave radar 162, a lidar 163, and a sonar 164. The camera unit 161 includes a front camera system and a surround camera system. The front camera system is mounted on the remotely driven vehicle 10 so as to be able to capture the range in front of the remotely driven vehicle 10. The surround camera system has a front camera, a rear camera, a left side camera, and a right side camera, and is able to capture the road surface all around the remotely driven vehicle 10. The millimeter wave radar 162 irradiates millimeter waves or quasi-millimeter waves toward the surroundings of the vehicle. The millimeter wave radar 162 outputs detection information generated by a process of receiving reflected waves reflected by moving objects, stationary objects, etc. The lidar 163 irradiates laser light toward the surroundings of the vehicle. The lidar 163 outputs detection information generated by a process of receiving laser light reflected by moving objects, stationary objects, etc. present in the irradiation range. The sonar 164 emits ultrasonic waves toward the surroundings of the vehicle. The sonar 164 receives ultrasonic waves reflected by moving objects and stationary objects in the vicinity of the vehicle and outputs detection information generated by the process.

[0044] Locator 170 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, etc. Locator 170 sequentially determines the current position and traveling direction, etc. of remotely operated vehicle 10 by combining the positioning signal received by the GNSS receiver, the measurement results of the inertial sensor, and vehicle speed information, etc. output to communication bus 190. Locator 170 sequentially outputs position information and direction information of remotely operated vehicle 10 based on the positioning results to communication bus 190 as locator information.

[0045] Locator 170 further includes a map database (hereinafter, map DB) 171 that stores map data. Map DB 171 is mainly composed of a large-capacity storage medium that stores a large amount of three-dimensional map data and two-dimensional map data. The three-dimensional map data is a so-called HD (High Definition) map, and includes road information necessary for automatic driving control. The three-dimensional map data includes information necessary for advanced driving assistance and automatic driving, such as three-dimensional shape information of roads and detailed information of each lane. Locator 170 reads map data around the current position from map DB 171, and provides it to driving assistance ECU 141, automatic driving ECU 142, etc. together with locator information.

[0046] The in-vehicle communication device 180 is an external communication unit mounted on the remote-operated vehicle 10, and functions as a V2X (Vehicle to Everything) communication device. The in-vehicle communication device 180 communicates with the remote communication device 33 directly or via a public communication line network. The in-vehicle communication device 180 may also transmit and receive information via wireless communication with a roadside device installed on the side of the road.

[0047] [Functions performed by the autonomous driving ECU 142] 4 shows functions executed by the autonomous driving ECU 142. The autonomous driving ECU 142 realizes the functions of a remote abnormality determination unit 1421, an in-vehicle driving permission unit 1422, an autonomous driving control execution unit 1423, and a driving information acquisition unit 1424 by executing a program by the processing unit 143.

[0048] The remote side abnormality judgment unit 1421 judges sequentially whether or not there is an abnormality in the remote side judgment target during remote driving. The remote side judgment target is at least one of the remote driver 21 and the remote side system 30. It is preferable that the remote side judgment target is both the remote driver 21 and the remote side system 30. An abnormality in the remote driver 21 means that the remote driver 21 is not in a state suitable for driving. Whether or not there is an abnormality in the remote driver 21 is determined based on whether or not there is a notification from the remote side system 30 that the remote driver 21 is not in a state suitable for driving.

[0049] When the remote side abnormality determination unit 1421 determines that the remote driver 21 is not abnormal, it also sequentially determines whether the remote driver 21 has a sign of abnormality. The sign of abnormality is determined based on the wakefulness level of the remote driver 21. The presence of a sign of abnormality means that the wakefulness level is close to abnormality, but not abnormal. For example, the condition for determining that the remote driver 21 is abnormal is that the time ratio during which the wakefulness level of the remote driver 21 is in the abnormal range within a certain time is equal to or greater than a first ratio. In this case, it is assumed that the remote driver 21 has a sign of abnormality when the time ratio during which the wakefulness level of the remote driver 21 is in the abnormal range within a certain time is lower than the first ratio and higher than the second ratio. When the remote driver 21 looks slightly sleepy, it is determined that the remote driver 21 has a sign of abnormality.

[0050] The remote side abnormality determination unit 1421 determines whether there is an abnormality in the remote side system 30 based on whether information is periodically transmitted from the remote side system 30. If information is not periodically transmitted from the remote side system 30 even during remote operation, it determines that there is an abnormality in the remote side system 30. In addition, when a signal notifying of a system abnormality is acquired from the remote side system 30, the automatic driving control execution unit 1423 may determine that an abnormality has occurred in the remote side system 30. One example of an abnormality in the remote side system 30 is an abnormality in the remote communication device 33.

[0051] The in-vehicle driving permission unit 1422 sequentially determines whether to permit the in-vehicle driver 12 to drive the remotely-driven vehicle 10. When remote driving is in progress and the remote-side abnormality determination unit 1421 has not determined that there is an abnormality in the remote-side determination target, the in-vehicle driver 12 is not permitted to drive the remotely-driven vehicle 10.

[0052] However, when the remote driver 21 is in the middle of remote driving and shows signs of abnormality, the in-vehicle driving permission unit 1422 compares the alertness levels of the remote driver 21 and the in-vehicle driver 12. Then, the driver with the higher alertness level is permitted to drive the remotely driven vehicle 10.

[0053] Furthermore, when the remotely driven vehicle 10 is traveling at autonomous driving level 3 and is being remotely driven, the in-vehicle driving permission unit 1422 sequentially determines whether the remote driver 21 is in a state where he or she can take over driving. Even if the remote driver 21 is not in a state where he or she can take over driving (hereinafter, a driving handover state), if the in-vehicle driver 12 is in a driving handover state, the in-vehicle driving permission unit 1422 instructs the autonomous driving control execution unit 1423 to continue autonomous driving level 3. However, the driver who is scheduled to take over driving is switched to the in-vehicle driver 12.

[0054] The in-vehicle driving permission unit 1422 outputs information indicating whether the remote driver 21 or the in-vehicle driver 12 is permitted to drive to the driving control ECU 150. If the in-vehicle driver 12 is not permitted to drive, the driving control ECU 150 does not use the value detected by the operation amount detection sensor 151 for driving control, and performs driving control of the remote-operated vehicle 10 using the driving instruction value transmitted from the remote-side system 30.

[0055] When in-vehicle driving is permitted, that is, when remote driving is switched to in-vehicle driving, the in-vehicle driving permission unit 1422 outputs a notification indicating this from the in-vehicle notification unit 130.

[0056] The autonomous driving control execution unit 1423 recognizes the driving environment necessary for performing autonomous driving by combining the detection information acquired from the periphery monitoring sensor 160, the locator information and map data acquired from the locator 170, the traffic congestion information acquired from the in-vehicle communication device 180, and the like. The autonomous driving control execution unit 1423 sequentially determines the autonomous driving level to be currently performed based on the recognized driving environment and the ODD. If the determined autonomous driving level is an autonomous driving level without the obligation to monitor the surroundings, the autonomous driving control execution unit 1423 sequentially outputs a control command to the driving control ECU 150 to execute vehicle control to automatically drive the remotely driven vehicle 10 based on the recognized driving environment. However, if the determined autonomous driving level is autonomous driving level 3, the autonomous driving control execution unit 1423 sequentially determines whether the driver is in a state where he or she can take over driving in response to a request from the system.

[0057] The driving information acquisition unit 1424 acquires driving information and transmits the acquired driving information from the in-vehicle communication device 180 to the remote side system 30. As described above, the driving information includes information indicating an image of the surroundings of the remotely driven vehicle 10, and the vehicle speed and steering angle of the remotely driven vehicle 10. The image of the surroundings of the remotely driven vehicle 10 is acquired from the surroundings monitoring sensor 160. The vehicle speed and steering angle of the remotely driven vehicle 10 are acquired from the cruise control ECU 150.

[0058] [Processing Executed by the Remote Control Device 36] Fig. 5 shows the process executed by the remote control device 36. During remote driving, the remote control device 36 repeatedly executes the process shown in Fig. 5. In S1, the driving information acquisition unit 361 acquires driving information transmitted from the vehicle-side system 100 from the remote communication device 33, and displays the acquired driving information on the display 351. As described above, the driving information includes information indicating an image of the surroundings of the remotely-driven vehicle 10, the vehicle speed of the remotely-driven vehicle 10, and the steering angle.

[0059] S2 to S6 are executed by the remote driver state notification unit 362. In S2, the monitoring result of the remote driver 21 is obtained from the remote driver monitoring device 32. The monitoring result includes whether the remote driver 21 is in a state suitable for driving and the alertness level of the remote driver 21.

[0060] In S3, based on the result obtained in S2, it is determined whether the remote driver 21 is in a state suitable for driving. If it is determined that the remote driver 21 is not in a state suitable for driving, that is, if the result of the determination in S3 is NO, the process proceeds to S4.

[0061] In S4, a warning is output from the remote driver notification device 35. After that, in S5, the monitoring result of the remote driver 21 is obtained from the remote driver monitoring device 32, and it is determined whether the remote driver 21 has returned to a state suitable for driving. If the determination result in S5 is also NO, the process proceeds to S6. In S6, the administrator notification device 34 outputs that the remote driver 21 is abnormal.

[0062] If the determination result of S3 or the determination result of S5 is YES, the process proceeds to S7. In S7, the driving instruction value acquisition unit 363 acquires driving instruction values ​​from the remote operation member 31. S8 is executed by the remote driver state notification unit 362 and the driving instruction value acquisition unit 363. In S8, remote information is transmitted from the remote communication device 33 to the vehicle-side system 100. The remote information includes whether the remote driver 21 is in a state suitable for driving, the alertness level of the remote driver 21, and the driving instruction values.

[0063] [Processing Executed by Vehicle-Side System 100] 6 and 7 show the processing executed by the vehicle-side system 100 when the remote-driven vehicle 10 is being remotely driven. In the processing shown in Fig. 6 and Fig. 7, step S17 is executed by the cruise control ECU 150. The other processing is executed by the automatic driving ECU 142.

[0064] In S11, the remote side abnormality determination unit 1421 determines whether or not remote information transmitted from the remote side system 30 has been acquired. If the determination result in S11 is NO, the process proceeds to S21 in Fig. 7 described later. If the determination result in S11 is YES, the process proceeds to S12.

[0065] In S12, the remote side abnormality determination unit 1421 determines whether the remote side system 30 is normal or not based on the remote information. If the determination result in S12 is NO, that is, if it is determined that the remote side system 30 is abnormal, the process proceeds to S21 in Fig. 7. Note that if it is determined that the remote information cannot be acquired (S11: NO), it is also determined that the remote side system 30 is abnormal. If the determination result in S12 is YES, the process proceeds to S13.

[0066] In S13, the remote side abnormality determination unit 1421 determines whether the remote driver 21 is in a state suitable for driving based on the remote information. If the determination result in S13 is NO, the process also proceeds to S21 in Fig. 7. If the determination result in S13 is YES, the process proceeds to S14.

[0067] In S14, the remote side abnormality determination unit 1421 determines whether or not the remote driver 21 has a sign of abnormality based on the wakefulness level of the remote driver 21 included in the remote information. If the determination result in S14 is YES, the process proceeds to S15.

[0068] In S15, the in-vehicle driving permission unit 1422 compares the alertness level of the remote driver 21 with the alertness level of the in-vehicle driver 12. Then, it is determined whether the remote driver 21 has a higher alertness level. If the determination result in S15 is NO, the process proceeds to S21 in FIG. 7. If the alertness levels of the two drivers can be compared and the remote driver 21 has a higher alertness level, the determination result in S15 is YES. Also, if the alertness level of the in-vehicle driver 12 cannot be obtained, for example, because the in-vehicle driver 12 is not in the driver's seat 11, the determination result in S15 is YES. If the determination result in S15 is YES, the process proceeds to S16. Also, if the determination result in S14 is NO, the process proceeds to S16.

[0069] In S16, the in-vehicle driving permission unit 1422 does not permit in-vehicle driving. In the following S17, the driving control ECU 150 acquires the operation amount of the in-vehicle operation member 13 from the operation amount detection sensor 151. In the following S18, the automatic driving control execution unit 1423 outputs the driving instruction value transmitted from the remote side system 30 to the driving control ECU 150. When S18 is executed, the driving control ECU 150 executes driving control based on the driving instruction value transmitted from the remote side system 30, and does not use the operation amount acquired in S17 for control.

[0070] In S19 , the driving information acquisition unit 1424 acquires driving information and transmits the acquired driving information from the in-vehicle communication device 180 to the remote side system 30 .

[0071] Next, Fig. 7 will be described. In the processing of Fig. 7, S33 is executed by the driving control ECU 150. The other processing is executed by the in-vehicle driving permission unit 1422. In S21, the remote-driven vehicle 10 determines whether autonomous driving level 3 is being executed. If autonomous driving level 3 is being executed, the determination result of S21 becomes YES, and the process proceeds to S22.

[0072] In S22, the state of the in-vehicle driver 12 is obtained from the in-vehicle driver monitoring device 120. Note that before executing S22, the in-vehicle driver 12 may be notified from the in-vehicle notification unit 130 that the driver will be switched to in-vehicle driving. In S23, the in-vehicle driver 12 determines whether or not the driver is in a state in which he or she can take over driving in response to a request from the automatic driving system 140. If the determination result in S23 is YES, the process proceeds to S24.

[0073] In S24, the driver who is notified of the driving handover is switched to the in-vehicle driver 12, and autonomous driving level 3 is continued. Then, in S25, the driving control ECU 150 is notified of in-vehicle driving permission. This allows the in-vehicle driver 12 to override while driving at autonomous driving level 3. Then, in S26, the in-vehicle notification unit 130 notifies that switching to in-vehicle driving has occurred. This allows the in-vehicle driver 12 to recognize that in-vehicle driving is now possible.

[0074] If the determination result in S21 is NO, the process proceeds to S27. When proceeding to S27, the remotely-driven vehicle 10 is being remotely driven at an autonomous driving level equal to or lower than autonomous driving level 2. In S27, as in S22, the state of the in-vehicle driver 12 is obtained from the in-vehicle driver monitoring device 120. Also, as in the case of executing S22, before executing S27, a notification to switch to in-vehicle driving may be given from the in-vehicle notification unit 130.

[0075] In S28, it is determined whether the in-vehicle driver 12 is in a state suitable for driving. If the determination result in S28 is NO, the process proceeds to S29. If the determination result in S23 is NO, the process also proceeds to S29. In S29, the driving control ECU 150 is instructed to make an emergency stop of the remotely driven vehicle 10. Thereafter, in S30, the driving control ECU 150 is notified of in-vehicle driving permission. After stopping the remotely driven vehicle 10, the driving control ECU 150 automatically transitions to an in-vehicle driving mode in which the remotely driven vehicle 10 drives when the in-vehicle operating member 13 is operated.

[0076] In S31, the in-vehicle notification unit 130 notifies the driver that the driving has been switched to in-vehicle driving. The notification content in S31 differs from that in S26. In S31, the autonomous driving level is 2 or lower. Therefore, the notification in S31 is a notification that makes the driver aware of a higher level of urgency than the notification in S26. For example, the notification in S31 may be louder than the notification in S26 or may use a more eye-catching image than the notification in S26.

[0077] If the result of the determination in S28 is YES, the process proceeds to S32. In S32, the cruise control ECU 150 is notified of in-vehicle driving permission. In S33, cruise control is executed using the latest operation amount stored in S17. After that, the process proceeds to the above-mentioned S31, and the in-vehicle notification unit 130 notifies the driver that the mode has been switched to in-vehicle driving.

[0078] [Summary of the first embodiment] In the first embodiment described above, if there is an abnormality in the remote-side determination object (S11 to S13: NO), in-vehicle driving is permitted (S25, S30, S32). Therefore, even if there is an abnormality in the remote-side determination object, the remote-operated vehicle 10 can continue traveling by in-vehicle driving.

[0079] On the other hand, when the remote side abnormality determination unit 1421 determines that neither the remote driver 21 nor the remote side system 30 is abnormal, the in-vehicle driving permission unit 1422 does not permit in-vehicle driving (S16). This makes it possible to prevent the in-vehicle driver 12 from performing driving operations unintended by the remote driver 21 during remote driving.

[0080] The driving control ECU 150 sequentially acquires and stores signals indicating the amount of operation of the in-vehicle operation member 13 even when the in-vehicle driving permission unit 1422 does not permit in-vehicle driving (S17). Then, when the in-vehicle driving permission unit 1422 permits in-vehicle driving, driving control is performed based on the stored amount of operation. As a result, when the in-vehicle driver 12 senses an abnormality in the remote-side judgment object and operates the in-vehicle operation member 13 before in-vehicle driving is permitted, the operation is reflected and the remotely-operated vehicle 10 is driven. Therefore, the in-vehicle driver 12 can drive the remotely-operated vehicle 10 by quickly responding to the abnormality in the remote-side judgment object.

[0081] When the remote driver 21 shows signs of abnormality (S14: YES) and the alertness level of the in-vehicle driver 12 has been determined, the in-vehicle driving permission unit 1422 permits driving operation to the one with the higher alertness level between the remote driver 21 and the in-vehicle driver 12 (S15). This makes it possible to prevent a driver with a low alertness level from driving the remotely driven vehicle 10.

[0082] If the remote side abnormality determination unit 1421 determines that there is an abnormality in the remote side determination target during remote driving at autonomous driving level 3 (S11 to S13: NO), the in-vehicle driving permission unit 1422 permits in-vehicle driving (S25). In addition, if the in-vehicle driver 12 is in a state where he or she can take over driving (S23: YES), the driver who notifies the driver of the driving takeover is switched to the in-vehicle driver 12, and driving at autonomous driving level 3 is continued (S24). As a result, autonomous driving level 3 may be able to be continued even if there is an abnormality in the remote side determination target.

[0083] When the remote-side abnormality determination unit 1421 determines that there is an abnormality in the remote-side determination target (S11 to S13: NO) and the autonomous driving level is 2 or lower (S21: NO), the in-vehicle driving permission unit 1422 first stops the remotely driven vehicle 10 (S29). After that, in-vehicle driving is permitted, and when the in-vehicle operation member 13 is operated, the remotely driven vehicle 10 automatically transitions to an in-vehicle driving mode in which the remotely driven vehicle 10 runs (S30). Therefore, after the remotely driven vehicle 10 has stopped, the in-vehicle driver 12 can drive the remotely driven vehicle 10 by operating the in-vehicle operation member 13 without needing to perform a driving mode switching operation.

[0084] When the remote-side abnormality determination unit 1421 determines that there is an abnormality in the remote-side determination target (S11 to S13: NO), the in-vehicle driving permission unit 1422 notifies the in-vehicle notification unit 130 that the driving has been switched to in-vehicle driving (S26, S31). This allows the occupant of the remote-operated vehicle 10 to recognize that the driving has been switched to in-vehicle driving.

[0085] In addition, the in-vehicle driving permission unit 1422 makes different notification contents when it is determined that there is an abnormality in the remote-side determination target depending on whether autonomous driving level 3 is being executed or whether autonomous driving level 2 or lower is being executed (S26, S31). Specifically, when autonomous driving level 2 or lower is being executed, a notification is issued to make the driver aware of a higher degree of urgency (S31). This increases the likelihood that the passengers of the remote-driven vehicle 10 will be able to quickly respond to driving control for autonomous driving level 2 or lower when autonomous driving level 2 or lower is being executed.

[0086] Based on the fact that the remote driver 21 indicates that he / she is not in a state suitable for remote driving, the administrator notification device 34 notifies the remote driver 21 that he / she is not in a state suitable for driving. This enables the remote driving facility 20 to quickly take measures such as replacing the remote driver 21.

[0087] Second embodiment In the remote vehicle driving system 5 of the second embodiment shown in Fig. 8, the operation of a plurality (a large number) of remotely-operated vehicles 10 is managed by one remote side system 30. The remote vehicle driving system 5 includes a plurality (a large number) of infrastructure side systems 60 in addition to the remote side system 30 and the plurality of remotely-operated vehicles 10.

[0088] In the remote driving facility 20 in which the remote side system 30 is installed, one or more remote managers 25 and two or more remote drivers 21 are engaged in the operation of multiple remotely driven vehicles 10. The remote manager 25 manages the operation of multiple remotely driven vehicles 10 under the management of the remote side system 30. The remote driver 21 monitors each remotely driven vehicle 10 and remotely drives the remotely driven vehicle 10 by taking over a dynamic driving task from the automatic driving system 140 when necessary. One of the remote drivers 21 is a main remote driver 22. When an abnormality occurs in the remotely driven vehicle 10, the main remote driver 22 prioritizes remote driving of the remotely driven vehicle 10. The other remote driver 21 is a sub-remote driver 23. When an abnormality occurs in the main remote driver 22 or when an abnormality occurs in multiple remotely driven vehicles 10, the sub-remote driver 23 remotely drives the remotely driven vehicle 10 in place of the remote driver 21.

[0089] The remote-operated vehicle 10 can perform autonomous driving equivalent to autonomous driving level 3 while being remotely monitored by the remote driver 21, as in the first embodiment. The remote-operated vehicle 10 is an unmanned autonomous vehicle that can run without a driver on board. When the remote-operated vehicle 10 is a vehicle for mobility services, all of the occupants 14 of the remote-operated vehicle 10 are passengers. In the remote-operated vehicle 10, the driving control state is switched among a plurality of modes including at least an autonomous driving mode in which the vehicle runs by an autonomous driving function while being remotely monitored, a remote control mode in which the remote driver 21 remotely controls the vehicle, and an in-vehicle driving mode in which the in-vehicle driver 12 controls the vehicle. The current driving control state of the remote-operated vehicle 10 is displayed on an external display unit 135 mounted on the remote-operated vehicle 10, and is notified to pedestrians, cyclists, drivers, and the like around the remote-operated vehicle 10.

[0090] The infrastructure-side system 60 includes a surveillance camera 61. The infrastructure-side system 60 is installed on the travel route of the remote-controlled vehicle 10. A large number of infrastructure-side systems 60 may be installed so as to capture the entire travel route of the remote-controlled vehicle 10 without omission, or may be installed only at specific points such as bus stops and intersections on the travel route.

[0091] The infrastructure-side system 60 can communicate with the remote-side system 30 via a public communication line network. The infrastructure-side system 60 transmits surveillance video of the driving route captured by the surveillance camera 61 to the remote-side system 30 as infrastructure information. The infrastructure-side system 60 may include an infrastructure sensor different from the surveillance camera 61. In this embodiment, detection information of the infrastructure sensor is transmitted from the infrastructure-side system 60 to the remote-side system 30 as infrastructure information.

[0092] [Configuration of remote system 30] 8 and 9, the remote side system 30 of the second embodiment includes a remote operation member 31, a remote driver monitoring device 32, a remote communication device 33, an administrator notification device 34, a remote driver notification device 35, and a remote control device 36, as in the first embodiment. Two remote operation members 31, two remote driver monitoring devices 32, and two remote driver notification devices 35 are provided for each of the main remote pilot 22 and the sub-remote pilot 23.

[0093] The remote driver monitoring device 32 or the remote driver notification device 35 performs an identification process to determine whether the remote driver 21 is a legitimate person registered in advance. The identification process may be a face recognition process using the remote driver monitoring device 32, or an authentication process of an ID and a password input to the remote driver notification device 35. Only a legitimate person who has been authenticated by such an identification process can perform remote monitoring and remote driving as the remote driver 21. Similarly, the administrator notification device 34 may perform an identification process to determine whether the remote administrator 25 is a legitimate person.

[0094] The remote control device 36 operates as a remote monitoring unit 366, a presentation control unit 367, and a remote assistance operation unit 368 by executing programs stored in the non-volatile memory.

[0095] The remote monitoring unit 366 includes the functions of the driving information acquisition unit 361 (see FIG. 2) of the first embodiment. The remote monitoring unit 366 sequentially acquires driving information and infrastructure information received by the remote communication device 33. The driving information is information necessary for remote monitoring and remote driving of the remotely driven vehicles 10, and is transmitted by each remotely driven vehicle 10. The infrastructure information is real-time monitoring video captured of the driving route of the remotely driven vehicle 10, and is transmitted by each infrastructure-side system 60.

[0096] The presentation control unit 367 includes the function of the remote driver status notification unit 362 (see FIG. 2) of the first embodiment. The presentation control unit 367 controls the presentation of information to the remote manager 25 by the manager notification device 34 and the presentation of information to the main remote driver 22 and the sub remote driver 23 by the two remote driver notification devices 35. Specifically, the presentation control unit 367 causes the driving information and infrastructure information acquired by the remote monitoring unit 366 to be displayed on each of the displays 341, 351. As a result, at least the image captured by the camera unit 161 (see FIG. 3) and the image captured by the monitoring camera 61 are displayed on each of the displays 341, 351.

[0097] The presentation control unit 367 grasps through the remote assistance operation unit 368 that an abnormality in the remote driver 21 has been detected. When the presentation control unit 367 detects an abnormality in the remote driver 21, it controls the remote driver notification device 35 to issue a warning. When the presentation control unit 367 detects an abnormality in the main remote driver 22, it issues a warning not only from the remote driver notification device 35 of the main remote driver 22, but also from the remote driver notification device 35 and the manager notification device 34 of the sub remote driver 23. Similarly, when an abnormality in the sub remote driver 23 is detected, the presentation control unit 367 issues a warning not only from the remote driver notification device 35 of the sub remote driver 23, but also from the remote driver notification device 35 and the manager notification device 34 of the main remote driver 22.

[0098] The remote assistance operation unit 368 includes the function of the driving instruction value acquisition unit 363 (see FIG. 2) of the first embodiment. When an abnormality occurs in a specific remotely operated vehicle 10 among the multiple remotely operated vehicles 10 under the management of the remote side system 30, the remote assistance operation unit 368 grasps the occurrence of the abnormality in the remotely operated vehicle 10 through the remote communication device 33 and the remote monitoring unit 366. Based on the abnormality of the remotely operated vehicle 10, the remote assistance operation unit 368 starts remote driving in cooperation with the vehicle side system 100. In this case, the remote assistance operation unit 368 determines one of the main remote driver 22 and the sub remote driver 23 to be the remote driver 21, and acquires driving instruction values ​​from the remote operation member 31 linked to the remote driver 21. The remote assistance operation unit 368 transmits the acquired driving instruction values ​​to the specific remotely operated vehicle 10 in which the abnormality occurs through the remote communication device 33.

[0099] Furthermore, the remote assistance operation unit 368 acquires the monitoring results of the remote driver 21 from the remote driver monitoring device 32. As described above, when an abnormality occurs in the remote driver 21, the remote assistance operation unit 368 cooperates with the presentation control unit 367 to issue a warning to the remote driver 21 using the remote driver notification device 35. When the abnormal condition of the remote driver 21 does not improve even after the warning, the remote assistance operation unit 368 makes a definitive determination of the abnormality of the remote driver 21.

[0100] Based on the determination, the remote assistance control unit 368 notifies each remotely driven vehicle 10 through the remote communication device 33 that an abnormality has occurred in the remote driver 21. As an example, when an abnormality in the main remote driver 22 is determined, the remote assistance control unit 368 transmits information indicating that the authority for remote driving has been transferred to the sub-remote driver 23 (hereinafter, "switch information") to each remotely driven vehicle 10. Furthermore, when an abnormality in the main remote driver 22 or the sub-remote driver 23 is determined, the remote assistance control unit 368 transmits information indicating the presence or absence of a spare remote driver 21 (hereinafter, "switch personnel") who can take over remote driving to each remotely driven vehicle 10.

[0101] [Configuration of vehicle-side system 100] 8 and 10, the vehicle-side system 100 of the second embodiment includes an in-vehicle notification unit 130, a cruise control ECU 150, a surroundings monitoring sensor 160, a locator 170, an in-vehicle communication device 180, and an automatic driving ECU 142, as in the first embodiment. In addition, the vehicle-side system 100 further includes an occupant information acquisition device 220 and an exterior display unit 135.

[0102] The occupant information acquisition device 220 includes the functions of the in-vehicle driver monitoring device 120 (see FIG. 3) of the first embodiment. The occupant information acquisition device 220 monitors the interior of the vehicle cabin (passenger compartment) of the remote-operated vehicle 10 and detects the occurrence of a person who suddenly falls ill or is injured. In addition, while the in-vehicle driver 12 is driving, the occupant information acquisition device 220 continuously monitors whether the in-vehicle driver 12 is in a state suitable for driving.

[0103] Furthermore, when driving in the vehicle is permitted, the occupant information acquisition device 220 acquires occupant information on a specific occupant 14 (hereinafter, specific occupant 14s) who sits in the driver's seat 11 and attempts to operate the in-vehicle operation member 13. As an example, the occupant information acquisition device 220 captures an image of the face of the specific occupant 14s and a driver's license presented by the specific occupant 14s, and acquires a facial image and a license image. The facial image and the license image are used as occupant information of the specific occupant 14s and are transmitted to the remote side system 30 via the autonomous driving ECU 142 and the in-vehicle communication device 180.

[0104] As described above, the vehicle exterior display unit 135 displays information toward the outside of the remotely driven vehicle 10. The vehicle exterior display unit 135 may be a display capable of displaying characters, or may be configured to display information by changing the light emission mode. The remotely driven vehicle 10 is provided with at least one of a front display unit 136, a side display unit 137, and a rear display unit 138 as the vehicle exterior display unit 135. The front display unit 136 is installed on the front of the remotely driven vehicle 10 with the display surface facing forward of the remotely driven vehicle 10. The side display units 137 are installed on both the left and right sides of the remotely driven vehicle 10. The rear display unit 138 is installed on the rear of the remotely driven vehicle 10 with the display surface facing rearward of the remotely driven vehicle 10.

[0105] The autonomous driving ECU 142 has the function of a remotely monitored and operated autonomous driving device. The autonomous driving ECU 142 realizes the functions of a state determination unit 1426, a mode switching unit 1427, a vehicle monitoring unit 1428, and a driving execution unit 1429 by executing a program by a processing unit 143 (see FIG. 3).

[0106] The state determination unit 1426 includes the function of the remote side abnormality determination unit 1421 (see FIG. 4) of the first embodiment. The state determination unit 1426 continuously monitors the remote side determination target based on information from the remote side system 30 received by the in-vehicle communication device 180. When the state determination unit 1426 determines that an abnormality has occurred in the remote side determination target, it shares information on the abnormality occurrence with the mode switching unit 1427. The abnormality of the remote side determination target includes an abnormality or a sign of an abnormality in either of the two remote drivers 21, an abnormality in communication with the remote side system 30, and the like. Furthermore, the abnormality of communication includes not only a deterioration of the communication state but also a malicious third party's takeover of communication (session hijacking), and the like. When the state determination unit 1426 determines that the abnormality has been resolved, or that the abnormality was a false detection, it shares information on the abnormality resolution with the mode switching unit 1427.

[0107] The mode switching unit 1427 includes the function of the in-vehicle driving permission unit 1422 (see FIG. 4) of the first embodiment. The mode switching unit 1427 cooperates with the driving control ECU 150 to switch the driving control state of the vehicle-side system 100 among a plurality of modes including the above-mentioned automatic driving mode, remote control mode, and in-vehicle driving mode.

[0108] The mode switching unit 1427 switches the driving control state from the autonomous driving mode to the remote control mode in a scene where the driving execution unit 1429 cannot continue autonomous driving at autonomous driving level 3 or higher. The mode switching unit 1427 decides to suspend the autonomous driving mode based on the suspension proposal presented by the vehicle monitoring unit 1428, and switches to the remote control mode. In this case, the mode switching unit 1427 transmits a request for operation to the remote side system 30 via the in-vehicle communication device 180, that is, a request to perform remote driving.

[0109] When the mode switching unit 1427 receives information indicating an abnormality from the state determination unit 1426 and detects an abnormality in the remote-side determination target, the mode switching unit 1427 switches the driving control state from the remote operation mode to the in-vehicle driving mode, as in the first embodiment. Note that, also in the second embodiment, when there is no abnormality in the remote-side determination target, the mode switching unit 1427 does not switch to the in-vehicle driving mode.

[0110] According to the above mode switching, when the driving control state is the autonomous driving mode, the driving control ECU 150 drives the remotely-operated vehicle 10 based on a control command input from the driving execution unit 1429. When the driving control state is the remote driving mode, the driving control ECU 150 drives the remotely-operated vehicle 10 based on a driving instruction value received from the remote-side system 30 via the in-vehicle communication device 180. When the driving control state is the in-vehicle driving mode, the driving control ECU 150 drives the remotely-operated vehicle 10 based on a driving instruction value input from the operation amount detection sensor 151.

[0111] Here, when the mode switching unit 1427 switches the driving control state, the autonomous driving ECU 142 and the driving control ECU 150 always stop the remotely driven vehicle 10. The mode switching unit 1427 performs start and end processing of the remote operation mode or the in-vehicle driving mode while the remotely driven vehicle 10 is stopped. As a result, handover of driving between the autonomous driving ECU 142 and the remote driver 21, and handover of driving between the remote driver 21 and the in-vehicle driver 12 are performed while the remotely driven vehicle 10 is stopped.

[0112] Furthermore, the mode switching unit 1427 controls information notification by the in-vehicle notification unit 130 and the outside-vehicle display unit 135. When switching the driving control state, the mode switching unit 1427 notifies the occupant 14 of the switching using the in-vehicle notification unit 130 before the remote-operated vehicle 10 stops. Furthermore, the mode switching unit 1427 causes the outside-vehicle display unit 135 to display the current driving control state.

[0113] The vehicle monitoring unit 1428 includes the functions of the driving information acquisition unit 1424 (see FIG. 4) of the first embodiment. The vehicle monitoring unit 1428 acquires various information indicating the external environment, internal environment, and vehicle state of the remotely-operated vehicle 10, and monitors the acquired information. Specifically, the vehicle monitoring unit 1428 acquires images of the surroundings of the remotely-operated vehicle 10 from the camera unit 161 (see FIG. 3), and sequentially transmits the images together with information indicating the driving state of the remotely-operated vehicle 10 to the remote-side system 30 via the in-vehicle communication device 180.

[0114] The vehicle monitoring unit 1428 proposes to suspend the autonomous driving mode to the mode switching unit 1427 based on information detected by the periphery monitoring sensor 160, etc. The vehicle monitoring unit 1428 proposes to suspend the autonomous driving mode when an obstacle that is difficult to recognize or avoid appears on the planned driving route of the remote-driven vehicle 10, or when the detection capability of the periphery monitoring sensor 160 is significantly reduced due to worsening weather, etc.

[0115] The vehicle monitoring unit 1428 grasps the state of the occupant 14 based on information acquired from the occupant information acquisition device 220. When the occupant information acquisition device 220 detects an abnormality of the occupant 14, the vehicle monitoring unit 1428 requests the driving execution unit 1429 to temporarily stop the remote-operated vehicle 10, and notifies the remote side system 30 via the in-vehicle communication device 180 of the occurrence of an abnormality in the passenger compartment.

[0116] The driving execution unit 1429 includes the functions of the automatic driving control execution unit 1423 (see FIG. 4) of the first embodiment. The driving execution unit 1429 recognizes the driving environment using detection information acquired from the surrounding monitoring sensor 160, and locator information and map data acquired from the locator 170, and executes automatic driving in the automatic driving mode. The driving execution unit 1429 generates a control command for executing vehicle control for automatically driving the remotely driven vehicle 10 based on the recognized driving environment, and sequentially outputs the generated control command to the driving control ECU 150. The driving execution unit 1429 may be capable of executing autonomous driving control of automatic driving level 4 or higher, or may be capable of executing driving assistance control of automatic driving level lower than 3.

[0117] Next, the details of each process executed by the automatic driving ECU 142 will be described based on FIGS. 11 to 13, and with reference to FIGS.

[0118] [Mode switching process when an abnormality occurs in the remote side judgment target] Based on the start of the remote operation mode, the autonomous driving ECU 142 executes the process shown in FIG. 11 (hereinafter, remote abnormality response process) to prepare for the occurrence of an abnormality in the remote-side judgment target. In S201 of the remote abnormality response process, the state determination unit 1426 determines whether or not an abnormality in the remote-side judgment target has been detected. In S201, the state determination unit 1426 allows erroneous detection of an abnormality and detects not only definite abnormalities but also abnormalities with low certainty. If the determination result in S201 is NO, the state determination unit 1426 continues the abnormality detection in S201. On the other hand, if the determination result in S201 is YES, the process proceeds to S202. In S202, when it is determined that there is an abnormality in the remote-side judgment target, the driving control ECU 150 stops the remote-operated vehicle 10 based on a command from the mode switching unit 1427 or the driving execution unit 1429.

[0119] In S203, the state determination unit 1426 determines whether the abnormality detected in S201 has been resolved. If the vehicle-side system 100 has resolved the abnormality, or if any of the occupant 14, the remote driver 21, and the remote manager 25 has performed a resolution process to resolve the abnormality, the state determination unit 1426 determines that the abnormality has been resolved. If the determination result in S203 is thus YES, the process proceeds to S204. Also, if it is confirmed that the abnormality detected in S201 was a false detection, the process proceeds to S204.

[0120] In S204, the mode switching unit 1427 decides to continue the remote operation mode and permits resumption of remote driving. As a result, the driving control ECU 150 resumes driving by remote control based on the driving instruction values ​​for remote driving. In this case, the control flow returns to S201, and the state determination unit 1426 continues to detect an abnormality. As a result, even if the remote-driven vehicle 10 has stopped due to a false detection, it can resume driving based on confirmation that this was a false detection.

[0121] If the determination result in S203 is NO, the state determination unit 1426 determines whether a timeout has occurred in S205. The state determination unit 1426 waits for the abnormality to be resolved until the timeout occurs. If the timeout occurs without the abnormality being resolved and the determination result in S205 becomes YES, the mode switching unit 1427 determines to end the remote control mode in S206. The mode switching unit 1427 switches the driving control state from the remote control mode to the in-vehicle driving mode in S206, thereby permitting in-vehicle driving.

[0122] In S207 when driving in the vehicle is permitted, the vehicle monitoring unit 1428 cooperates with the occupant information acquisition device 220 to acquire occupant information on the specific occupant 14s seated in the driver's seat 11. The vehicle monitoring unit 1428 provides the acquired occupant information to the remote side system 30 by transmission. The occupant information is displayed on the display 341, 351 by the remote side system 30 and presented to the remote driver 21 or the remote manager 25. Based on the displayed occupant information, the remote driver 21 or the remote manager 25 judges whether or not to permit the specific occupant 14s to drive in the vehicle, and inputs the judgment result to the remote side system 30. As an example, the remote driver 21 or the remote manager 25 permits driving in the vehicle when it can be determined that the specific occupant 14s is a driver's license holder based on the face photo on the driver's license and the face image showing the face of the specific occupant 14s. The judgment result in the remote side system 30 is transmitted to the automatic driving ECU 142.

[0123] In S208, the mode switching unit 1427 further determines whether or not to permit the specific occupant 14s to drive in the vehicle using the occupant information. In S208, the mode switching unit 1427 determines whether or not to permit the specific occupant 14s to drive in the vehicle according to the determination result based on the occupant information and acquired from the remote side system 30. However, when communication with the remote side system 30 is interrupted, the mode switching unit 1427 may itself perform the determination of whether or not to permit the specific occupant 14s to drive in the vehicle using the occupant information. In this case, the mode switching unit 1427 permits the specific occupant 14s to drive in the vehicle if it can be verified that the facial photograph on the driver's license is a photograph of the specific occupant 14s. If the determination result in S208 is NO, the remote abnormality response process is terminated. In this case, the remotely driven vehicle 10 is in a standby state while remaining stopped. On the other hand, if the determination result in S208 is YES, the process proceeds to S209.

[0124] In S209, the vehicle monitoring unit 1428 registers the specific occupant 14s as an in-vehicle driver 12. In this case, occupant information of the specific occupant 14s is stored in at least one of the remote control device 36 and the automatic driving ECU 142.

[0125] In S210, the mode switching unit 1427 and the cruise control ECU 150 set a driving restriction. In particular, when the mode switching unit 1427 permits in-vehicle driving, the driving speed of the remotely-driven vehicle 10 is more restricted than when the vehicle is driven by remote driving. As an example, when the remotely-driven vehicle 10 is a vehicle that drives on a general road, the driving speed is restricted to about half the speed limit of the road (about 30 km / h). Also, when the remotely-driven vehicle 10 is a low-speed driving vehicle, the driving speed is restricted to 5 to 10 km / h. Then, in S211, the mode switching unit 1427 starts in-vehicle driving. As a result, the cruise control ECU 150 resumes driving of the remotely-driven vehicle 10 based on the driving instruction value for in-vehicle driving. With the above, the remote abnormality response process is completed.

[0126] [External display process of driving control status] 12 (hereinafter, referred to as "exterior display processing") based on the start of the vehicle-side system 100. The mode switching unit 1427 continues the exterior display processing until the power supply of the vehicle-side system 100 is turned off. By the exterior display processing, driving status information indicating whether the remotely-driven vehicle 10 is being driven by automatic driving, remote driving, or in-vehicle driving is continuously displayed on the exterior display unit 135.

[0127] Specifically, in S231 of the vehicle exterior display process, the driving control state is grasped. Then, in S232, the driving mode in operation is determined based on the driving control state grasped in S232. When the driving control state is the automatic driving mode, the mode switching unit 1427 causes the vehicle exterior display unit 135 to display, as an automatic driving display, characters or images indicating that the remotely driven vehicle 10 is in automatic driving, in S233. On the other hand, when the driving control state is the remote operation mode, the mode switching unit 1427 causes the vehicle exterior display unit 135 to display, as a remote driving display, characters or images indicating that the remotely driven vehicle 10 is in remote driving, in S234. Furthermore, when the driving control state is the in-vehicle driving mode, the mode switching unit 1427 causes the vehicle exterior display unit 135 to display, as an in-vehicle driving display, characters or images indicating that the remotely driven vehicle 10 is in in-vehicle driving, in S235.

[0128] The autonomous driving mode is a driving control state in which the remotely driven vehicle 10 is driving normally. Therefore, the mode switching unit 1427 does not need to display autonomous driving. That is, the mode switching unit 1427 may use the exterior display unit 135 to display only the remote driving status and the in-vehicle driving status.

[0129] [Vehicle-side processing in the event of a remote driver's abnormality] The mode switching unit 1427 executes the process shown in FIG. 13 (hereinafter, vehicle-side process) based on the start of the remote driving mode. In the vehicle-side process, the automatic driving ECU 142 remotely monitors the state of the remote driver 21. In S251 of the vehicle-side process, the mode switching unit 1427 cooperates with the state determination unit 1426 to grasp an abnormality of the remote driver 21 at the remote driving facility 20. Specifically, in S251, the mode switching unit 1427 determines whether there is change information indicating a change of the remote driver 21 at the remote driving facility 20. If there is no change information, the determination in S251 is continued. On the other hand, when change information is acquired from the remote side system 30, the mode switching unit 1427 performs a notification based on the change information using the in-vehicle notification unit 130 in S252.

[0130] Furthermore, in S253, the mode switching unit 1427 cooperates with the state determination unit 1426 to determine whether there is a replacement for remote driving. As an example, when there is an abnormality in the main remote driver 22 and the remote driver 21 is only the sub-remote driver 23, in other words, when there are no multiple remote drivers 21 in the remote driving facility 20, the mode switching unit 1427 determines that there is no replacement (S253: NO). On the other hand, when there is an abnormality in the main remote driver 22 but there is a remote driver 21 other than the sub-remote driver 23 in the remote driving facility 20, the mode switching unit 1427 determines that there is a replacement (S253: YES).

[0131] If the determination result in S253 is YES, the process returns to S251 and vehicle-side processing continues. On the other hand, if the determination result in S253 is NO, the mode switching unit 1427 cooperates with the driving control ECU 150 in S254 to restrict some of the driving functions of the remotely-driven vehicle 10 during remote driving. For example, the mode switching unit 1427 sets the upper limit of the driving speed of the remotely-driven vehicle 10 lower than when a replacement person is present. Furthermore, the mode switching unit 1427 may apply restrictions such as prohibiting lane changing to an overtaking lane, or prohibiting driving itself while only allowing the operation of opening the door of the remotely-driven vehicle 10.

[0132] The mode switching unit 1427 may perform vehicle-side processing even during driving in the autonomous driving mode. In this case, the mode switching unit 1427 limits the autonomous driving driving function by the driving execution unit 1429 in consideration of the deterioration of the monitoring ability of the remote driver 21.

[0133] Next, the details of each process executed by the remote system 30 will be described based on FIGS. 14 and 15, with reference to FIGS.

[0134] [Remote response processing when multiple abnormalities occur simultaneously] When the remote control device 36 receives a notification indicating the occurrence of an abnormality from one of the multiple remote-operated vehicles 10 under the management of the remote-side system 30, the remote control device 36 starts the process shown in Fig. 14 (hereinafter, vehicle abnormality response process). The remote control device 36 prepares for the simultaneous occurrence of abnormalities in the remote-operated vehicles 10 by executing the vehicle abnormality response process.

[0135] In S271 of the vehicle abnormality response processing, the remote monitoring unit 366 grasps the contents of the abnormality that has occurred in the remote-operated vehicle 10. In S271, the remote monitoring unit 366 further grasps whether or not a request to perform remote driving has been acquired from the remote-operated vehicle 10. In S272, the presentation control unit 367 notifies the remote driver 21 and the remote manager 25 of the contents of the abnormality grasped in S271.

[0136] In S273, the remote monitoring unit 366 determines whether or not remote driving execution requests have been received from multiple remotely driven vehicles 10. If the determination result in S273 is NO, the process proceeds to S276. On the other hand, if the determination result in S273 is YES, the remote monitoring unit 366 determines the priority of remote driving of each remotely driven vehicle 10 in S274 based on the abnormality state occurring in each remotely driven vehicle 10.

[0137] The remote monitoring unit 366 sets a remotely-operated vehicle 10 that is encountering a scene related to a traffic accident as the remotely-operated vehicle 10 with the highest remote driving priority. The remote monitoring unit 366 also sets a remotely-operated vehicle 10 that is encountering a scene that would block a road and prevent other vehicles from passing if it were to stop as the remotely-operated vehicle 10 with the second highest remote driving priority. The remote monitoring unit 366 also sets a remotely-operated vehicle 10 that is blocking the road by being stopped as the remotely-operated vehicle 10 with the third highest remote driving priority. Note that when remotely-operated vehicles 10 with the same priority occur, the priority of the remotely-operated vehicle 10 that encountered the abnormality first may be set higher.

[0138] In S275, the presentation control unit 367 notifies the remote driver 21 of the remotely driven vehicle 10 with high priority for remote driving, together with the details of the abnormality occurring in this remotely driven vehicle 10. The remote driver 21 refers to the notification of priority and determines the remotely driven vehicle 10 to be remotely driven, i.e., the remotely driven vehicle 10 to which the remote assistance operation unit 368 will provide driving instruction values ​​for remote driving.

[0139] In S276, the remote monitoring unit 366 determines whether the abnormality has been resolved. If one or more remote-operated vehicles 10 are still in an abnormal state (276: NO), the process returns to S271, and notification of the abnormality and priority is continued. On the other hand, if all abnormalities have been resolved (276: YES), the abnormality notification is terminated in S277. In S277, notification of the resolution of the abnormality may be given.

[0140] [Handling abnormalities while driving in a convoy] The remote control device 36 starts the process shown in Fig. 15 (hereinafter, referred to as the convoy management process) when an abnormality occurs in one of the multiple remotely driven vehicles 10 that are traveling in a convoy. Platooning is a state in which multiple (two or more) remotely driven vehicles 10 travel in a line while automatically maintaining an inter-vehicle distance.

[0141] In S291 of the convoy management process, the remote monitoring unit 366 determines the nature of the abnormality occurring in the remotely driven vehicle 10 (hereinafter, the abnormality vehicle) and the position of the abnormality vehicle in the convoy. Furthermore, in S292, the remote monitoring unit 366 determines whether the abnormality vehicle can continue traveling. If the determination result in S292 is NO, the convoy management process ends. In this case, the remote driving facility 20 (remote driver 21 or remote manager 25) takes the initiative to temporarily stop all remotely driven vehicles 10 traveling in the convoy and move the occupants 14 aboard the abnormality vehicle to other normal remotely driven vehicles 10.

[0142] On the other hand, if the determination result in S292 is YES, the remote monitoring unit 366 determines in S293 whether the abnormality occurring vehicle is a front remotely-operated vehicle. A front remotely-operated vehicle 10 is a remotely-operated vehicle 10 that travels ahead of the other remotely-operated vehicles 10. If the determination result in S293 is YES, the remote assistance operation unit 368 provides a platoon exchange instruction to each remotely-operated vehicle 10 in the platoon in S294. The platoon exchange instruction is an instruction to exchange the front and rear positions of the front remotely-operated vehicle, which is the abnormality occurring vehicle, and the rear remotely-operated vehicle, which is a normal remotely-operated vehicle 10. According to the platoon exchange instruction, the normal remotely-operated vehicle 10 overtakes the stopped abnormality occurring vehicle and moves ahead of the abnormality occurring vehicle. As a result, the platoon continues to travel in a form in which the abnormality occurring vehicle follows the normal remotely-operated vehicle 10.

[0143] On the other hand, if the determination result in S293 is NO, in S295, the remote assistance operation unit 368 provides instructions to each remote-controlled vehicle 10 to maintain the formation. As a result of the above, the platooning continues with the abnormal vehicle following the normal remote-controlled vehicle 10.

[0144] When three or more remotely operated vehicles 10 are traveling in a convoy, the remote monitoring unit 366 determines whether the abnormal vehicle is the leading vehicle or the last vehicle in the convoy. If the abnormal vehicle is the leading vehicle or the last vehicle, the remote assistance operation unit 368 provides each remotely operated vehicle 10 with an instruction to move the abnormal vehicle to the middle of the convoy. As a result, the convoy continues traveling with the abnormal vehicle protected in front and behind by normal remotely operated vehicles 10.

[0145] [Summary of the second embodiment] The second embodiment described above also provides the same effects as the first embodiment. In addition, in the second embodiment, when the mode switching unit 1427 permits in-vehicle driving, the driving control ECU 150 that controls the driving of the remotely driven vehicle 10 limits the driving speed of the remotely driven vehicle 10 more than when the vehicle is driven by remote driving (S210). Therefore, even after the in-vehicle driver 12 who is less accustomed to driving than the remote driver 21 takes over the control of the driving operation, the remotely driven vehicle 10 can continue to drive stably.

[0146] In addition, when the mode switching unit 1427 permits the specific occupant 14s to drive in the vehicle, the mode switching unit 1427 further determines whether to permit the specific occupant 14s to drive in the vehicle based on the occupant information on the specific occupant 14s who attempts to operate the in-vehicle operation member 13 (S208). Therefore, it is possible to prevent a situation in which the control of the driving operation is handed over to an undesirable occupant 14.

[0147] Furthermore, the mode switching unit 1427 acquires from the remote side system 30 a judgment result based on the occupant information, which is a judgment result as to whether or not to permit the specific occupant 14s to drive in the vehicle. Then, the mode switching unit 1427 decides whether or not to permit the specific occupant 14s to drive in the vehicle according to the acquired judgment result (S208). As a result, it is possible to more appropriately judge whether or not the specific occupant 14s is suitable to drive the remotely driven vehicle 10. As a result, even after transitioning to in-vehicle driving, the remotely driven vehicle 10 can continue to run stably.

[0148] Furthermore, when the mode switching unit 1427 acquires changeover information indicating a change of the remote driver 21 performing remote driving from the remote side system 30, it uses the in-vehicle notification unit 130 provided in the remotely driven vehicle 10 to provide a notification based on the changeover information (S252). As a result, even if a change in behavior occurs due to a change in the remote driver 21, the occupant 14 of the remotely driven vehicle 10 is less likely to feel uneasy.

[0149] In addition, the mode switching unit 1427 checks whether there is a replacement for the remote driver 21 who performs remote driving, and if there is no replacement, restricts some of the driving functions of the remotely driven vehicle 10 (S254). As a result of the above, even if the operation of multiple remotely driven vehicles 10 is managed by the remote side system 30, appropriate risk management is possible.

[0150] In addition, the remote assistance operation unit 368 provides driving instruction values ​​for remote driving to a specific remotely driven vehicle 10 among the multiple remotely driven vehicles 10 under the management of the remote side system 30. When a request to perform remote driving is acquired from such multiple remotely driven vehicles 10, the presentation control unit 367 notifies the remote driver 21 of the remotely driven vehicle 10 with a high priority to perform remote driving (S275). As described above, even in a form in which the operation of multiple remotely driven vehicles 10 is managed by the remote side system 30, the remote driver 21 can appropriately select the remotely driven vehicle 10 that should take over the driving operation and perform remote driving.

[0151] Furthermore, when an abnormality occurs in a front remotely-operated vehicle among the multiple remotely-operated vehicles 10 that are traveling in a convoy, the remote assistance operation unit 368 provides the rear remotely-operated vehicle 10, which is a normal remotely-operated vehicle 10, with an instruction to switch the front and rear positions of the rear remotely-operated vehicle and the front remotely-operated vehicle (S294). As a result, even if a minor abnormality occurs, the convoy can continue traveling. As a result, the convenience of the occupants 14 can be ensured while appropriately managing risks.

[0152] In addition, the mode switching unit 1427 displays driving status information indicating whether the remotely-driven vehicle 10 is being driven by remote driving or in-vehicle driving on the outside-vehicle display unit 135, which displays information to the outside of the vehicle. As a result, pedestrians and others around the remotely-driven vehicle 10 are less likely to feel uneasy about the remotely-driven vehicle 10.

[0153] In the second embodiment, the occupant information acquisition device 220 corresponds to an "in-vehicle driver monitoring device", the driving control ECU 150 corresponds to a "driving control unit", the presentation control unit 367 corresponds to a "notification control unit", and the remote assistance operation unit 368 corresponds to an "instruction providing unit". In addition, the state determination unit 1426 corresponds to a "remote side abnormality determination unit", and the mode switching unit 1427 corresponds to an "in-vehicle driving permission unit".

[0154] Although several embodiments have been described above, the disclosed technology is not limited to the above-described embodiments, and the following modifications are also included within the scope of the disclosure. Furthermore, in addition to the modifications described below, various modifications can be made without departing from the spirit of the invention.

[0155] <Variation 1> When permitting in-vehicle driving, the in-vehicle driving permission unit 1422 may permit the occupant to only operate the brakes as in-vehicle driving.

[0156] <Variation 2> In S6, the remote-driven vehicle 10 may output a different notification from the administrator notification device 34 when autonomous driving level 3 is being executed than when autonomous driving level 2 or lower is being executed. Specifically, the notification is to make the administrator aware that there is a higher degree of urgency when autonomous driving level 2 or lower is being executed than when autonomous driving level 3 is being executed.

[0157] <Other Modifications> Even when autonomous driving level 3 is being executed, when switching from remote driving to in-vehicle driving, the remotely driven vehicle 10 may be stopped once.

[0158] During remote driving, the mechanical mechanism between the in-vehicle operating member 13 and the corresponding actuator may be locked or disconnected, thereby disabling driving operations using the in-vehicle operating member 13.

[0159] In the embodiment, the remote side abnormality determination unit 1421 determines an abnormality in the remote driver 21 and the remote side system 30. However, the remote side abnormality determination unit 1421 may determine an abnormality in only one of the remote driver 21 and the remote side system 30.

[0160] The remote-controlled vehicle 10 may be a vehicle that does not have a driver's seat 11. In cases where the in-vehicle operating member 13 is only a button for stopping the remote-controlled vehicle 10, the driver's seat 11 may not be necessary.

[0161] The ECU and control device described in the present disclosure are the control units described below. In the present disclosure, the control unit and the method thereof may be realized by a dedicated computer that configures a processor programmed to execute one or more functions embodied in a computer program. Alternatively, the control unit and the method thereof described in the present disclosure may be realized by a dedicated hardware logic circuit. Alternatively, the control unit and the method thereof described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. The hardware logic circuit is, for example, an ASIC or an FPGA.

[0162] The storage medium for storing the computer program is not limited to a ROM. The various programs may be stored in a computer-readable non-transitory tangible storage medium as instructions executed by a computer. For example, the above programs may be stored in a flash memory. Furthermore, the form of the storage medium may be changed as appropriate. Such a storage medium is not limited to a configuration provided on a circuit board, but may be provided in the form of a memory card or the like, inserted into a slot, and electrically connected to a control circuit such as an autonomous driving ECU or a remote control device. Furthermore, the storage medium may be an optical disk or a hard disk drive, etc., from which the program is copied or distributed to the autonomous driving ECU or the remote control device.

[0163] (Disclosure of technical ideas) This specification discloses multiple technical ideas described in the following multiple dependent claims. Some of the claims may be described in a multiple dependent form, where the subsequent claim alternatively refers to the preceding claim. Furthermore, some of the claims may be described in a multiple dependent form, where the subsequent claim alternatively refers to the preceding claim. The claims described in these multiple dependent forms define multiple technical ideas.

[0164] (Technical thought 1) A vehicle remote driving system (5) for controlling a remote driving vehicle (10) capable of remote driving by a remote driver (21) and in-vehicle driving by operating an in-vehicle operating member (13) located inside the vehicle, a remote side abnormality determination unit (1421, 1426) for sequentially determining whether or not there is an abnormality in a remote side determination object, which is at least one of the remote driver and a remote side system (30) operated by the remote driver; A vehicle remote driving system comprising: an in-vehicle driving permission unit (1422, 1427) that permits in-vehicle driving when the remote-side abnormality judgment unit judges that there is an abnormality in the remote-side judgment target. (Technical thought 2) A vehicle remote driving system according to Technical Concept 1, The remote side abnormality determination unit sequentially determines whether there is an abnormality in the remote driver and whether there is an abnormality in the remote side system, The in-vehicle driving permission unit does not permit the in-vehicle driving when the remote side abnormality determination unit determines that neither the remote driver nor the remote side system is abnormal. (Technical Thought 3) A vehicle remote driving system according to Technical Concept 2, a driving control unit (150) provided in the remote-controlled vehicle and controlling driving of the remote-controlled vehicle based on driving instruction values ​​acquired from the remote-side system; The driving control unit is also capable of controlling the driving of the remotely driven vehicle based on the amount of operation by which the in-vehicle operating member is operated, and even when the in-vehicle driving permission unit does not permit the in-vehicle driving, the driving control unit sequentially acquires signals indicating the amount of operation by which the in-vehicle operating member is operated, and when the in-vehicle driving permission unit permits the in-vehicle driving, controls the driving of the remotely driven vehicle based on the signals indicating the amount of operation that have already been acquired, in a vehicle remote driving system. (Technical Thought 4) A vehicle remote driving system according to any one of technical concepts 1 to 3, The remote-operated vehicle includes a driver's seat (11) in which the driver sits when operating the in-vehicle operation member, and an in-vehicle driver monitoring device (120, 220) that sequentially determines an alertness level of the in-vehicle driver, which is a passenger sitting in the driver's seat; The remote side abnormality determination unit sequentially determines whether or not there is an abnormality in at least the remote driver, and also sequentially determines whether or not there is a sign of an abnormality in the remote driver based on the wakefulness level of the remote driver; The in-vehicle driving permission unit permits driving operations to be performed by the remote driver or the in-vehicle driver having the higher alertness level when the remote side abnormality judgment unit judges that the remote driver has signs of abnormality and the in-vehicle driver monitoring device judges the alertness level of the in-vehicle driver. (Technical Thought 5) A vehicle remote driving system according to any one of technical concepts 1 to 3, The remote-controlled vehicle is capable of automatic driving without a monitoring obligation, which requires the driver to be in a state where he or she is able to take over driving, but is not required to monitor the surroundings, and is equipped with a driver's seat (11) in which the driver sits when operating the in-vehicle operation members, and an in-vehicle driver monitoring device (120, 220) that monitors whether the in-vehicle driver, who is a passenger sitting in the driver's seat, is in a state where he or she is able to take over driving, The in-vehicle driving permission unit permits the in-vehicle driving when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment target during the remote driving and the autonomous driving without monitoring obligation, and when the in-vehicle driver is in a state where he or she is able to take over driving, switches the driver in the autonomous driving without monitoring obligation to the in-vehicle driver and continues the autonomous driving without monitoring obligation. (Technical Thought 6) A vehicle remote driving system according to any one of technical concepts 1 to 4, The in-vehicle driving permission unit permits only brake operation as the in-vehicle driving. (Technical Thought 7) A vehicle remote driving system according to any one of technical concepts 1 to 6, A driving control unit (150) for controlling driving of the remote-operated vehicle is provided, A vehicle remote driving system, wherein the driving control unit, when the in-vehicle driving permission unit permits the in-vehicle driving, limits the driving speed of the remotely driven vehicle more than when the vehicle is driven by remote driving. (Technical Thought 8) A vehicle remote driving system according to Technical Concept 6 or 7, The in-vehicle driving permission unit, when permitting the in-vehicle driving, further determines whether or not to permit the specific occupant (14s) to drive in the vehicle based on occupant information regarding the specific occupant (14s) who attempts to operate the in-vehicle operating member, in a vehicle remote driving system. (Technical Thought 9) A vehicle remote driving system according to Technical Concept 8, The in-vehicle driving permission unit, obtaining, from the remote system, a judgment result based on the occupant information, the judgment result being a judgment as to whether or not to permit the specific occupant to drive in the vehicle; A vehicle remote driving system that determines whether or not to permit the specific occupant to drive in the vehicle based on the judgment result. (Technical Thought 10) A vehicle remote driving system according to any one of technical concepts 1 to 9, The in-vehicle driving permission unit, stopping the remote-operated vehicle when the remote-side abnormality determination unit determines that there is an abnormality in the remote-side determination target; A vehicle remote driving system that permits resumption of driving by remote driving when a resolution process is performed to resolve the abnormality of the remote-side judgment target after the remote-driving vehicle is stopped. (Technical Thought 11) A vehicle remote driving system according to any one of technical concepts 1 to 10, The in-vehicle driving permission unit stops the remotely driven vehicle when the remote-side abnormality judgment unit judges that there is an abnormality in the remote-side judgment target, then permits the in-vehicle driving, and automatically transitions to an in-vehicle driving mode in which the remotely driven vehicle runs when the in-vehicle operating member is operated. (Technical Thought 12) A vehicle remote driving system according to Technical Concept 1, A vehicle remote driving system in which, when the in-vehicle driving permission unit obtains change information indicating a change of the remote driver performing the remote driving from the remote side system, the in-vehicle driving permission unit provides a notification based on the change information using an in-vehicle notification unit (130) provided in the remotely driven vehicle. (Technical Thought 13) A vehicle remote driving system according to Technical Concept 12, The in-vehicle driving permission unit determines whether or not there is a replacement driver for the remote driver who performs the remote driving, and if there is no replacement driver, restricts some of the driving functions of the remotely driven vehicle. (Technical Thought 14) A vehicle remote driving system according to any one of technical concepts 1 to 13, The in-vehicle driving permission unit notifies, when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment target, that switching to in-vehicle driving has been made from an in-vehicle notification unit (130) provided in the remotely driven vehicle, in a vehicle remote driving system. (Technical Thought 15) A vehicle remote driving system according to Technical Concept 14, The remote-operated vehicle can switch between automatic driving without a monitoring obligation in which the remote driver has no obligation to monitor the surroundings and automatic driving with a monitoring obligation in which the remote driver has an obligation to monitor the surroundings, A vehicle remote driving system in which the in-vehicle driving permission unit differs in the content of the notification when the remote-side abnormality judgment unit determines that there is an abnormality in the remote-side judgment target depending on whether the autonomous driving without monitoring obligation is being performed or the autonomous driving with monitoring obligation is being performed. (Technical Thought 16) A vehicle remote driving system according to any one of technical concepts 1 to 15, a remote driver monitoring device (32) that continuously monitors whether the remote driver is in a state suitable for remote driving; and an administrator notification device (34) that notifies an administrator (25), who is capable of operating the remote system and is separate from the remote driver, that the remote driver is not in a state suitable for remote driving based on the monitoring results of the remote driver monitoring device indicating that the remote driver is not in a state suitable for remote driving. (Technical Thought 17) A vehicle remote driving system according to Technical Concept 16, The remote-operated vehicle can switch between automatic driving without a monitoring obligation in which the remote driver has no obligation to monitor the surroundings and automatic driving with a monitoring obligation in which the remote driver has an obligation to monitor the surroundings, The administrator notification device is a vehicle remote driving system in which the content of the notification when the remote side abnormality judgment unit determines that there is an abnormality in the remote driver differs depending on whether the autonomous driving without monitoring obligation is being performed or the autonomous driving with monitoring obligation is being performed. (Technical Thought 18) A vehicle remote driving system according to any one of technical concepts 1 to 17, An instruction providing unit (368) that provides driving instruction values ​​for the remote driving to a specific remote-operated vehicle among the multiple remote-operated vehicles under the management of the remote side system; A vehicle remote driving system comprising: a notification control unit (367) that, when receiving requests to perform the remote driving from multiple remotely driven vehicles, notifies the remote driver of the remotely driven vehicle that has a high priority to perform the remote driving. (Technical Thought 19) A vehicle remote driving system according to any one of technical concepts 1 to 18, An instruction providing unit (368) that provides driving instruction values ​​for the remote driving to a specific remote-operated vehicle among a plurality of remote-operated vehicles under the management of the remote side system, In a vehicle remote driving system, when an abnormality occurs in a front remote vehicle traveling ahead of the other remotely driven vehicles among the multiple remotely driven vehicles traveling in a convoy, the instruction providing unit provides the rear remote vehicle and the front remote vehicle with an instruction to swap the front and rear positions of the rear remote vehicle, which is a normal remotely driven vehicle, and the front remote vehicle. (Technical Thought 20) A vehicle remote driving system according to any one of technical concepts 1 to 19, The in-vehicle driving permission unit displays driving status information indicating whether the remotely driven vehicle is being driven by remote driving or in-vehicle driving on an exterior display unit (135) of the remotely driven vehicle, which displays information to the outside of the vehicle. [Explanation of symbols]

[0165] 5: vehicle remote driving system, 10: remote driving vehicle, 11: driver's seat, 12: in-vehicle driver, 13: in-vehicle operation member, 14s: specific occupant, 21: remote driver, 25: remote manager (manager), 30: remote side system, 32: remote driver monitoring device, 34: manager notification device, 120: in-vehicle driver monitoring device, 130: in-vehicle notification unit, 135: outside-vehicle display unit, 140: automatic driving system, 142: automatic driving ECU, 150: driving Control ECU (driving control unit), 220: occupant information acquisition device (in-vehicle driver monitoring device), 361: driving information acquisition unit, 362: remote driver state notification unit, 363: driving instruction value acquisition unit, 367: presentation control unit (notification control unit), 368: remote support operation unit (instruction providing unit), 1421: remote side abnormality determination unit, 1422: in-vehicle driving permission unit, 1426: state determination unit (remote side abnormality determination unit), 1427: mode switching unit (in-vehicle driving permission unit)

Claims

1. A vehicle remote driving system (5) for controlling a remote driving vehicle (10) for a mobility service that can be remotely driven by a remote driver (21) and in-vehicle driven by operating an in-vehicle operating member (13) in the vehicle, a remote side abnormality determination unit (1421, 1426) for sequentially determining whether or not there is an abnormality in a remote side determination object, which is at least one of the remote driver and a remote side system (30) operated by the remote driver; an in-vehicle driving permission unit (1422, 1427) that, when the remote side abnormality judgment unit does not judge that there is an abnormality in the remote side judgment object, does not permit the in-vehicle driving, and that, when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment object, permits the in-vehicle driving; A driving control unit (150) that controls driving of the remote-operated vehicle; Equipped with A vehicle remote driving system in which the driving control unit limits the driving speed of the remotely driven vehicle when the in-vehicle driving permission unit permits the in-vehicle driving, more than when the remotely driven vehicle is driven by the remote driving.

2. A vehicle remote driving system (5) for controlling a remote driving vehicle (10) for a mobility service that can be remotely driven by a remote driver (21) and in-vehicle driven by operating an in-vehicle operating member (13) in the vehicle, a remote side abnormality determination unit (1421, 1426) for sequentially determining whether or not there is an abnormality in a remote side determination object, which is at least one of the remote driver and a remote side system (30) operated by the remote driver; an in-vehicle driving permission unit (1422, 1427) that, when the remote side abnormality judgment unit does not judge that there is an abnormality in the remote side judgment object, does not permit the in-vehicle driving, and, when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment object, permits the in-vehicle driving, The vehicle remote driving system further determines whether or not to permit a specific occupant (14s) to drive in the vehicle when the in-vehicle driving is permitted, based on occupant information regarding the specific occupant (14s) who is a passenger in the remotely driven vehicle and attempts to operate the in-vehicle operating member.

3. 3. The vehicle remote driving system according to claim 2, The in-vehicle driving permission unit, obtaining, from the remote system, a judgment result based on the occupant information, the judgment result being a judgment as to whether or not to permit the specific occupant to drive in the vehicle; A vehicle remote driving system that determines whether or not to permit the specific occupant to drive in the vehicle based on the judgment result.

4. A vehicle remote driving system (5) for controlling a remote driving vehicle (10) for a mobility service that can be remotely driven by a remote driver (21) and in-vehicle driven by operating an in-vehicle operating member (13) in the vehicle, a remote side abnormality determination unit (1421, 1426) for sequentially determining whether or not there is an abnormality in a remote side determination object, which is at least one of the remote driver and a remote side system (30) operated by the remote driver; an in-vehicle driving permission unit (1422, 1427) that, when the remote side abnormality judgment unit does not judge that there is an abnormality in the remote side judgment object, does not permit the in-vehicle driving, and that, when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment object, permits the in-vehicle driving, The in-vehicle driving permission unit determines whether or not there is a replacement driver for the remote driver performing the remote driving, and if there is no replacement driver, restricts some of the driving functions of the remotely driven vehicle during remote driving.

5. A vehicle remote driving system according to any one of claims 1, 2 and 4, The remote side abnormality determination unit sequentially determines whether there is an abnormality in the remote driver and whether there is an abnormality in the remote side system, The in-vehicle driving permission unit does not permit the in-vehicle driving when the remote side abnormality determination unit determines that neither the remote driver nor the remote side system is abnormal.

6. 6. The vehicle remote driving system according to claim 5, A driving control unit (150) is provided in the remote-controlled vehicle and controls driving of the remote-controlled vehicle based on driving instruction values ​​acquired from the remote-side system; The driving control unit is also capable of controlling the driving of the remotely driven vehicle based on the amount of operation by which the in-vehicle operating member is operated, and even when the in-vehicle driving permission unit does not permit the in-vehicle driving, the driving control unit sequentially acquires signals indicating the amount of operation by which the in-vehicle operating member is operated, and when the in-vehicle driving permission unit permits the in-vehicle driving, controls the driving of the remotely driven vehicle based on the signals indicating the amount of operation that have already been acquired, in a vehicle remote driving system.

7. A vehicle remote driving system according to any one of claims 1, 2 and 4, The in-vehicle driving permission unit stops the remotely driven vehicle when the remote-side abnormality judgment unit judges that there is an abnormality in the remote-side judgment target, and then permits the in-vehicle driving, and when the in-vehicle operating member is operated, automatically transitions to an in-vehicle driving mode in which the remotely driven vehicle is running.

8. A vehicle remote driving system according to any one of claims 1, 2 and 4, The remote-operated vehicle includes a driver's seat (11) in which the driver sits when operating the in-vehicle operation member, and an in-vehicle driver monitoring device (120, 220) that sequentially determines an alertness level of the in-vehicle driver who is an occupant sitting in the driver's seat, The remote side abnormality determination unit sequentially determines whether or not there is an abnormality in at least the remote driver, and also sequentially determines whether or not there is a sign of an abnormality in the remote driver based on the wakefulness level of the remote driver; The in-vehicle driving permission unit permits driving operations to be performed by the remote driver or the in-vehicle driver having the higher alertness level when the remote side abnormality judgment unit judges that the remote driver has signs of abnormality and the in-vehicle driver monitoring device judges the alertness level of the in-vehicle driver.

9. A vehicle remote driving system according to any one of claims 1, 2 and 4, The remote-operated vehicle is capable of automatic driving without a monitoring obligation, which requires the driver to be in a state where he or she is able to take over driving, but is not required to monitor the surroundings, and is equipped with a driver's seat (11) in which the driver sits when operating the in-vehicle operation member, and an in-vehicle driver monitoring device (120, 220) that monitors whether the in-vehicle driver, who is an occupant sitting in the driver's seat, is in a state where he or she is able to take over driving, The in-vehicle driving permission unit permits the in-vehicle driving when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment target during the remote driving and the autonomous driving without monitoring obligation, and when the in-vehicle driver is in a state where he or she is able to take over driving, switches the driver in the autonomous driving without monitoring obligation to the in-vehicle driver and continues the autonomous driving without monitoring obligation.

10. A vehicle remote driving system according to any one of claims 1, 2 and 4, The in-vehicle driving permission unit permits only brake operation as the in-vehicle driving.

11. A vehicle remote driving system according to any one of claims 1, 2 and 4, The in-vehicle driving permission unit, stopping the remote-operated vehicle when the remote-side abnormality determination unit determines that there is an abnormality in the remote-side determination target; A vehicle remote driving system that permits resumption of driving by remote driving when a resolution process is performed to resolve the abnormality of the remote-side judgment target after the remote-driving vehicle is stopped.

12. A vehicle remote driving system according to any one of claims 1, 2 and 4, A vehicle remote driving system in which, when the in-vehicle driving permission unit obtains change information indicating a change of the remote driver performing the remote driving from the remote side system, the in-vehicle driving permission unit provides a notification based on the change information using an in-vehicle notification unit (130) provided in the remotely driven vehicle.

13. A vehicle remote driving system according to any one of claims 1, 2 and 4, The in-vehicle driving permission unit notifies the remote driving vehicle of a switch to in-vehicle driving from an in-vehicle notification unit (130) provided in the remote driving vehicle when the remote side abnormality judgment unit judges that there is an abnormality in the remote side judgment target.

14. 14. The vehicle remote driving system according to claim 13, The remote-operated vehicle can switch between automatic driving without a monitoring obligation in which the remote driver has no obligation to monitor the surroundings and automatic driving with a monitoring obligation in which the remote driver has an obligation to monitor the surroundings, A vehicle remote driving system in which the in-vehicle driving permission unit differs in the content of the notification when the remote-side abnormality judgment unit determines that there is an abnormality in the remote-side judgment target depending on whether the autonomous driving without monitoring obligation is being performed or the autonomous driving with monitoring obligation is being performed.

15. A vehicle remote driving system according to any one of claims 1, 2 and 4, A remote driver monitoring device (32) that continuously monitors whether the remote driver is in a state suitable for the remote driving; and an administrator notification device (34) that notifies an administrator (25), who is capable of operating the remote system and is separate from the remote driver, that the remote driver is not in a state suitable for remote driving based on the monitoring results of the remote driver monitoring device indicating that the remote driver is not in a state suitable for remote driving.

16. 16. The vehicle remote driving system according to claim 15, The remote-operated vehicle can switch between automatic driving without a monitoring obligation in which the remote driver has no obligation to monitor the surroundings and automatic driving with a monitoring obligation in which the remote driver has an obligation to monitor the surroundings, The administrator notification device is a vehicle remote driving system in which the content of the notification when the remote side abnormality judgment unit determines that there is an abnormality in the remote driver differs depending on whether the autonomous driving without monitoring obligation is being performed or the autonomous driving with monitoring obligation is being performed.

17. A vehicle remote driving system according to any one of claims 1, 2 and 4, An instruction providing unit (368) that provides driving instruction values ​​for the remote driving to a specific remote-driving vehicle among a plurality of remote-driving vehicles under the management of the remote side system; A vehicle remote driving system comprising: a notification control unit (367) that, when a request to perform the remote driving is received from a plurality of the remote driving vehicles, notifies the remote driver of the remote driving vehicle that has a high priority to perform the remote driving.

18. A vehicle remote driving system according to any one of claims 1, 2 and 4, An instruction providing unit (368) that provides driving instruction values ​​for the remote driving to a specific remote driving vehicle among a plurality of remote driving vehicles under the management of the remote side system; In a vehicle remote driving system, when an abnormality occurs in a front remote vehicle traveling ahead of the other remotely driven vehicles among the multiple remotely driven vehicles traveling in a convoy, the instruction providing unit provides the rear remote vehicle and the front remote vehicle with an instruction to swap the front and rear positions of the rear remote vehicle, which is a normal remotely driven vehicle, and the front remote vehicle.

19. A vehicle remote driving system according to any one of claims 1, 2 and 4, The in-vehicle driving permission unit displays driving status information indicating whether the remotely driven vehicle is being driven by remote driving or in-vehicle driving on an exterior display unit (135) of the remotely driven vehicle, which displays information to the outside of the vehicle.

20. A vehicle remote driving method for controlling a remote driving vehicle (10) for a mobility service that can be remotely driven by a remote driver (21) and in-vehicle driven by operating an in-vehicle operating member (13) in the vehicle, comprising: Sequentially judging whether or not there is an abnormality in a remote side judgment object, which is at least one of the remote driver and a remote side system (30) operated by the remote driver; When it is determined that there is no abnormality in the remote side judgment object, the in-vehicle driving is not permitted, and when it is determined that there is an abnormality in the remote side judgment object, the in-vehicle driving is permitted; When the in-vehicle driving is permitted, the driving speed of the remotely driven vehicle is limited to a speed lower than when the vehicle is driven by the remote driving. The vehicle remote driving method includes the above in the processing executed by the processing unit.

21. A vehicle remote driving method for controlling a remote driving vehicle (10) for a mobility service that can be remotely driven by a remote driver (21) and in-vehicle driven by operating an in-vehicle operating member (13) in the vehicle, comprising: Sequentially judging whether or not there is an abnormality in a remote side judgment object, which is at least one of the remote driver and a remote side system (30) operated by the remote driver; When it is determined that there is no abnormality in the remote side judgment object, the in-vehicle driving is not permitted, and when it is determined that there is an abnormality in the remote side judgment object, the in-vehicle driving is permitted; When the driving in the vehicle is permitted, it is further determined whether or not to permit the specific occupant (14s) to drive in the vehicle based on occupant information regarding the specific occupant (14s) who is a passenger of the remote driving vehicle and attempts to operate the in-vehicle operating member. The vehicle remote driving method includes the above in the processing executed by the processing unit.

22. A vehicle remote driving method for controlling a remote driving vehicle (10) for a mobility service that can be remotely driven by a remote driver (21) and in-vehicle driven by operating an in-vehicle operating member (13) in the vehicle, comprising: Sequentially judging whether or not there is an abnormality in a remote side judgment object, which is at least one of the remote driver and a remote side system (30) operated by the remote driver; When it is determined that there is no abnormality in the remote side judgment object, the in-vehicle driving is not permitted, and when it is determined that there is an abnormality in the remote side judgment object, the in-vehicle driving is permitted; Grasp the presence or absence of a replacement driver for the remote driving, and if there is no replacement driver, restrict a part of the driving function of the remote driving vehicle in the remote driving. The vehicle remote driving method includes the above in the processing executed by the processing unit.

Images