Remote driving application and remote driving system

Abstract

To provide a remote travel application and a remote travel system which enable a user of an operation terminal to easily recognize that a shift state of a transmission device of an own vehicle is changed during the execution of remote travel control of remotely making the own vehicle autonomous.SOLUTION: A remote travel application is installed in an operation terminal 100 for the execution of remote travel control of performing wireless communication between an own vehicle 200 and the operation terminal, responding to operation to the operation terminal and making the own vehicle autonomous. When a control state of the own vehicle is in a state of changing a shift state of a transmission device during the execution of remote travel control, the remote travel application generates vibration in the operation terminal by a pattern different from that in a state excluding the state, or outputs a notification sound from the operation terminal.SELECTED DRAWING: Figure 42

Description

[Technical field]

[0001] The present invention relates to a remote driving application and a remote driving system. [Background technology]

[0002] There are known remote driving applications and remote driving systems that allow a vehicle's control device to be remotely operated by operating a terminal (operation terminal) such as a mobile phone, causing the vehicle to drive autonomously and park in a parking space (see, for example, Patent Document 1).

[0003] In the conventional remote driving application and remote driving system (hereinafter, remote driving application, etc.), when an obstacle that hinders the driving of the vehicle is detected while the vehicle is being driven autonomously, the operation terminal is vibrated to notify the user of the operation terminal of the presence of the obstacle. In addition, when the conventional remote driving application, etc., stops the autonomously driving vehicle and when the stopped vehicle is started autonomously, the operation terminal is vibrated to notify the user of the operation terminal that the vehicle is stopping and starting. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2015-516772 Summary of the Invention

[0005] The conventional remote driving applications described above change the vibration frequency, etc., when vibrating the operating terminal to notify the user of the presence of an obstacle, when the vehicle is about to stop, and when the vehicle is about to start, allowing the user to distinguish from the vibration of the operating terminal whether an obstacle is present, the vehicle is about to stop, or the vehicle is about to start.

[0006] However, the presence of an obstacle, the stopping of the vehicle, and the starting of the vehicle are events that the user operating the operation terminal can relatively easily recognize by looking at the vehicle, but for example, the change in the shift state of the transmission device in order for the vehicle to turn is an event that the user cannot easily recognize even if he or she is looking at the vehicle. Therefore, being able to easily allow the user to recognize events that the user cannot easily recognize even if he or she is looking at the vehicle using the operation terminal provides a certain benefit to the user. However, conventional remote driving applications and the like are not designed to allow the user to easily recognize such events that the user cannot easily recognize by visual inspection using the operation terminal.

[0007] The object of the present invention is to provide a remote driving application and a remote driving system that can allow a user of an operating terminal to easily recognize that the shift state of a transmission device of a vehicle has been changed while remote driving control is being executed to remotely and autonomously drive the vehicle.

[0008] The remote driving application according to the present invention performs wireless communication between a vehicle control device of a vehicle itself and a terminal control device of an operation terminal, Display Operations on An operation in which a user of the operation terminal slides a finger on the display while keeping the finger in contact with the display. In response to Transmitting a continuous touch operation signal to the vehicle control device; The remote driving application is installed in the terminal control device to cause the vehicle control device to execute remote driving control for autonomously driving the host vehicle. The remote driving application according to the present invention is configured to cause the operation terminal to generate vibrations or output a notification sound from the operation terminal in accordance with a control state of the host vehicle when the vehicle control device is executing the remote driving control. In addition, the vehicle control device is configured to perform autonomous driving and braking of the vehicle and autonomous change of the shift state of the transmission device of the vehicle through the remote driving control, on the condition that the continuous touch operation signal is received. The remote driving application according to the present invention is configured to: The vehicle control device but In a state where the vehicle is autonomously braked and stopped, before Note Soft state When the vehicle is autonomously changed from a forward drive state in which the vehicle is driven forward to a reverse drive state in which the vehicle is driven backward, or when the vehicle is autonomously changed from the reverse drive state to the forward drive state sometimes, The vehicle control device before Note Soft state When a control state of the host vehicle occurs other than a control state of the host vehicle in which the forward drive state is autonomously changed to the reverse drive state, and when a control state of the host vehicle occurs other than a control state of the host vehicle in which the vehicle control device autonomously changes to the forward drive state from the reverse drive state. The control terminal is configured to generate vibrations having different vibration patterns or output notification sounds having different output patterns from the control terminal when the control terminal is turned on.

[0009] According to the present invention, the shift state of the transmission device of the host vehicle is changed during execution of remote driving control. did Since vibrations of different vibration patterns are generated in the operation terminal or notification sounds of different output patterns are output from the operation terminal when the shift state of the transmission device of the vehicle is changed and when the shift state is not changed, the user of the operation terminal can easily recognize that the shift state of the transmission device of the vehicle has been changed by the vibration or notification sound of the operation terminal.

[0010] In particular, the remote driving application according to the present invention is configured to: The vehicle control device The vehicle is Autonomous Running And When the vehicle control device is executing the remote driving control, the remote driving application is configured to generate a vibration of a predetermined driving vibration pattern in the operation terminal or output a notification sound of a predetermined driving notification sound output pattern from the operation terminal. The vehicle control device but In a state where the vehicle is autonomously braked and stopped, before Note Soft state When the forward drive state is autonomously changed to the reverse drive state or when the reverse drive state is autonomously changed to the forward drive state Sometimes, the device is configured to generate vibrations in a shift change vibration pattern different from the driving vibration pattern in the operating terminal, or to output a notification sound in a shift change notification sound output pattern different from the driving notification sound output pattern from the operating terminal.

[0011] According to the present invention, when remote driving control is being executed, the vibration pattern or output pattern is different when the vehicle is driving and when the shift state of the transmission device of the vehicle is changed, so that the user of the operation terminal can easily recognize whether the vehicle is driving or whether the shift state of the transmission device of the vehicle has been changed by the vibration or notification sound of the operation terminal.

[0012] In addition, the remote driving application according to the present invention is configured to execute the remote driving control by causing the vehicle control device to execute the remote driving control. The vehicle control device The vehicle is Autonomous Braking And In this case, the remote driving application according to the present invention may be further configured to cause the operation terminal to generate vibrations of a predetermined braking vibration pattern or to output a notification sound of a predetermined braking notification sound output pattern from the operation terminal when the vehicle control device is executing the remote driving control. The vehicle control device but In a state where the vehicle is autonomously braked and stopped, before Note Soft state When the forward drive state is autonomously changed to the reverse drive state or when the reverse drive state is autonomously changed to the forward drive state Sometimes, the device may be configured to generate vibrations in a shift change vibration pattern different from the braking vibration pattern in the operating terminal, or to output a notification sound in a shift change notification sound output pattern different from the braking notification sound output pattern from the operating terminal.

[0013] According to the present invention, when remote driving control is being executed, the vibration pattern or output pattern is different when the vehicle is being braked and when the shift state of the transmission device of the vehicle is changed, so that the user of the operating terminal can easily recognize whether the vehicle is being braked or whether the shift state of the transmission device of the vehicle has been changed.

[0014] Furthermore, the remote driving application of the present invention may be configured to, when an obstacle is detected in the vicinity of the vehicle while the vehicle control device is executing the remote driving control, cause the operation terminal to generate vibrations having a vibration pattern different from the vibration pattern generated in the operation terminal in accordance with the control state of the vehicle, or to output from the operation terminal a notification sound having an output pattern different from the output pattern for outputting a notification sound from the operation terminal in accordance with the control state of the vehicle.

[0015] According to the present invention, when an obstacle is detected during remote driving control, a vibration having a vibration pattern different from the vibration pattern corresponding to the control state of the vehicle is generated on the operating terminal, or a notification sound having an output pattern different from the output pattern corresponding to the control state of the vehicle is output from the operating terminal, so that the vibration or notification sound of the operating terminal allows the user of the operating terminal to easily recognize the control state of the vehicle and the presence of an obstacle.

[0016] In addition, the remote driving system according to the present invention performs wireless communication between a vehicle control device of the vehicle itself and a terminal control device of an operation terminal, Display Operations on An operation in which a user of the operation terminal slides a finger on the display while keeping the finger in contact with the display. In response to Transmitting a continuous touch operation signal to the vehicle control device; The remote driving system according to the present invention is configured to cause the operation terminal to generate vibrations or output a notification sound in accordance with a control state of the host vehicle when the vehicle control device is executing the remote driving control. In addition, the vehicle control device is configured to perform autonomous driving and braking of the vehicle and autonomous change of the shift state of the transmission device of the vehicle through the remote driving control, on the condition that the continuous touch operation signal is received. The terminal control device, when causing the vehicle control device to execute the remote driving control, The vehicle control device but In a state where the vehicle is autonomously braked and stopped, before Note Soft state When the vehicle is autonomously changed from a forward drive state in which the vehicle is driven forward to a reverse drive state in which the vehicle is driven backward, or when the vehicle is autonomously changed from the reverse drive state to the forward drive state sometimes, The vehicle control device In front Note Soft state When a control state of the host vehicle occurs other than a control state of the host vehicle in which the forward drive state is autonomously changed to the reverse drive state, and when a control state of the host vehicle occurs other than a control state of the host vehicle in which the vehicle control device autonomously changes to the forward drive state from the reverse drive state. When the user presses the button, the operation terminal generates a vibration having a different vibration pattern or outputs a notification sound having a different output pattern from the operation terminal. do.

[0017] According to the present invention, for the same reasons as described above, the user of the operating terminal can easily recognize that the shift state of the transmission device of the vehicle has been changed by the vibration or notification sound of the operating terminal.

[0018] The components of the present invention are not limited to the embodiments of the present invention described below with reference to the drawings. Other objects, other features, and associated advantages of the present invention will be easily understood from the description of the embodiments of the present invention. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a diagram showing a remote driving system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a remote driving system according to a modified embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing a remote driving system according to another modified example of the embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing a terminal control device constituting the remote driving system according to the embodiment of the present invention, an operation terminal equipped with the terminal control device, and the like. [Diagram 5] FIG. 5 is a diagram showing a vehicle control device constituting a remote driving system according to an embodiment of the present invention, a vehicle (host vehicle) equipped with the vehicle control device, and the like. [Figure 6] FIG. 6 is a diagram showing a scene in which the host vehicle is stopped near a parking space. [Figure 7] FIG. 7 is a diagram showing an operation terminal displaying an initial screen image on a terminal display. [Figure 8] FIG. 8 is a diagram showing an operation terminal displaying an entry / exit selection image on a terminal display. [Figure 9] FIG. 9 is a diagram showing an operation terminal displaying a parking space selection image on the terminal display. [Figure 10] FIG. 10 is a diagram showing a target storage route. [Figure 11] FIG. 11 is a diagram showing an operation terminal on which a storage operation image is displayed on a terminal display. [Figure 12] FIG. 12 is a diagram showing an operation terminal on which a storage operation image is displayed on a terminal display. [Figure 13] FIG. 13 is a diagram showing a scene in which the vehicle moves forward while turning right under the remote entry control. [Figure 14] FIG. 14 is a diagram showing an operation terminal on which a storage operation image is displayed on a terminal display. [Figure 15] FIG. 15 is a diagram showing a scene in which the vehicle is reversed while turning left due to the remote entry control. [Figure 16] FIG. 16 is a diagram showing a scene in which the vehicle is backed up straight by the remote entry control. [Figure 17] FIG. 17 is a diagram showing a scene where the entry of the vehicle under the remote entry control is completed. [Figure 18] FIG. 18 is a diagram showing an operation terminal on which a storage operation image is displayed on a terminal display. [Figure 19] FIG. 19 is a diagram showing a scene in which the vehicle is parked in a parking space. [Figure 20] FIG. 20 is a diagram showing an operation terminal on which an image for selecting a leaving direction is displayed. [Figure 21] FIG. 21 is a diagram showing a target delivery route. [Figure 22] FIG. 22 is a diagram showing an operation terminal on which a take-out operation image is displayed on a terminal display. [Diagram 23] FIG. 23 is a diagram showing an operation terminal on which a take-out operation image is displayed on a terminal display. [Figure 24] FIG. 24 is a diagram showing a scene in which the vehicle moves forward straight by remote leaving control. [Diagram 25] FIG. 25 is a diagram showing a scene in which the vehicle has completed leaving the garage under the remote leaving control. [Figure 26]FIG. 26 is a diagram showing an operation terminal on which a take-out operation image is displayed. [Figure 27] FIG. 27 is a diagram showing a time chart illustrating vibration of the operation terminal during execution of remote driving control. [Figure 28] FIG. 28 is a diagram showing a time chart illustrating vibration of the operation terminal during execution of remote driving control. [Figure 29] FIG. 29 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. [Diagram 30] FIG. 30 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. [Diagram 31] FIG. 31 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. [Diagram 32] FIG. 32 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. [Diagram 33] FIG. 33 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. [Diagram 34] FIG. 34 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. [Diagram 35] FIG. 35 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. [Diagram 36] FIG. 36 is a flowchart showing a routine executed by the vehicle control device according to the embodiment of the present invention. [Figure 37] FIG. 37 is a flowchart showing a routine executed by the vehicle control device according to the embodiment of the present invention. [Figure 38] FIG. 38 is a flowchart showing a routine executed by the vehicle control device according to the embodiment of the present invention. [Figure 39] FIG. 39 is a flowchart showing a routine executed by the vehicle control device according to the embodiment of the present invention. [Diagram 40] FIG. 40 is a flowchart showing a routine executed by the vehicle control device according to the embodiment of the present invention. [Diagram 41] FIG. 41 is a flowchart showing a routine executed by the vehicle control device according to the embodiment of the present invention. [Diagram 42] FIG. 42 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. [Diagram 43] FIG. 43 is a flowchart showing a routine executed by the vehicle control device according to the embodiment of the present invention. [Diagram 44] FIG. 44 is a flowchart showing a routine executed by the terminal control device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, a remote driving application and a remote driving system according to an embodiment of the present invention will be described with reference to the drawings.

[0021] The remote driving application according to the embodiment of the present invention described below is an application (remote entry / exit application) that is installed on the terminal control device to enable the vehicle control device to perform remote driving control (remote entry / exit control) by wirelessly communicating between the control device of the vehicle (vehicle control device) and the control device of the operation terminal (terminal control device) and causing the vehicle control device to autonomously drive the vehicle to park (enter) in a parking space in response to an operation on the operation terminal, and to autonomously drive the vehicle to exit from a parking space in response to an operation on the operation terminal.

[0022] Furthermore, the remote driving system described below is a system (remote entry / exit system) that performs wireless communication between a vehicle control device and a terminal control device, and causes the vehicle control device to execute the above-mentioned remote driving control (remote entry / exit control).

[0023] However, the remote driving application and the remote driving system according to the present invention (hereinafter, "remote driving application, etc.") are not limited to such a remote entry / exit application and a remote entry / exit system. The remote driving application, etc. according to the present invention may be any application or system that executes remote driving control to remotely and autonomously drive the vehicle from outside the vehicle in response to an operation on an operation terminal.

[0024] 1, the remote driving system 10 according to the embodiment of the present invention includes at least a control device (terminal control device 110) mounted on the operation terminal 100 and a control device (vehicle control device 210) mounted on the vehicle 200. The remote driving application according to the embodiment of the present invention is installed in the terminal control device 110.

[0025] In the remote driving system 10, the terminal control device 110 and the vehicle control device 210 are configured to be able to directly transmit and receive various signals wirelessly, but as shown in FIG. 2, the system may be configured to include at least the terminal control device 110, the vehicle control device 210, and the Internet 300, and to transmit and receive various signals between the terminal control device 110 and the vehicle control device 210 via the Internet 300.

[0026] Furthermore, as shown in FIG. 3, the remote driving system 10 may include at least a terminal control device 110, a vehicle control device 210, the Internet 300, and a server 301 arranged on the Internet 300, and may be configured to transmit and receive various signals between the terminal control device 110 and the vehicle control device 210 via the Internet 300 and the server 301.

[0027] <Operation terminal> In this example, the operation terminal 100 is a so-called smartphone, which is a telephone device that can be carried by a person, but it may be any terminal that is separate from the vehicle 200 and can be taken outside the vehicle 200 by a user such as the driver DR of the vehicle 200, for example, a so-called smart key that can be carried by a person or a terminal dedicated to remote entry and exit.

[0028] 4, the operation terminal 100 includes a terminal control device 110. The terminal control device 110 includes a terminal ECU 190.

[0029] ECU is an abbreviation for electronic control unit. The terminal ECU 190 has a microcomputer as its main component. The terminal ECU 190 includes a CPU (terminal CPU 191), a ROM, a RAM, a non-volatile memory, an interface, and the like. The terminal CPU 191 is configured to realize various functions by executing instructions (programs, routines) stored in the ROM. In particular, the terminal ECU 190 has installed therein a remote driving application (hereinafter, "remote driving app") for causing the ECU (vehicle ECU 290) of the vehicle 200 to execute the remote entry and exit control described later.

[0030] In addition, when the remote driving system 10 is configured as shown in Fig. 3, the remote driving system 10 may be configured to have a program stored in the server 301 take on some of the functions of the remote driving app installed in the terminal ECU 190. In addition, when the remote driving system 10 is configured as shown in Fig. 3, the remote driving system 10 may be configured to update the remote driving app installed in the terminal ECU 190 by the server 301.

[0031] The operation terminal 100 also includes a terminal display 120 , a vibration device 130 , an audio device 140 , and a terminal transmitting / receiving device 180 .

[0032] The terminal display 120 is a device that displays an image. The terminal display 120 is designed so that a specific physical property changes when an object touches the terminal display 120. In particular, in this example, the terminal display 120 is designed so that a specific physical property changes when a person's finger touches the terminal display 120.

[0033] The terminal display 120 is electrically connected to the terminal ECU 190. The terminal ECU 190 can display various images on the terminal display 120. In addition, the terminal ECU 190 can detect a change in a specific physical property of the terminal display 120 when an object comes into contact with the terminal display 120, and can identify the part of the terminal display 120 that the object has come into contact with based on the detected change.

[0034] In addition, since the terminal ECU 190 can identify the part of the terminal display 120 that has been touched by an object based on a change in a specific physical property of the terminal display 120, the terminal display 120 is a device that provides information for identifying the part that has been touched by an object (contact information providing device). In this example, the contact information providing device is the terminal display 120, but is not limited thereto, and may be any device that can provide information for identifying the part that has been touched by an object, for example, a device that does not have a function for displaying an image but can provide information for identifying the part that has been touched by an object.

[0035] The vibration device 130 is a device that generates vibration, and is, for example, a vibrator. The vibration device 130 is electrically connected to the terminal ECU 190. The terminal ECU 190 can vibrate the operation terminal 100 by activating the vibration device 130.

[0036] The audio device 140 is a device that outputs a notification sound such as a buzzer sound, for example, a buzzer. The audio device 140 is electrically connected to the terminal ECU 190. The terminal ECU 190 can output the notification sound via the audio device 140.

[0037] The terminal transceiver device 180 is a device that receives signals wirelessly from outside the operation terminal 100 and wirelessly transmits signals to outside the operation terminal 100. The terminal transceiver device 180 is electrically connected to the terminal ECU 190. The terminal ECU 190 can wirelessly transmit various signals to the outside of the operation terminal 100 via the terminal transceiver device 180. In addition, the terminal ECU 190 can receive various signals wirelessly transmitted to the outside of the vehicle 200 via the vehicle transceiver device 280 by the ECU (vehicle ECU 290) of the vehicle 200 via the terminal transceiver device 180.

[0038] In this way, the terminal control device 110 is configured to be able to communicate wirelessly with the vehicle control device 210 .

[0039] <Own vehicle> As shown in FIG. 5, the host vehicle 200 is equipped with a vehicle control device 210.

[0040] The vehicle control device 210 is a device that executes remote entry control and remote exit control as remote entry and exit control. The remote entry control is a control that parks the vehicle 200 in a designated parking space by automatically driving the vehicle 200 in response to a signal wirelessly transmitted to the outside from an operation terminal and automatically stopping the vehicle 200 at a point within the designated parking space. The remote exit control is a control that leaves the vehicle 200 from a designated parking space by automatically driving the vehicle 200 in response to a signal wirelessly transmitted to the outside from an operation terminal and automatically stopping the vehicle 200 at a point outside the designated parking space.

[0041] As shown in Fig. 5, the vehicle control device 210 includes a vehicle ECU 290. The vehicle ECU 290 includes a microcomputer as a main part. The vehicle ECU 290 includes a CPU (vehicle CPU 291), a ROM, a RAM, a non-volatile memory, an interface, and the like. The vehicle CPU 291 is adapted to realize various functions by executing instructions (programs, routines) stored in the ROM. In particular, the vehicle ECU 290 stores a program for executing remote entry and exit control.

[0042] In addition, when the remote traveling system 10 is configured as shown in Fig. 3, the remote traveling system 10 may be configured to have a program stored in the server 301 take on part of the processing carried out by the program that executes the remote entering and leaving control stored in the vehicle ECU 290. In addition, when the remote traveling system 10 is configured as shown in Fig. 3, the remote traveling system 10 may be configured so that the program that executes the remote entering and leaving control stored in the vehicle ECU 290 is updated by the server 301.

[0043] <Vehicle driving device> Furthermore, the host vehicle 200 is equipped with a vehicle driving device 220. The vehicle driving device 220 is a device that drives, brakes, steers, and changes gears of the host vehicle 200, and in this example, includes a drive device 221, a brake device 222, a steering device 223, and a transmission device 224.

[0044] <Drive unit> The drive device 221 is a device that outputs a drive force provided to the host vehicle 200 to make the host vehicle 200 run, and is, for example, an internal combustion engine and / or a motor. The drive device 221 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can control the drive force output from the drive device 221 by controlling the operation of the drive device 221.

[0045] <Brake device> The braking device 222 is a device that outputs a braking force applied to the host vehicle 200 in order to brake the host vehicle 200, and is, for example, a hydraulic brake device. The braking device 222 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can control the braking force output from the braking device 222 by controlling the operation of the braking device 222.

[0046] <Steering system> The steering device 223 is a device that outputs a steering force applied to the host vehicle 200 in order to steer the host vehicle 200, and is, for example, a power steering device. The steering device 223 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can control the steering force output from the steering device 223 by controlling the operation of the steering device 223.

[0047] <Transmission device> The transmission device 224 is a device that switches whether or not the driving force output from the drive device 221 is transmitted to the driving wheels of the host vehicle 200, and switches whether the driving force is transmitted to the driving wheels so as to move the host vehicle 200 forward or to the driving wheels so as to move the host vehicle 200 backward. Furthermore, the transmission device 224 is also a device that holds the host vehicle 200 in a stopped state by locking the gear so as not to rotate by engaging a claw-shaped part (parking lock pole) on the gear of the transmission device 224. Therefore, the transmission device 224 also functions as a stop holding device that holds the host vehicle 200 in a stopped state.

[0048] The transmission device 224 operates in one of the following states (shift states): a state in which the driving force is transmitted to the drive wheels to move the vehicle 200 forward (forward drive state SD), a state in which the driving force is transmitted to the drive wheels to move the vehicle 200 backward (reverse drive state SR), a state in which the driving force is not transmitted to the drive wheels of the vehicle 200 (neutral state SN), and a state in which the vehicle 200 is kept stopped (parking state SP).

[0049] The transmission device 224 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 controls the operation of the transmission device 224 to set the shift state of the transmission device 224 to any one of a forward drive state SD, a reverse drive state SR, a neutral state SN, and a parking state SP.

[0050] <Stopping holding device> The host vehicle 200 is also equipped with a vehicle stop maintaining device 230. The vehicle stop maintaining device 230 is a device that maintains the host vehicle 200 in a stopped state, and in this example, is a parking brake device 231. The parking brake device 231 may be an electric parking brake device or a manual parking brake device. The parking brake device 231 is a device that maintains the host vehicle 200 in a stopped state by applying a braking force to the wheels of the host vehicle 200 that is stopped. In particular, the parking brake device 231 is a device that maintains the host vehicle 200 in a stopped state by applying a braking force to the wheels by pressing a brake pad against a brake disc provided on the wheels of the host vehicle 200 that is stopped. In this example, the parking brake device 231 is an electric parking brake device, and is therefore electrically connected to the vehicle ECU 290. The vehicle ECU 290 can maintain the host vehicle 200 in a stopped state by operating the parking brake device 231.

[0051] <Sensors, etc.> Furthermore, the vehicle 200 is equipped with an accelerator pedal 241, an accelerator pedal operation amount sensor 242, a brake pedal 243, a brake pedal operation amount sensor 244, a steering wheel 245, a steering shaft 246, a steering angle sensor 247, a steering torque sensor 248, a remote entry / exit request operator 249, a vehicle speed detection device 250, a shift lever 251, a shift sensor 252, a surrounding information detection device 260, and a vehicle receiving / transmitting device 280.

[0052] <Accelerator pedal operation amount sensor> Accelerator pedal operation amount sensor 242 is a sensor that detects the amount of operation of accelerator pedal 241. Accelerator pedal operation amount sensor 242 is electrically connected to vehicle ECU 290. Accelerator pedal operation amount sensor 242 transmits information on the detected amount of operation of accelerator pedal 241 to vehicle ECU 290. Vehicle ECU 290 acquires the amount of operation of accelerator pedal 241 as accelerator pedal operation amount AP based on that information.

[0053] Except when executing the remote entering / leaving control described later, the vehicle ECU 290 calculates and obtains a required driving force (required driving torque) based on the accelerator pedal operation amount AP and the traveling speed (host vehicle speed) of the host vehicle 200. The vehicle ECU 290 controls the operation of the drive device 221 so that the required driving force is output. Furthermore, when executing the remote entering / leaving control described later, the vehicle ECU 290 determines a driving force required to run the host vehicle 200 as desired by the remote entering / leaving control, and controls the operation of the drive device 221 so that the driving force is output.

[0054] <Brake pedal operation amount sensor> Brake pedal operation amount sensor 244 is a sensor that detects the operation amount of brake pedal 243. Brake pedal operation amount sensor 244 is electrically connected to vehicle ECU 290. Brake pedal operation amount sensor 244 transmits information on the detected operation amount of brake pedal 243 to vehicle ECU 290. Vehicle ECU 290 obtains the operation amount of brake pedal 243 as brake pedal operation amount BP based on the information.

[0055] Except when executing the remote entering / leaving control described later, the vehicle ECU 290 acquires a required braking force (required braking torque) by calculation from the brake pedal operation amount BP. The vehicle ECU 290 controls the operation of the braking device 222 so that the required braking force is output. When executing the remote entering / leaving control described later, the vehicle ECU 290 determines a braking force required to brake the host vehicle 200 as desired by the remote entering / leaving control, and controls the operation of the braking device 222 so that the braking force is output.

[0056] <Steering angle sensor> Steering angle sensor 247 is a sensor that detects the rotation angle of steering shaft 246 with respect to the neutral position, and is electrically connected to vehicle ECU 290. Steering angle sensor 247 transmits information on the detected rotation angle of steering shaft 246 to vehicle ECU 290. Vehicle ECU 290 acquires the rotation angle of steering shaft 246 as steering angle θ based on the information.

[0057] <Steering torque sensor> The steering torque sensor 248 is a sensor that detects the torque input by the driver DR of the vehicle 200 to the steering shaft 246 via the steering wheel 245, and is electrically connected to the vehicle ECU 290. The steering torque sensor 248 transmits information on the detected torque to the vehicle ECU 290. The vehicle ECU 290 acquires the torque input by the driver DR to the steering shaft 246 via the steering wheel 245 (driver input torque) based on the information.

[0058] Except when executing the remote entering / leaving control described later, the vehicle ECU 290 obtains a required steering force (required steering torque) based on the steering angle θ, the driver input torque, and the traveling speed (host vehicle speed) of the host vehicle 200, and controls the operation of the steering device 223 so that the required steering torque is output from the steering device 223. Furthermore, when executing the remote entering / leaving control described later, the vehicle ECU 290 determines a steering force required to make the host vehicle 200 travel as desired by the remote entering / leaving control, and controls the operation of the steering device 223 so that the steering force is output.

[0059] <Remote Entry / Exit Request Operator> The remote entering / leaving request operation device 249 is a device, such as a switch, operated by the driver DR to request the vehicle ECU 290 to execute remote entering / leaving control, which will be described later. The remote entering / leaving request operation device 249 is electrically connected to the vehicle ECU 290. When operated by the driver DR, the remote entering / leaving request operation device 249 transmits a predetermined signal to the vehicle ECU 290. When the vehicle ECU 290 receives the predetermined signal, it determines that execution of remote entering / leaving control has been requested.

[0060] <Vehicle speed detection device> The vehicle speed detection device 250 is a device that detects the traveling speed of the host vehicle 200, and is, for example, a wheel speed sensor. The vehicle speed detection device 250 is electrically connected to the vehicle ECU 290. The vehicle speed detection device 250 transmits information on the detected traveling speed of the host vehicle 200 to the vehicle ECU 290. The vehicle ECU 290 acquires the traveling speed of the host vehicle 200 (host vehicle speed V) based on the information.

[0061] <Shift sensor> The shift sensor 252 is a sensor that detects the setting position of the shift lever 251. The shift lever 251 is a device operated by the driver DR of the vehicle 200, and the setting positions of the shift lever 251 that the driver DR can set are a forward position (drive range D), a reverse position (rearrange R), a neutral position (neutral range N), and a parking position (parking range P). The shift sensor 252 is electrically connected to the vehicle ECU 290. The shift sensor 252 transmits a signal indicating the detected setting position of the shift lever 251 to the vehicle ECU 290.

[0062] When the shift lever 251 is set to drive range D, the shift sensor 252 transmits a signal indicating that the set position of the shift lever 251 is drive range D to the vehicle ECU 290. Upon receiving the signal, the vehicle ECU 290 controls the shift state of the transmission device 224 so that the transmission device 224 is in a forward drive state SD.

[0063] Furthermore, when the shift lever 251 is set to the rear range R, the shift sensor 252 transmits a signal indicating that the set position of the shift lever 251 is the rear range R to the vehicle ECU 290. Upon receiving the signal, the vehicle ECU 290 controls the shift state of the transmission device 224 so that the transmission device 224 is in the reverse drive state SR.

[0064] Furthermore, when the shift lever 251 is set to the neutral range N, the shift sensor 252 transmits a signal indicating that the set position of the shift lever 251 is the neutral range N to the vehicle ECU 290. Upon receiving the signal, the vehicle ECU 290 controls the shift state of the transmission device 224 so that the transmission device 224 is in the neutral state SN.

[0065] Furthermore, when the shift lever 251 is set to the parking range P, the shift sensor 252 transmits a signal indicating that the set position of the shift lever 251 is the parking range P to the vehicle ECU 290. Upon receiving this signal, the vehicle ECU 290 controls the shift state of the transmission device 224 so that the transmission device 224 is in the parking state SP.

[0066] When executing the remote entering and leaving control described later, the vehicle ECU 290 controls the shift state of the transmission device 224 (performs a shift change) as necessary to make the host vehicle 200 travel as desired through the remote entering and leaving control.

[0067] <Peripheral information detection device> The surrounding information detection device 260 is a device that detects information about the surroundings of the vehicle 200, and in this example, includes a radio wave sensor 261 and an image sensor 262.

[0068] <Radio wave sensor> The radio wave sensor 261 is a sensor that detects information about an object present in the vicinity of the vehicle 200 using radio waves, and is, for example, at least one of a radar sensor (such as a millimeter wave radar), a sonic sensor such as an ultrasonic sensor (clearance sonar), and an optical sensor such as a laser radar (LiDAR). The radio wave sensor 261 is electrically connected to the vehicle ECU 290. The radio wave sensor 261 transmits radio waves and receives radio waves (reflected waves) reflected by an object. The radio wave sensor 261 transmits information related to the transmitted radio waves and the received radio waves (reflected waves) to the vehicle ECU 290. In other words, the radio wave sensor 261 detects an object present in the vicinity of the vehicle 200 and transmits information related to the detected object to the vehicle ECU 290. The vehicle ECU 290 can obtain information (periphery detection information IS) related to an object present in the vicinity of the vehicle 200 based on the information (radio wave information IR or radio wave data). Objects detected using the radio wave sensor 261 are, for example, vehicles, walls, bicycles, people, and the like.

[0069] <Image sensor> The image sensor 262 is a sensor, such as a camera, that captures an image of the periphery of the host vehicle 200. The image sensor 262 is electrically connected to the vehicle ECU 290. The image sensor 262 captures an image of the periphery of the host vehicle 200 and transmits information related to the captured image to the vehicle ECU 290.

[0070] <Vehicle Transmitting and Receiving Device> The vehicle receiving / transmitting device 280 is a device that receives a signal wirelessly from outside the vehicle 200 and wirelessly transmits a signal to the outside of the vehicle 200. The vehicle receiving / transmitting device 280 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can wirelessly transmit various signals to the outside of the vehicle 200 via the vehicle receiving / transmitting device 280. In addition, the vehicle ECU 290 can receive various signals wirelessly transmitted to the outside of the operation terminal 100 via the terminal receiving / transmitting device 180 by a terminal ECU 190 described later via the vehicle receiving / transmitting device 280.

[0071] <Outline of operation of the remote driving system> Next, an overview of the operation of the remote driving system 10 will be described.

[0072] When the vehicle 200 is stopped and a request is made to execute remote entry / exit control, the vehicle control device 210 receives a wireless signal from the outside and enters a state in which it determines whether the wireless signal is a wireless signal transmitted from a registered operation terminal. The registered operation terminal is an operation terminal registered in the vehicle control device 210 as an operation terminal that causes the vehicle control device 210 to execute remote entry / exit control.

[0073] When the driver DR leaves his / her vehicle 200 and performs an application startup operation on the terminal display 120 of the operating terminal 100, the terminal control device 110 responds to the application startup operation by starting a remote driving application and displays an initial screen image G10 on the terminal display 120, as shown in FIG. 7.

[0074] The application start operation is a touch operation on the terminal display 120 for starting the remote driving application. In this example, the touch operation on the terminal display 120 is an operation (contact operation) in which a person touches the terminal display 120 with a finger. The initial screen image G10 includes a remote entry / exit start image G11 and an application end image G15. The remote entry / exit start image G11 is an image that displays the text "Remote entry / exit start". The application end image G15 is an image that displays the text "End". The remote entry / exit start image portion P11 and the application end image portion P15 are portions that accept touch operations by a user (terminal user UR) of the operation terminal 100. The remote entry / exit start image portion P11 is a portion of the terminal display 120 that displays the remote entry / exit start image G11. The application end image portion P15 is a portion of the terminal display 120 that displays the application end image G15.

[0075] When the terminal control device 110 starts the remote driving application, it wirelessly transmits an application start signal S10 to the outside. The application start signal S10 is a signal representing information such as an ID for determining whether the operation terminal 100 is a registered operation terminal (for identifying the operation terminal 100).

[0076] When the vehicle control device 210 receives the application start signal S10, it judges whether or not the operation terminal 100 indicated by the application start signal S10 is a registered operation terminal. A registered operation terminal is an operation terminal registered as an operation terminal that causes the vehicle control device 210 to perform remote entry and exit control. Since the operation terminal 100 is a registered operation terminal, when the vehicle control device 210 receives the application start signal S10, it judges that the operation terminal 100 is a registered operation terminal. In this case, the vehicle control device 210 thereafter performs remote entry and exit control based on various signals wirelessly transmitted from the operation terminal 100 to the outside.

[0077] When the vehicle control device 210 determines that the operation terminal 100 is a registered operation terminal, it operates vehicle driving devices such as the surrounding information detection device 260 and the vehicle driving device 220 or makes preparations for operating the vehicle driving devices.

[0078] The vehicle control device 210 may be configured to determine the type of executable assistance based on the surrounding detection information IS when it is determined that the operation terminal 100 is a registered operation terminal, and to wirelessly transmit the determination result to the outside. For example, as described later, the vehicle control device 210 is configured to be able to execute the remote entry control and the remote exit control, but when it is not possible to execute the remote exit control due to the surrounding conditions of the vehicle 200 and only the remote entry control is executable, it may be configured to wirelessly transmit a signal indicating that only the remote entry control is executable to the outside. In this case, when the terminal control device 110 receives the signal, it adjusts the image displayed on the terminal display 120 so that only the remote entry control can be selected.

[0079] Furthermore, the remote driving system 10 described below is a system that can select only control for moving the vehicle 200 forward to park it in parallel in the designated parking space 31D as a control for entering the designated parking space 31D, but in addition to that control, the system may also be capable of selecting control for moving the vehicle 200 forward to park it in parallel in the designated parking space 31D, control for moving the vehicle 200 forward to park it in parallel in the designated parking space 31D, and control for moving the vehicle 200 forward to park it in parallel in the designated parking space 31D.

[0080] When a touch operation is applied to the remote entry / exit start image portion P11, the terminal control device 110 displays an entry / exit selection image G20 on the terminal display 120, as shown in Fig. 8. In this example, a valid touch operation that can be applied to the remote entry / exit start image portion P11 is a swipe operation. A swipe operation is one of slide operations, and is one of touch operations (sliding touch operations) in which a finger touching the terminal display 120 is linearly slid in the same direction on the terminal display 120.

[0081] The entry / exit selection image G20 includes an entry selection image G21, an exit selection image G22, and an application exit image G25. The entry selection image G21 is an image displaying the text "Entry". The exit selection image G22 is an image displaying the text "Exit". The application exit image G25 is an image displaying the text "Exit". The entry selection image portion P21, the exit selection image portion P22, and the application exit image portion P25 are portions that accept touch operations by the terminal user UR. The entry selection image portion P21 is a portion of the terminal display 120 that displays the entry selection image G21. The exit selection image portion P22 is a portion of the terminal display 120 that displays the exit selection image G22. The application exit image portion P25 is a portion of the terminal display 120 that displays the application exit image G25.

[0082] <Remote Stock Control> When a touch operation is applied to the warehousing selection image portion P21, the terminal control device 110 wirelessly transmits a warehousing selection signal S11 to the outside. The warehousing selection signal S11 is a signal indicating that a touch operation has been applied to the warehousing selection image portion P21. In this example, a valid touch operation to be applied to the warehousing selection image portion P21 is a tap operation. A tap operation is one of the touch operations in which a finger is touched to the terminal display 120 and then immediately released.

[0083] When the vehicle control device 210 receives the entrance selection signal S11, it detects a section (parking section 31C) where the vehicle 200 can be parked based on the surrounding detection information IS. For example, as shown in FIG. 6, when the vehicle 200 is stopped near three parking sections 311 to 313 where no other vehicle is parked, the vehicle control device 210 detects these parking sections 311 to 313 as parking sections 311C to 313C. The example shown in FIG. 6 is an example where the parking sections 311 to 313 are present on the left side of the vehicle 200, but when one or more parking sections are present on the right side of the vehicle 200, the vehicle control device 210 also detects these parking sections as parking sections. Note that the reference numeral 400 in the figure indicates a section line dividing the parking section 31.

[0084] When the vehicle control device 210 detects the parking space 31C, it acquires the position of the vehicle 200 relative to the detected parking space 31C.

[0085] When the vehicle control device 210 detects the parking space 31C and acquires the position of the vehicle 200 relative to the parking space 31C, the vehicle control device 210 wirelessly transmits a designated parking space signal S12 and a vehicle position signal S13 to the outside. The designated parking space signal S12 is one of the signals that indicate information about the driving state of the vehicle 200, and is a signal that represents information about the parking space 31C. The vehicle position signal S13 is one of the signals that indicate information about the driving state of the vehicle 200, and is a signal that represents information about the position of the vehicle 200 relative to the parking space 31C, and is a signal that represents information about the vehicle 200.

[0086] When the terminal control device 110 receives the designated parking space signal S12 and the vehicle position signal S13, it displays a parking space selection image G30 on the terminal display 120, as shown in Fig. 9. The parking space selection image G30 includes an available parking space image G31, a vehicle image G32, a selection confirmation image G34, an application termination image G35, and an entry / exit reselection image G36.

[0087] The available parking space image G31 is an image that displays the available parking space 31C. Since FIG. 9 shows an example corresponding to the example shown in FIG. 6, the available parking space image G31 shown in FIG. 9 includes available parking space images G311 to G313 that display the available parking spaces 311C to 313C, respectively. The vehicle image G32 is an image that displays the vehicle 200. The selection confirmation image G34 is an image that displays the word "confirmed." The application termination image G35 is an image that displays the word "end." The entry / exit reselection image G36 is an image that displays the word "entry / exit reselection."

[0088] Also, the available parking space image portion P31, the selection confirmation image portion P34, the application termination image portion P35, and the entrance / exit reselection image portion P36 are portions that accept touch operations by the terminal user UR. The available parking space image portion P31 is a portion of the terminal display 120 that displays the available parking space image G31. Since FIG. 9 shows an example corresponding to the example shown in FIG. 6, the available parking space image portion P31 shown in FIG. 9 includes three portions of the terminal display 120 (available parking space image portions P311 to P313) that respectively display the three available parking space images G311 to G313. The selection confirmation image portion P34 is a portion of the terminal display 120 that displays the selection confirmation image G34. The application termination image portion P35 is a portion of the terminal display 120 that displays the application termination image G35. The entrance / exit reselection image portion P36 is a portion of the terminal display 120 that displays the entrance / exit reselection image G36.

[0089] When a touch operation is applied to the parking space image portion P31 and then a touch operation is applied to the selection confirmation image portion P34, the terminal control device 110 wirelessly transmits a designated parking space signal S14 (driving type confirmation signal) to the outside. In the example shown in FIG. 9, when a touch operation is applied to any of the three parking space image portions P311 to P313 and then a touch operation is applied to the selection confirmation image portion P34, the terminal control device 110 wirelessly transmits a designated parking space signal S14 to the outside. The designated parking space signal S14 is a signal indicating the parking space 31C designated by the terminal user UR as a space for parking the vehicle 200 by applying a touch operation to the parking space image portion P31. The designated parking space signal S14 is also a driving type confirmation signal that confirms the driving type of the vehicle 200 by the vehicle driving process described later.

[0090] In this example, a valid touch operation applied to the parking space image portion P31 is a long press operation. The long press operation can also be called a long tap operation, and is one of the touch operations in which a finger continues to touch the terminal display 120 for a relatively long fixed period of time.

[0091] The parking section selection image G30 may also include a parking direction selection image for selecting whether to park the vehicle 200 forward or backward into the designated parking section 31D. In this case, when a touch operation is performed on the available parking section image portion P31 and on the parking direction selection image, and then a touch operation is performed on the selection confirmation image portion P34, the terminal control device 110 wirelessly transmits the designated parking section signal S14 to the outside.

[0092] When the vehicle control device 210 receives the designated parking section signal S14, it sets the available parking section 31C designated by the terminal user UR as the designated parking section 31D. In the example shown in Fig. 6, when the vehicle control device 210 receives the designated parking section signal S14, it sets the available parking section 31C (312C) designated by the terminal user UR from among the available parking sections 311C to 313C as the designated parking section 31D.

[0093] When the designated parking section 31D is set, the vehicle control device 210 starts remote entry control. When the remote entry control starts, the vehicle control device 210 sets a target entry route Rin_tgt as shown in Fig. 10. The target entry route Rin_tgt is a route along which the host vehicle 200 travels in order to enter (park) the host vehicle 200 in the designated parking section 31D. Note that Fig. 10 shows an example in which the parking available section 312C is set as the designated parking section 31D in the example shown in Fig. 6.

[0094] When the target entrance route Rin_tgt is set, the vehicle control device 210 wirelessly transmits an entrance remaining distance signal S15 to the outside. The entrance remaining distance signal S15 is one of the signals indicating information related to the traveling state of the vehicle 200, and is a signal indicating the distance (entry remaining distance Din) between the vehicle 200 and a point (target parking point Pin_tgt) in the designated parking section 31D to which the vehicle 200 should reach by remote entrance control. The vehicle control device 210 acquires the entrance remaining distance Din based on the surrounding detection information IS.

[0095] On the other hand, when a touch operation is performed on the available parking section image portion P31, the terminal control device 110 displays an entry operation image G40 on the terminal display 120 as shown in Fig. 11. Fig. 11 shows an example corresponding to the example shown in Fig. 10, in which the designated parking section 31D is the available parking section 31C (312C) on the left side of the vehicle 200.

[0096] The entry operation image G40 includes a designated parking space image G41, a vehicle image G42, a remaining entry distance image G43, an entry travel operation image G44, an application termination image G45, and an entry / exit reselection image G46. The designated parking space image G41 is an image that displays the designated parking space 31D. The vehicle image G42 is an image that displays the vehicle 200. The remaining entry distance image G43 is an image that displays the remaining entry distance Din. The entry travel operation image G44 is an image that defines an area of ​​a predetermined size. The application termination image G45 is an image that displays the word "END". The entry / exit reselection image G46 is an image that displays the word "RE-SELECT ENTRY / EXIT". In addition, the entry travel operation image portion P44, the application termination image portion P45, and the entry / exit reselection image portion P46 are portions that accept touch operations by the terminal user UR. The entering / leaving driving operation image portion P44 is a portion of the terminal display 120 displaying the entering / leaving driving operation image G44. The application termination image portion P45 is a portion of the terminal display 120 displaying the application termination image G45. The entering / leaving reselection image portion P46 is a portion of the terminal display 120 displaying the entering / leaving reselection image G46.

[0097] When the terminal control device 110 receives the remaining entry distance signal S15, it displays on the terminal display 120 a remaining entry distance image G43 that displays the remaining entry distance Din indicated by the remaining entry distance signal S15.

[0098] The terminal control device 110 wirelessly transmits a continuous touch operation signal S16 (predetermined operation signal) to the outside while a predetermined operation is being applied to the entrance travel operation image portion P44. The continuous touch operation signal S16 is a signal indicating that a predetermined operation has been applied to the entrance travel operation image portion P44. In this example, the predetermined operation applied to the entrance travel operation image portion P44 is a touch operation, and a valid touch operation as the touch operation is a continuous touch operation. The continuous touch operation is one of slide operations, and is an operation in which the terminal user UR touches the entrance travel operation image portion P44 with his / her finger and slides (slides) the finger over the entrance travel operation image portion P44 as shown by the line L in FIG. 12. Therefore, when the finger of the terminal user UR is touching the entrance driving operation image portion P44 and performing a touch operation, but the finger is stopped on the entrance driving operation image portion P44 and a continuous touch operation is not being performed, the terminal control device 110 does not transmit the continuous touch operation signal S16.

[0099] The terminal control device 110 may be configured to wirelessly transmit to the outside a signal indicating the position of the entrance travel operation image portion P44 where the touch operation is being performed, instead of the continuous touch operation signal S16. In this case, the vehicle control device 210 that receives the signal determines whether or not the continuous touch operation is being performed on the entrance travel operation image portion P44.

[0100] When the vehicle control device 210 receives the continuous touch operation signal S16, it controls the operation of the transmission device 224 so that the transmission device 224 is in the forward drive state SD, and then executes vehicle driving processing to control the operation of the vehicle driving device 220 so that the vehicle 200 moves forward while turning right along the target entrance route Rin_tgt, as shown in Figure 13, and stops when it has moved forward a predetermined distance.

[0101] While receiving the successive touch operation signal S16, the vehicle control device 210 executes a vehicle driving process so that the host vehicle 200 drives along the target entrance route Rin_tgt.

[0102] In addition, the vehicle control device 210 continues to wirelessly transmit the vehicle position signal S13, the remaining entry distance signal S15, and the traveling direction signal S17 to the outside from the start of receiving the continuous touch operation signal S16 until parking of the vehicle 200 in the designated parking space 31D is completed. The traveling direction signal S17 is one of the signals that indicate information regarding the traveling state of the vehicle 200, and is a signal that indicates the traveling direction of the vehicle 200.

[0103] When the terminal control device 110 receives the vehicle position signal S13, it displays a vehicle image G42 in a portion of the terminal display 120 that corresponds to the position of the host vehicle 200 relative to the designated parking section 31D indicated by the vehicle position signal S13, as shown in Fig. 14. The vehicle image G42 shown in Fig. 14 is displayed when the host vehicle 200 has traveled to the position shown in Fig. 13.

[0104] Furthermore, when the terminal control device 110 receives the remaining entry distance signal S15, it displays on the terminal display 120, as shown in FIG. 14, a remaining entry distance image G43 indicating the remaining entry distance Din indicated by the remaining entry distance signal S15.

[0105] In addition, when the terminal control device 110 receives the travel direction signal S17, it displays a travel direction image G47 indicating the travel direction of the vehicle 200 indicated by the travel direction signal S17 in a portion of the terminal display 120 near the vehicle image G42, as shown in FIG. 14.

[0106] When the terminal user UR removes his / her finger from the entrance travel operation image portion P44, the terminal control device 110 stops wirelessly transmitting the continuous touch operation signal S16 to the outside. In this case, the vehicle control device 210 stops receiving the continuous touch operation signal S16. When the vehicle control device 210 stops receiving the continuous touch operation signal S16, it executes a stopping process to temporarily stop the host vehicle 200 by the braking device 222.

[0107] As shown in FIG. 13, the vehicle control device 210 causes the vehicle 200 to move forward while turning right and stop, then changes the shift state of the transmission device 224 from the forward drive state SD to the reverse drive state SR in order to turn the vehicle 200 around, and then, as shown in FIG. 15, executes vehicle driving processing to control the operation of the vehicle driving device 220 so that the steering angle θ becomes zero when the fore-and-aft direction of the vehicle 200 becomes parallel to the longitudinal direction of the designated parking space 31D.

[0108] Thereafter, the vehicle control device 210 executes a vehicle driving process for controlling the operation of the vehicle driving device 220 so as to move the host vehicle 200 backward in a straight line, as shown in FIG.

[0109] In this example, the vehicle control device 210 controls the driving force output from the driving device 221 and the braking force output from the braking device 222 so that the vehicle speed V is equal to or less than a predetermined limit vehicle speed Vlimit while the vehicle 200 is traveling under remote entry control.

[0110] 17, when the vehicle 200 reaches the target parking point Pin_tgt, the vehicle control device 210 stops the vehicle 200 by a stop process, and further holds the vehicle 200 in a stopped state by a stop holding process, stops the operation of the surrounding information detection device 260, the vehicle driving device 220, and the like, and ends the remote entry control. This completes the entry (parking) of the vehicle 200 into the designated parking section 31D.

[0111] In this example, the vehicle stop process is a process for stopping the host vehicle 200 by the braking device 222. The vehicle stop maintenance process is a process for maintaining the host vehicle 200 in a stopped state by operating the parking brake device 231, or for maintaining the host vehicle 200 in a stopped state by controlling the operation of the transmission device 224 so that the transmission device 224 is in the parking state SP.

[0112] Furthermore, when the entry (parking) of the vehicle 200 is completed, the vehicle control device 210 wirelessly transmits an entry completion signal S18 to the outside. The entry completion signal S18 is a signal indicating that the entry (parking) of the vehicle 200 is completed.

[0113] Furthermore, if the vehicle control device 210 detects an obstacle based on the surrounding detection information IS while driving the vehicle 200 under remote entry control, and it becomes necessary to stop the vehicle 200 because the vehicle 200 is about to come into contact with the obstacle, the vehicle control device 210 stops the vehicle 200 by performing a stopping process, sets a new target entry route Rin_tgt (resets the target entry route Rin_tgt), and resumes driving the vehicle 200.

[0114] When the terminal control device 110 receives the entry completion signal S18, as shown in Fig. 18, it displays an entry completion image G49 on the terminal display 120. The entry completion image G49 is an image indicating that the entry of the host vehicle 200 has been completed.

[0115] <Remote shipping control> When a touch operation is applied to the output selection image portion P22, the terminal control device 110 wirelessly transmits an output selection signal S21 to the outside. The output selection signal S21 is a signal indicating that a touch operation is applied to the output selection image portion P22. In this example, the touch operation adopted as a valid touch operation applied to the output selection image portion P22 is a tap operation.

[0116] When the vehicle control device 210 receives the exit selection signal S21, the vehicle control device 210 acquires the position of the host vehicle 200 relative to the parking space in which the host vehicle 200 is parked (current parking space 31N) based on the surrounding detection information IS. For example, when the host vehicle 200 is parked as shown in Fig. 19, the parking space 31 is the current parking space 31N, and the vehicle control device 210 acquires the position of the host vehicle 200 relative to the current parking space 31N.

[0117] When the vehicle control device 210 acquires the position of the vehicle 200 relative to the current parking section 31N, the vehicle control device 210 wirelessly transmits a vehicle position signal S23 to the outside. The vehicle position signal S23 is a signal that represents information regarding the position of the vehicle 200 relative to the current parking section 31N.

[0118] When the terminal control device 110 receives the vehicle position signal S23, as shown in Fig. 20, it displays an exit direction selection image G50 on the terminal display 120. The exit direction selection image G50 includes a current parking space image G51, a vehicle image G52, a possible exit direction image G53, a selection confirmation image G54, an application termination image G55, and an entry / exit reselection image G56.

[0119] The current parking space image G51 is an image that displays the current parking space 31N. The vehicle image G52 is an image that displays the vehicle 200. The possible exit direction image G53 is an image that displays the direction in which the vehicle 200 can be exited (exit direction). Since FIG. 20 shows an example corresponding to the example shown in FIG. 19, the possible exit direction image G53 shown in FIG. 20 includes an image that displays the forward direction (forward direction image G531) and an image that displays the reverse direction (reverse direction image G532). The selection confirmation image G54 is an image that displays the word "Confirmed". The application termination image G55 is an image that displays the word "End". The entry / exit reselection image G56 is an image that displays the word "Entry / Exit Reselection".

[0120] Also, the available exit direction image portion P53, the selection confirmation image portion P54, the application termination image portion P55, and the entrance / exit reselection image portion P56 are portions that accept touch operations by the terminal user UR. The available exit direction image portion P53 is a portion of the terminal display 120 that displays the available exit direction image G53. Since FIG. 20 shows an example corresponding to the example shown in FIG. 19, the available exit direction image portion P53 shown in FIG. 20 includes two portions of the terminal display 120 that display the forward direction image G531 and the backward direction image G532, respectively (the forward direction image portion P531 and the backward direction image portion P532). The selection confirmation image portion P54 is a portion of the terminal display 120 that displays the selection confirmation image G54. The application termination image portion P55 is a portion of the terminal display 120 that displays the application termination image G55. The entrance / exit reselection image portion P56 is a portion of the terminal display 120 that displays the entrance / exit reselection image G56.

[0121] When a touch operation is applied to the possible exit direction image portion P53 and then a touch operation is applied to the selection confirmation image portion P54, the terminal control device 110 wirelessly transmits the designated exit direction signal S22 to the outside. In the example shown in FIG. 20, when a touch operation is applied to either the forward direction image portion P531 or the backward direction image portion P532 and then a touch operation is applied to the selection confirmation image portion P54, the terminal control device 110 wirelessly transmits the designated exit direction signal S22 to the outside. The designated exit direction signal S22 is a signal representing the possible exit direction designated by the terminal user UR as the direction for exiting the vehicle 200 by applying a touch operation to the possible exit direction image portion P53. The designated exit direction signal S22 is also a driving type confirmation signal that confirms the driving type of the vehicle 200 by the vehicle driving process described later.

[0122] In this example, a valid touch operation that can be applied to the available exit direction image portion P53 is a long press operation.

[0123] When the vehicle control device 210 receives the designated leaving direction signal S22, it sets the possible leaving direction designated by the terminal user UR as the designated leaving direction. In the example shown in Fig. 19, when the vehicle control device 210 receives the designated leaving direction signal S22, it sets the possible leaving direction (forward direction) designated by the terminal user UR from among the forward direction and the backward direction as the designated leaving direction.

[0124] When the designated leaving direction is set, the vehicle control device 210 starts remote leaving control. When the remote leaving control is started, the vehicle control device 210 sets a target leaving route Rout_tgt as shown in Fig. 21. The target leaving route Rout_tgt is a route along which the host vehicle 200 travels in order to leave the host vehicle 200 from the current parking space 31N by traveling in the designated leaving direction. Fig. 21 shows an example in which the forward direction is set as the designated leaving direction in the example shown in Fig. 19.

[0125] When the vehicle control device 210 sets the target exit route Rout_tgt, it wirelessly transmits a remaining exit distance signal S25 to the outside. The remaining exit distance signal S25 is a signal that indicates the distance (remaining exit distance Dout) between the vehicle 200 and a point (target exit point Pout_tgt) outside the current parking section 31N to which the vehicle 200 should arrive by remote exit control. The vehicle control device 210 acquires the remaining exit distance Dout based on the surrounding detection information IS.

[0126] On the other hand, when a touch operation is performed on the available leaving direction image portion P53, the terminal control device 110 displays a leaving operation image G60 on the terminal display 120 as shown in Fig. 22. Fig. 22 shows an example corresponding to the example shown in Fig. 21, in which the designated leaving direction is the forward direction.

[0127] The leaving operation image G60 includes a current parking space image G61, a vehicle image G62, a remaining leaving distance image G63, a leaving driving operation image G64, an application termination image G65, an entry / exit reselection image G66, and an exit direction image G67. The current parking space image G61 is an image that displays the current parking space 31N. The vehicle image G62 is an image that displays the vehicle 200. The remaining leaving distance image G63 is an image that displays the remaining leaving distance Dout. The leaving driving operation image G64 is an image that defines an area of ​​a specified size. The application termination image G65 is an image that displays the text "End". The entry / exit reselection image G66 is an image that displays the text "Entry / exit reselection". The exit direction image G67 is an image that displays the designated leaving direction.

[0128] Also, the leaving travel operation image portion P64, the application termination image portion P65, and the entering / leaving reselection image portion P66 are portions that accept touch operations by the terminal user UR. The leaving travel operation image portion P64 is a portion of the terminal display 120 that displays the leaving travel operation image G64. The application termination image portion P65 is a portion of the terminal display 120 that displays the application termination image G65. The entering / leaving reselection image portion P66 is a portion of the terminal display 120 that displays the entering / leaving reselection image G66.

[0129] When the terminal control device 110 receives the remaining distance from the garage signal S25, it displays on the terminal display 120 a remaining distance from the garage image G63 indicating the remaining distance from the garage Dout indicated by the remaining distance from the garage signal S25.

[0130] Then, the terminal control device 110 wirelessly transmits a continuous touch operation signal S26 (predetermined operation signal) to the outside while a touch operation is being applied to the leaving travel operation image portion P64. The continuous touch operation signal S26 is a signal indicating that a predetermined operation has been applied to the leaving travel operation image portion P64. In this example, the predetermined operation applied to the leaving travel operation image portion P64 is a touch operation, and a valid touch operation as the touch operation is a continuous touch operation. The continuous touch operation is one of slide operations, and is an operation in which the terminal user UR touches the leaving travel operation image portion P64 with his / her finger and slides (slides) the finger on the leaving travel operation image portion P64 as shown by the line L in FIG. 23. Therefore, when the finger of the terminal user UR touches the leaving travel operation image portion P64 to perform a touch operation, but the finger stops on the leaving travel operation image portion P64 and a continuous touch operation is not performed, the terminal control device 110 does not transmit the continuous touch operation signal S26.

[0131] The terminal control device 110 may be configured to wirelessly transmit to the outside a signal indicating the position of the leaving travel operation image portion P64 where the touch operation is being performed, instead of the continuous touch operation signal S26. In this case, the vehicle control device 210 that receives the signal determines whether or not the leaving travel operation image portion P64 is being subjected to a continuous touch operation.

[0132] When the vehicle control device 210 receives the continuous touch operation signal S26, it sets the shift state of the transmission device 224 to the forward drive state SD, and then, as shown in FIG. 24, executes a vehicle driving process that controls the operation of the vehicle driving device 220 so as to move the vehicle 200 forward along the target departure route Rout_tgt and stop the vehicle 200 when the vehicle 200 reaches the target departure point Pout_tgt.

[0133] While receiving the successive touch operation signal S26, the vehicle control device 210 executes a vehicle driving process so that the host vehicle 200 drives along the target leaving route Rout_tgt.

[0134] In addition, the vehicle control device 210 continues to wirelessly transmit the vehicle position signal S23 and the remaining distance to exit signal S25 to the outside from the time when it starts to receive the continuous touch operation signal S26 until the exit of the vehicle 200 from the current parking space 31N is completed.

[0135] When the terminal control device 110 receives the vehicle position signal S23, it displays a vehicle image G62 in a portion of the terminal display 120 corresponding to the position of the host vehicle 200 relative to the current parking space 31N indicated by the vehicle position signal S23, as shown in Fig. 23. The vehicle image G62 shown in Fig. 23 is displayed when the host vehicle 200 has traveled to the position shown in Fig. 24.

[0136] Furthermore, when the terminal control device 110 receives the remaining distance to leave signal S25, it displays on the terminal display 120 a remaining distance to leave image G63 indicating the remaining distance to leave Dout indicated by the remaining distance to leave signal S25, as shown in FIG.

[0137] When the terminal user UR removes his / her finger from the leaving vehicle travel operation image portion P64, the terminal control device 110 stops wirelessly transmitting the continuous touch operation signal S26 to the outside. In this case, the vehicle control device 210 stops receiving the continuous touch operation signal S26. When the vehicle control device 210 stops receiving the continuous touch operation signal S26, it executes a stopping process to temporarily stop the host vehicle 200 by the braking device 222.

[0138] In this example, the vehicle control device 210 controls the driving force output from the driving device 221 and the braking force output from the braking device 222 so that the vehicle speed V is equal to or less than a predetermined limit vehicle speed Vlimit while the vehicle 200 is traveling under remote leaving control.

[0139] 25, when the host vehicle 200 reaches the target exit point Pout_tgt, the vehicle control device 210 stops the host vehicle 200 by a stop process, and further holds the host vehicle 200 in a stopped state by a stop holding process, stops the operation of the surrounding information detection device 260, the vehicle driving device 220, and the like, and ends the remote exit control. This completes the exit of the host vehicle 200 from the current parking space 31N.

[0140] Furthermore, when the leaving of the vehicle 200 is completed, the vehicle control device 210 wirelessly transmits a leaving completion signal S28 to the outside. The leaving completion signal S28 is a signal indicating that the leaving of the vehicle 200 is completed.

[0141] When the terminal control device 110 receives the leaving completion signal S28, as shown in Fig. 26, it displays a leaving completion image G69 on the terminal display 120. The leaving completion image G69 is an image indicating that the leaving of the vehicle 200 has been completed.

[0142] Furthermore, when the vehicle control device 210 determines that the vehicle 200 needs to be turned in order to leave the parking lot during remote leaving control, the vehicle control device 210 is configured to change the shift state of the transmission device 224 and turn the vehicle 200.

[0143] <Exiting the app, etc.> Furthermore, when a touch operation is performed on the application end image portion P15 or the like (i.e., the application end image portion P15, P25, P35, P45, P55, or P65), the terminal control device 110 ends the remote driving application and wirelessly transmits a control end command signal S30 to the outside. The control end command signal S30 is a signal that commands the end of the remote entry and exit control.

[0144] In addition, when a touch operation is performed on the entry / exit reselection image portion P36 or the like (i.e., the entry / exit reselection image portion P36, P46, P56 or P66), the terminal control device 110 displays the entry / exit selection image G20 on the terminal display 120 and wirelessly transmits a control end command signal S30 to the outside.

[0145] When the vehicle control device 210 receives the control end command signal S30, it stops the vehicle 200 by a vehicle stop processing, and further maintains the vehicle 200 in a stopped state by a vehicle stop maintenance processing, stops the operation of devices such as the surrounding information detection device 260 and the vehicle driving device 220, terminates the remote entry control if it is executing the remote entry control, and terminates the remote exit control if it is executing the remote exit control.

[0146] Furthermore, when there are many screen transitions on the terminal display 120 while the remote driving app is running, the terminal control device 110 may be configured to display an image on the terminal display 120 that specifies the part of the terminal display 120 to which a touch operation should be applied in order to return the screen to the previous screen.

[0147] <Vibration treatment> As explained above, when the terminal user UR is driving the vehicle 200 autonomously using remote driving control (remote entry / exit control), the terminal user UR continuously performs touch operations with his / her finger on the terminal display 120 of the operation terminal 100. However, in order to ensure the driving safety of the vehicle 200 at that time, the terminal user UR is required to perform touch operations on the terminal display 120 while visually checking the driving status of the vehicle 200, rather than performing touch operations on the terminal display 120 while looking at the terminal display 120 in front of him / her.

[0148] Therefore, if information regarding the control status of the vehicle 200 can be provided to the terminal user UR from the operation terminal 100, the terminal user UR can obtain more information regarding the control status of the vehicle 200, which is preferable for the terminal user UR.

[0149] Therefore, when the vehicle ECU 290 is causing the host vehicle 200 to travel autonomously by remote travel control (when the host vehicle 200 is moving forward autonomously and when the host vehicle 200 is moving backward autonomously), the vehicle ECU 290 is configured to operate the vibration device 130 and transmit a signal (traveling vibration request signal S41) requesting that the operation terminal 100 be vibrated in a predetermined vibration pattern (traveling vibration pattern) to the outside via the vehicle transceiver 280. In other words, the vehicle ECU 290 is configured to transmit the traveling vibration request signal S41 to the outside via the vehicle transceiver 280 when the control state of the host vehicle 200 is in a state in which the host vehicle 200 is traveling during execution of the remote travel control.

[0150] On the other hand, the terminal ECU 190 is configured to execute vibration processing (running vibration processing) for operating the vibration device 130 so as to generate vibration of a running vibration pattern when the terminal ECU 190 receives the running vibration request signal S41 via the terminal transceiver device 180. In other words, the terminal ECU 190 and the vehicle ECU 290 are configured to execute running vibration processing when the control state of the host vehicle 200 is in a state in which the host vehicle 200 is running during execution of the remote running control.

[0151] According to this, the terminal user UR can visually recognize that the vehicle 200 is traveling, and can also recognize that the vehicle 200 is traveling from the vibration of the operation terminal 100.

[0152] In this example, the vibration pattern is defined by the duration of one vibration (vibration duration), the number of vibrations in one vibration process (vibration count), the time interval between vibrations in one vibration process (vibration time interval), the strength of the vibration (vibration strength), and the frequency of the vibration (vibration frequency). Therefore, by varying one or more of the vibration duration, vibration count, vibration time interval, vibration strength, and vibration frequency, the vibration pattern can be made different.

[0153] Also, in this example, the operation terminal 100 vibrates with the same vibration pattern whether the vehicle 200 is moving forward or backward due to remote driving control, but the vibration pattern may be different when the vehicle 200 is moving forward and when the vehicle 200 is moving backward.

[0154] Furthermore, when the vehicle 200 is autonomously traveling by remote traveling control (when the vehicle 200 is autonomously traveling forward and autonomously traveling backward), in addition to or instead of transmitting the traveling vibration request signal S41 to the outside, the vehicle ECU 290 may be configured to transmit a signal (traveling notification sound request signal S51) requesting that the audio device 140 output a notification sound in a predetermined notification sound output pattern (traveling notification sound output pattern) to the outside via the vehicle transceiver / transmitter 280. In this case, when the terminal ECU 190 receives the traveling notification sound request signal S51 via the terminal transceiver / transmitter 180, it is configured to execute a notification sound process (traveling notification sound process) that outputs a notification sound of the traveling notification sound output pattern from the audio device 140.

[0155] Furthermore, when the host vehicle 200 is stopped by the remote driving control, the vehicle ECU 290 is configured not to transmit to the outside a signal requesting vibration by the vibration device 130 (vibration request signal S40). In other words, the vehicle ECU 290 is configured not to transmit to the outside the vibration request signal S40 when the control state of the host vehicle 200 is in a state in which the host vehicle 200 is stopped during execution of the remote driving control.

[0156] On the other hand, when the terminal ECU 190 does not receive the vibration request signal S40, it stops the operation of the vibration device 130. That is, the terminal ECU 190 does not perform any vibration processing. In other words, when the control state of the host vehicle 200 is in a state in which the host vehicle 200 is stopped during execution of the remote driving control, the terminal ECU 190 does not perform any vibration processing.

[0157] Furthermore, when the shift state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR in order to turn the vehicle 200, the vehicle ECU 290 is configured to transmit a signal (shift change vibration request signal S42) requesting that the vibration device 130 be activated to vibrate the operation terminal 100 in a predetermined vibration pattern (shift change vibration pattern) to the outside via the vehicle transceiver 280. In other words, the vehicle ECU 290 is configured to transmit the shift change vibration request signal S42 to the outside via the vehicle transceiver 280 when the control state of the vehicle 200 is in a state in which the shift state of the transmission device 224 is changed during execution of the remote driving control.

[0158] The shift change vibration pattern is set to a pattern different from the driving vibration pattern. The vehicle ECU 290 may be configured to transmit the shift change vibration request signal S42 to the outside via the vehicle transceiver 280 when the shift state of the transmission device 224 is changed from the reverse drive state SR to the forward drive state SD in order to turn the vehicle 200.

[0159] In addition, the vehicle ECU 290 may be configured to transmit the shift change vibration request signal S42 to the outside via the vehicle transceiver device 280 when the shift state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR, and when the shift state is changed from the reverse drive state SR to the forward drive state SD, for purposes other than turning the vehicle 200.

[0160] On the other hand, when the terminal ECU 190 receives the shift change vibration request signal S42 via the terminal transceiver 180, it is configured to execute vibration processing (shift change vibration processing) for activating the vibration device 130 so as to generate vibration of a shift change vibration pattern. In other words, the terminal ECU 190 is configured to execute the shift change vibration processing when the control state of the host vehicle 200 is in a state in which the shift state of the transmission device 224 is changed during execution of the remote driving control.

[0161] According to this, although it is difficult for the terminal user UR to visually recognize that the shift state of the transmission device 224 has been changed for the purpose of, for example, turning the vehicle 200, the terminal user UR can recognize that the shift state of the transmission device 224 has been changed from the vibration of the operation terminal 100.

[0162] When changing the shift state of the transmission device 224, the vehicle ECU 290 may be configured to transmit a signal (shift change notification sound request signal S52) requesting that a notification sound be output from the audio device 140 in a predetermined notification sound output pattern (shift change notification sound output pattern) to the outside via the vehicle transceiver device 280 in addition to or instead of transmitting the shift change vibration request signal S42 to the outside. In this case, when the terminal ECU 190 receives the shift change notification sound request signal S52 via the terminal transceiver device 180, it is configured to execute a notification sound process (shift change notification sound process) that outputs a notification sound of the shift change notification sound output pattern from the audio device 140. In this case, the shift change notification sound output pattern is a pattern different from the driving notification sound output pattern.

[0163] The notification sound output pattern is determined by the duration of one notification sound output (notification sound output duration), the number of times the notification sound is output in one notification sound processing (notification sound output count), the time interval between notifications in one notification sound processing (notification sound output time interval), the intensity of the notification sound (notification sound intensity), and the frequency or pitch of the notification sound (notification sound frequency). Therefore, by varying one or more of the notification sound output duration, notification sound output count, notification sound output time interval, notification sound intensity, and notification sound frequency, the notification sound output pattern can be made different.

[0164] According to the remote traveling system described above, for example, during execution of remote traveling control, vibration of the operation terminal 100 occurs as shown in Fig. 27. Fig. 27 shows an example in which, during execution of remote traveling control, the host vehicle 200 starts moving forward at time t270, the host vehicle 200 is temporarily stopped at time t271 to turn the host vehicle 200, the shift state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR at time t272, and the host vehicle 200 starts moving backward at time t273.

[0165] In the example shown in FIG. 27, when the host vehicle 200 starts moving forward at time t270, the traveling vibration process is started and the vibration device 130 of the operation terminal 100 generates vibrations of the traveling vibration pattern on the operation terminal 100.

[0166] After that, at time t271, when the host vehicle 200 is stopped, the vibration processing (running vibration processing) is stopped.

[0167] Then, at time t272, when the shift state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR, a shift change vibration process is executed and the vibration device 130 of the operation terminal 100 generates a vibration of a shift change vibration pattern on the operation terminal 100.

[0168] After that, at time t273, when the host vehicle 200 starts to move backward, the traveling vibration process is started, and the vibration device 130 of the operation terminal 100 generates vibrations of the traveling vibration pattern on the operation terminal 100.

[0169] When the vehicle 200 is being braked by the remote driving control, it is preferable for the terminal user UR to be notified of this.

[0170] Therefore, when the host vehicle 200 is being braked by the remote driving control, the vehicle ECU 290 may be configured to transmit a signal (braking vibration request signal S43) requesting that the vibration device 130 be activated to vibrate the operation terminal 100 in a predetermined vibration pattern (braking vibration pattern) to the outside via the vehicle transceiver device 280. In other words, the vehicle ECU 290 may be configured to transmit the braking vibration request signal S43 to the outside via the vehicle transceiver device 280 when the control state of the host vehicle 200 is in a state in which the host vehicle 200 is being braked during execution of the remote driving control.

[0171] On the other hand, in this case, when the terminal ECU 190 receives the braking vibration request signal S43 via the terminal transceiver device 180, it is configured to execute vibration processing (braking vibration processing) for operating the vibration device 130 so as to generate vibration of a braking vibration pattern. In other words, the terminal ECU 190 is configured to execute braking vibration processing when the control state of the host vehicle 200 is in a state in which the host vehicle 200 is being braked during execution of the remote driving control. In this case, the braking vibration pattern is set to a pattern different from both the driving vibration pattern and the shift change vibration pattern.

[0172] According to this, even when it is difficult for the terminal user UR to visually recognize that the vehicle 200 is being braked, the terminal user UR can recognize that the vehicle 200 is being braked from the vibration of the operation terminal 100 as well.

[0173] When the vehicle 200 is braked by remote driving control, the vehicle ECU 290 may be configured to transmit a signal (braking notification sound request signal S53) requesting that a notification sound be output from the audio device 140 in a predetermined notification sound output pattern (braking notification sound output pattern) via the vehicle transceiver 280 in addition to or instead of transmitting the braking vibration request signal S43 to the outside. In this case, when the terminal ECU 190 receives the braking notification sound request signal S53 via the terminal transceiver 180, it is configured to execute notification sound processing (braking notification sound processing) for outputting a notification sound of the braking notification sound output pattern from the audio device 140. In this case, the braking notification sound output pattern is a pattern different from both the driving notification sound output pattern and the shift change notification sound output pattern.

[0174] In addition, when the vehicle 200 is being driven by remote driving control, if an obstacle such as a person that is obstructing the driving of the vehicle 200 (or an obstacle that is likely to obstruct the driving of the vehicle 200) is detected based on the surrounding detection information IS, it would be preferable for the terminal user UR to be notified of this.

[0175] Therefore, the vehicle ECU 290 may be configured to, when an obstacle is detected while driving the vehicle 200 by remote driving control, activate the vibration device 130 and transmit a signal (obstacle vibration request signal S44) to the outside via the vehicle transceiver device 280 requesting that the operation terminal 100 be vibrated in a predetermined vibration pattern (obstacle vibration pattern).

[0176] On the other hand, in this case, when the terminal ECU 190 receives the obstacle vibration request signal S44 via the terminal transceiver device 180, it is configured to execute vibration processing (obstacle vibration processing) that activates the vibration device 130 so as to generate vibrations of an obstacle vibration pattern.

[0177] In this case, the obstacle vibration pattern is set to a pattern different from the running vibration pattern, the shift change vibration pattern, and the braking vibration pattern. When an obstacle is detected, braking of the vehicle 200 is started. In this example, however, in this case, the obstacle vibration process is executed instead of the braking vibration process.

[0178] According to this, the terminal user UR can recognize the presence of an obstacle not only visually but also from the vibration of the operation terminal 100.

[0179] When an obstacle is detected while the vehicle 200 is traveling by remote travel control, the vehicle ECU 290 may be configured to transmit a signal (obstacle notification sound request signal S54) requesting that a notification sound be output from the audio device 140 in a predetermined notification sound output pattern (obstacle notification sound output pattern) via the vehicle transceiver 280 in addition to or instead of transmitting the obstacle vibration request signal S44 to the outside. In this case, when the terminal ECU 190 receives the obstacle notification sound request signal S54 via the terminal transceiver 180, it is configured to execute notification sound processing (obstacle notification sound processing) for outputting a notification sound of the obstacle notification sound output pattern from the audio device 140. In this case, the obstacle notification sound output pattern is a pattern different from the traveling notification sound output pattern, the shift change notification sound output pattern, and the braking notification sound output pattern.

[0180] According to this remote traveling system, for example, during remote traveling control, vibration of the operation terminal 100 occurs as shown in Fig. 28. Fig. 28 shows an example in which, during remote traveling control, the vehicle 200 starts moving forward at time t280, braking of the vehicle 200 starts to turn the vehicle 200 at time t281, the vehicle 200 is stopped once at time t282, the shift state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR at time t283, the vehicle 200 starts moving backward at time t284, braking of the vehicle 200 starts due to detection of an obstacle at time t285, the vehicle 200 is stopped at time t286, and the vehicle 200 starts moving backward due to the disappearance of the obstacle at time t287.

[0181] In the example shown in FIG. 28, when the host vehicle 200 starts moving forward at time t280, the traveling vibration process is started and the vibration device 130 of the operation terminal 100 generates vibrations of the traveling vibration pattern on the operation terminal 100.

[0182] After that, at time t281, braking of the host vehicle 200 is started, the running vibration process is stopped, braking vibration process is started, and the vibration device 130 of the operation terminal 100 generates vibration of the braking vibration pattern on the operation terminal 100.

[0183] Thereafter, at time t282, when the host vehicle 200 is stopped, the braking vibration processing is stopped.

[0184] Then, at time t283, when the shift state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR, a shift change vibration process is executed, and the vibration device 130 of the operation terminal 100 generates a vibration of a shift change vibration pattern on the operation terminal 100.

[0185] After that, at time t284, when the host vehicle 200 starts to move backward, the traveling vibration process is started, and the vibration device 130 of the operation terminal 100 generates vibrations of the traveling vibration pattern on the operation terminal 100.

[0186] After that, at time t285, when an obstacle is detected and braking of the host vehicle 200 is started, the traveling vibration process is stopped, the obstacle vibration process is started, and the vibration device 130 of the operation terminal 100 generates vibration of the obstacle vibration pattern on the operation terminal 100. After that, at time t286, the host vehicle 200 is stopped.

[0187] Then, at time t287, when the obstacle is no longer detected and the vehicle 200 begins to back up, the obstacle vibration processing is stopped, the driving vibration processing is started, and the vibration device 130 of the operation terminal 100 generates vibrations in the driving vibration pattern on the operation terminal 100.

[0188] The above has described the remote driving system 10 which executes driving vibration processing when the host vehicle 200 is moving forward or backward by remote driving control, and which executes shift change vibration processing when the host vehicle 200 turns, and the remote driving system 10 which, in addition to these vibration processing, executes obstacle vibration processing when an obstacle is detected. However, the remote driving system 10 may be configured to vibrate the operation terminal 100 by the vibration device 130 in a vibration pattern different from the vibration pattern for notifying other information, in order to notify the terminal user UR of information that is preferable to be notified of by vibration, in addition to information such as the driving state (forward and backward) of the host vehicle 200, turning of the host vehicle 200, and detection of an obstacle.

[0189] The above is an overview of the operation of the remote traveling system 10. According to the remote traveling system 10, the vibration pattern of the operation terminal 100 is different when the host vehicle 200 is traveling by remote traveling control, when the host vehicle 200 is braked, when the shift state of the transmission device 224 is changed, and when an obstacle is detected, so that the terminal user UR can recognize the control state of the host vehicle 200 and the presence of an obstacle from the vibration of the operation terminal 100 while watching the host vehicle 200 without looking at the terminal display 120. In other words, since the vibration pattern is different for each control state of the host vehicle 200 and the situation around the host vehicle 200, the terminal user UR can recognize the control state of the host vehicle 200 and the situation around the host vehicle 200 from the vibration of the operation terminal 100 while watching the host vehicle 200 without looking at the terminal display 120.

[0190] <Specific operation of the remote driving system> Next, a specific operation of the remote driving system 10 will be described. The terminal CPU 191 of the terminal ECU 190 of the terminal control device 110 executes the routine shown in FIG. 29 at a predetermined calculation cycle. Therefore, at a predetermined timing, the terminal CPU 191 starts processing from step 2900 in FIG. 29, advances the processing to step 2905, and determines whether the value of the initial screen image display flag X10 is "0". The initial screen image display flag X10 is a flag indicating whether the initial screen image G10 is displayed on the terminal display 120, and its value is set to "1" when the initial screen image G10 is displayed on the terminal display 120. Note that when the value of the initial screen image display flag X10 is "0", the initial screen image G10 is not displayed on the terminal display 120.

[0191] If the terminal CPU 191 judges "No" in step 2905, the process proceeds to step 2910, where it judges whether or not an application start operation has been performed. That is, the terminal CPU 191 judges whether or not a predetermined touch operation for starting the remote driving application has been applied to the terminal display 120. If the terminal CPU 191 judges "Yes" in step 2910, the process proceeds to step 2915, where it starts the remote driving application and displays the initial screen image G10. Next, the terminal CPU 191 proceeds to step 2920, where it wirelessly transmits the application start signal S10 to the outside. Next, the terminal CPU 191 proceeds to step 2925, where it sets the value of the initial screen image display flag X10 to "1". Next, the terminal CPU 191 proceeds to step 2995, where it temporarily ends this routine.

[0192] On the other hand, if the terminal CPU 191 determines "No" in step 2910, it advances the process directly to step 2995 and temporarily ends this routine.

[0193] Also, when the terminal CPU 191 judges "No" in step 2905, the process proceeds to step 2930, where it judges whether or not an application termination operation has been performed. That is, the terminal CPU 191 judges whether or not a tap operation has been performed on the application termination image portion P15, etc. When the terminal CPU 191 judges "Yes" in step 2930, the process proceeds to step 2935, where it terminates the remote driving application. Next, the terminal CPU 191 proceeds to step 2940, where it wirelessly transmits a control termination command signal S30 to the outside. Next, the terminal CPU 191 proceeds to step 2945, where it sets the values ​​of the initial screen image display flag X10, the entrance / exit selection image display flag X11, the parking space selection image display flag X12, the exit direction selection image display flag X13, the entrance operation image display flag X14, and the exit operation image display flag X15 to "0", respectively. Next, the terminal CPU 191 advances the process to step 2995, and temporarily ends this routine.

[0194] On the other hand, if the terminal CPU 191 determines "No" in step 2930, it advances the process directly to step 2995 and temporarily ends this routine.

[0195] Furthermore, the terminal CPU 191 executes the routine shown in FIG. 30 at a predetermined calculation cycle. Therefore, at a predetermined timing, the terminal CPU 191 starts the process from step 3000 in FIG. 30, and proceeds to step 3005 to determine whether the value of the initial screen image display flag X10 is "1". When the terminal CPU 191 determines "Yes" at step 3005, it proceeds to step 3010 to determine whether the value of the entry / exit selection image display flag X11 is "0". The entry / exit selection image display flag X11 is a flag indicating whether the entry / exit selection image G20 is displayed on the terminal display 120, and its value is set to "1" when the entry / exit selection image G20 is displayed on the terminal display 120. When the value of the entry / exit selection image display flag X11 is "0", the entry / exit selection image G20 is not displayed on the terminal display 120.

[0196] If the terminal CPU 191 judges "Yes" in step 3010, the process proceeds to step 3015, where it judges whether or not a remote entry / exit start operation has been performed. That is, the terminal CPU 191 judges whether or not a touch operation has been performed on the remote entry / exit start image portion P11. If the terminal CPU 191 judges "Yes" in step 3015, it proceeds to step 3020, where it displays the entry / exit selection image G20 on the terminal display 120. Next, the terminal CPU 191 proceeds to step 3025, where it sets the value of the entry / exit selection image display flag X11 to "1". Next, the terminal CPU 191 proceeds to step 3095, where it temporarily ends this routine.

[0197] On the other hand, if the terminal CPU 191 determines "No" in step 3015, it advances the process directly to step 3095 and temporarily ends this routine.

[0198] Also, if the terminal CPU 191 determines "No" in step 3005 or step 3010, it advances the process directly to step 3095 and temporarily ends this routine.

[0199] Furthermore, the terminal CPU 191 executes the routine shown in FIG. 31 at a predetermined calculation cycle. Therefore, at a predetermined timing, the terminal CPU 191 starts the process from step 3100 in FIG. 31, and proceeds to step 3105 to determine whether the value of the entry / exit selection image display flag X11 is "1". When the terminal CPU 191 determines "Yes" in step 3105, it proceeds to step 3110 to determine whether the value of the parking section selection image display flag X12 is "0". The parking section selection image display flag X12 is a flag indicating whether the parking section selection image G30 is displayed on the terminal display 120, and its value is set to "1" when the parking section selection image G30 is displayed on the terminal display 120. When the value of the parking section selection image display flag X12 is "0", the parking section selection image G30 is not displayed on the terminal display 120.

[0200] If the terminal CPU 191 judges "Yes" in step 3110, the process proceeds to step 3115, where it judges whether or not an entry selection operation has been performed. That is, the terminal CPU 191 judges whether or not a touch operation has been performed on the entry selection image portion P21. If the terminal CPU 191 judges "Yes" in step 3115, it proceeds to step 3120, where it displays the parking space selection image G30 on the terminal display 120. Next, the terminal CPU 191 proceeds to step 3125, where it sets the value of the parking space selection image display flag X12 to "1". Next, the terminal CPU 191 proceeds to step 3130.

[0201] On the other hand, if the terminal CPU 191 determines “No” in step 3115 , it advances the process directly to step 3130 .

[0202] When the terminal CPU 191 advances the process to step 3130, it determines whether the value of the leaving direction selection image display flag X13 is "0". The leaving direction selection image display flag X13 is a flag indicating whether the leaving direction selection image G50 is displayed on the terminal display 120, and its value is set to "1" when the leaving direction selection image G50 is displayed on the terminal display 120. When the value of the leaving direction selection image G50 is "0", the leaving direction selection image G50 is not displayed on the terminal display 120.

[0203] If the terminal CPU 191 judges "Yes" in step 3130, the process proceeds to step 3140, where it judges whether or not a warehouse-leaving selection operation has been applied to the terminal display 120. That is, the terminal CPU 191 judges whether or not a touch operation has been applied to the warehouse-leaving selection image portion P22. If the terminal CPU 191 judges "Yes" in step 3135, it proceeds to step 3140, where it displays the warehouse-leaving direction selection image G50 on the terminal display 120. Next, the terminal CPU 191 proceeds to step 3145, where it sets the value of the warehouse-leaving direction selection image display flag X13 to "1". Next, the terminal CPU 191 proceeds to step 3195, where it temporarily ends this routine.

[0204] On the other hand, if the terminal CPU 191 determines "No" in step 3135, it advances the process directly to step 3195 and temporarily ends this routine.

[0205] Also, if the terminal CPU 191 judges "No" in step 3105, step 3110, or step 3130, it advances the process directly to step 3195 and temporarily ends this routine.

[0206] Furthermore, the terminal CPU 191 executes the routine shown in FIG. 32 at a predetermined calculation cycle. Therefore, at a predetermined timing, the terminal CPU 191 starts the process from step 3200 in FIG. 32, and proceeds to step 3205 to determine whether the value of the parking space selection image display flag X12 is "1". When the terminal CPU 191 determines "Yes" in step 3205, it proceeds to step 3210 to determine whether the value of the entry operation image display flag X14 is "0". The entry operation image display flag X14 is a flag indicating whether the entry operation image G40 is displayed on the terminal display 120, and its value is set to "1" when the entry operation image G40 is displayed on the terminal display 120. When the value of the entry operation image display flag X14 is "0", the entry operation image G40 is not displayed on the terminal display 120.

[0207] If the terminal CPU 191 judges "Yes" in step 3210, the process proceeds to step 3215, where it determines whether or not a parking space selection operation has been performed on the terminal display 120. That is, the terminal CPU 191 judges whether or not a touch operation has been performed on the available parking space image portion P31. If the terminal CPU 191 judges "Yes" in step 3210, the process proceeds to step 3220, where it determines whether or not a confirmation operation has been performed on the terminal display 120. That is, the terminal CPU 191 determines whether or not a touch operation has been performed on the selection confirmation image portion P34.

[0208] If the terminal CPU 191 judges "Yes" in step 3220, the process proceeds to step 3225 and displays the entrance operation image G40. Next, the terminal CPU 191 proceeds to step 3230 and wirelessly transmits the designated parking space signal S12 to the outside. At this time, the terminal CPU 191 displays a vehicle image G42 in a portion of the terminal display 120 corresponding to the position of the vehicle 200 indicated by the vehicle position signal S13, displays on the terminal display 120 a remaining entrance distance image G43 indicating the remaining entrance distance Din indicated by the remaining entrance distance signal S15, and displays on the terminal display 120 a traveling direction image G47 indicating the traveling direction of the vehicle 200 indicated by the traveling direction signal S17.

[0209] Next, the terminal CPU 191 advances the process to step 3235, and sets the value of the warehousing operation image display flag X14 to “1.” Next, the terminal CPU 191 advances the process to step 3240.

[0210] On the other hand, if the terminal CPU 191 determines “No” in step 3220 , it advances the process directly to step 3240 .

[0211] Also, if the terminal CPU 191 determines "No" in step 3215, it advances the process directly to step 3240.

[0212] When the terminal CPU 191 advances the process to step 3240, it judges whether or not an entry / exit reselection operation has been applied to the terminal display 120. That is, the terminal CPU 191 judges whether or not a touch operation has been applied to the entry / exit reselection image portion P36. When the terminal CPU 191 judges "Yes" in step 3240, it advances the process to step 3245, and displays the entry / exit selection image G20 on the terminal display 120. Next, the terminal CPU 191 advances the process to step 3250, and sets the values ​​of the entry / exit selection image display flag X11, the parking space selection image display flag X12, and the entry operation image display flag X14 to "0". Next, the terminal CPU 191 advances the process to step 3295, and temporarily ends this routine.

[0213] On the other hand, if the terminal CPU 191 determines "No" in step 3240, it advances the process directly to step 3295 and temporarily ends this routine.

[0214] Also, if the terminal CPU 191 judges "No" in step 3205 or step 3210, it advances the process directly to step 3295 and temporarily ends this routine.

[0215] Furthermore, the terminal CPU 191 executes the routine shown in FIG. 33 at a predetermined calculation cycle. Therefore, at a predetermined timing, the terminal CPU 191 starts processing from step 3300 in FIG. 33, and proceeds to step 3305 to determine whether the value of the leaving direction selection image display flag X13 is "1". When the terminal CPU 191 determines "Yes" in step 3305, it proceeds to step 3310 to determine whether the value of the leaving operation image display flag X15 is "0". The leaving operation image display flag X15 is a flag indicating whether the leaving operation image G60 is displayed on the terminal display 120, and in that case, it is set to "1" when the leaving operation image G60 is displayed on the terminal display 120. Note that when the value of the leaving operation image display flag X15 is "0", the leaving operation image G60 is not displayed on the terminal display 120.

[0216] If the terminal CPU 191 determines "Yes" in step 3310, it determines whether or not an exit direction selection operation has been applied to the terminal display 120. That is, the terminal CPU 191 determines whether or not a touch operation has been applied to the available exit direction image portion P53. If the terminal CPU 191 determines "Yes" in step 3315, it advances the process to step 3320, and determines whether or not a confirmation operation has been applied to the terminal display 120. That is, the terminal CPU 191 determines whether or not a touch operation has been applied to the selection confirmation image portion P54.

[0217] If the terminal CPU 191 determines "Yes" in step 3320, the process proceeds to step 3325, where it displays an exit operation image G60 on the terminal display 120. Next, the terminal CPU 191 proceeds to step 3330, where it wirelessly transmits a designated exit direction signal S22 to the outside. At this time, the terminal CPU 191 displays a vehicle image G62 in a portion of the terminal display 120 corresponding to the position of the host vehicle 200 indicated by the vehicle position signal S23, and displays on the terminal display 120 a remaining exit distance image G63 indicating the remaining exit distance Dout indicated by the remaining exit distance signal S25.

[0218] Next, the terminal CPU 191 advances the process to step 3335, and sets the value of the retrieval operation image display flag X15 to “1.” Next, the terminal CPU 191 advances the process to step 3340.

[0219] On the other hand, if the terminal CPU 191 determines “No” in step 3320 , it advances the process directly to step 3340 .

[0220] Also, if the terminal CPU 191 determines "No" in step 3315, it advances the process directly to step 3340.

[0221] When the terminal CPU 191 advances the process to step 3340, it judges whether or not a warehousing / removal reselection operation has been applied to the terminal display 120. That is, the terminal CPU 191 judges whether or not a touch operation has been applied to the warehousing / removal reselection image portion P56. When the terminal CPU 191 judges "Yes" in step 3340, it advances the process to step 3345 and displays the warehousing / removal selection image G20 on the terminal display 120. Next, the terminal CPU 191 advances the process to step 3350 and sets the values ​​of the warehousing / removal selection image display flag X11, the warehousing direction selection image display flag X13, and the warehousing operation image display flag X15 to "0". Next, the terminal CPU 191 advances the process to step 3395 and ends this routine for the time being.

[0222] On the other hand, if the terminal CPU 191 determines "No" in step 3340, it advances the process directly to step 3395 and temporarily ends this routine.

[0223] Also, if the terminal CPU 191 judges "No" in step 3305 or step 3310, it advances the process directly to step 3395 and temporarily ends this routine.

[0224] Furthermore, the terminal CPU 191 executes the routine shown in FIG. 34 at a predetermined calculation cycle. Therefore, at a predetermined timing, the terminal CPU 191 starts the process from step 3400 in FIG. 34, and proceeds to step 3405 to determine whether the value of the entry operation image display flag X14 is "1". If the terminal CPU 191 determines "Yes" at step 3405, it proceeds to step 3410 to determine whether a travel operation has been performed. That is, the terminal CPU 191 determines whether a touch operation has been applied to the entry travel operation image portion P44. If the terminal CPU 191 determines "Yes" at step 3410, it proceeds to step 3415 to wirelessly transmit the continuous touch operation signal S16 to the outside. Next, the terminal CPU 191 proceeds to step 3420.

[0225] On the other hand, if the terminal CPU 191 determines “No” in step 3410 , it advances the process directly to step 3420 .

[0226] When the terminal CPU 191 advances the process to step 3420, it determines whether or not the entry of the vehicle 200 into the designated parking section 31D is complete. That is, the terminal CPU 191 determines whether or not the entry completion signal S18 has been received. When the terminal CPU 191 determines "Yes" in step 3420, it advances the process to step 3425 and displays an entry completion image G49 on the terminal display 120. Next, the terminal CPU 191 advances the process to step 3430.

[0227] On the other hand, if the terminal CPU 191 determines “No” in step 3420 , it advances the process directly to step 3430 .

[0228] When the terminal CPU 191 advances the process to step 3430, it judges whether or not an entry / exit reselection operation has been applied to the terminal display 120. That is, the terminal CPU 191 judges whether or not a touch operation has been applied to the entry / exit reselection image portion P46. When the terminal CPU 191 judges "Yes" in step 3430, it advances the process to step 3435, and displays the entry / exit selection image G20 on the terminal display 120. Next, the terminal CPU 191 advances the process to step 3440, and sets the values ​​of the parking space selection image display flag X12 and the entry operation image display flag X14 to "0", respectively. Next, the terminal CPU 191 advances the process to step 3495, and temporarily ends this routine.

[0229] On the other hand, if the terminal CPU 191 determines "No" in step 3430, it advances the process directly to step 3495 and temporarily ends this routine.

[0230] Also, if the terminal CPU 191 determines "No" in step 3405, it advances the process directly to step 3495 and temporarily ends this routine.

[0231] Furthermore, the terminal CPU 191 executes the routine shown in FIG. 35 at a predetermined calculation cycle. Therefore, at a predetermined timing, the terminal CPU 191 starts the process from step 3500 in FIG. 35, and proceeds to step 3505 to determine whether the value of the leaving operation image display flag X15 is "1". If the terminal CPU 191 determines "Yes" at step 3505, it proceeds to step 3510 to determine whether a traveling operation has been performed. That is, the terminal CPU 191 determines whether a touch operation has been applied to the leaving traveling operation image portion P64. If the terminal CPU 191 determines "Yes" at step 3510, it proceeds to step 3515 to wirelessly transmit the continuous touch operation signal S26 to the outside. Next, the terminal CPU 191 proceeds to step 3520.

[0232] On the other hand, if the terminal CPU 191 determines “No” in step 3510 , it advances the process directly to step 3520 .

[0233] When the terminal CPU 191 advances the process to step 3420, it determines whether or not the exit of the vehicle 200 from the current parking space 31N has been completed. That is, the terminal CPU 191 determines whether or not the exit completion signal S28 has been received. When the terminal CPU 191 determines "Yes" in step 3520, it advances the process to step 3425 and displays the exit completion image G69 on the terminal display 120. Next, the terminal CPU 191 advances the process to step 3530.

[0234] On the other hand, if the terminal CPU 191 determines “No” in step 3520 , it advances the process directly to step 3530 .

[0235] When the terminal CPU 191 advances the process to step 3530, it judges whether or not an entry / exit reselection operation has been applied to the terminal display 120. That is, the terminal CPU 191 judges whether or not a touch operation has been applied to the entry / exit reselection image portion P66. When the terminal CPU 191 judges "Yes" in step 3530, it advances the process to step 3535 and displays the entry / exit selection image G20 on the terminal display 120. Next, the terminal CPU 191 advances the process to step 3540 and sets the values ​​of the exit direction selection image display flag X13 and the exit operation image display flag X15 to "0", respectively. Next, the terminal CPU 191 advances the process to step 3595 and ends this routine for the time being.

[0236] On the other hand, if the terminal CPU 191 determines "No" in step 3530, it advances the process directly to step 3595 and temporarily ends this routine.

[0237] Also, if the terminal CPU 191 determines "No" in step 3505, it advances the process directly to step 3595 and temporarily ends this routine.

[0238] On the other hand, the vehicle CPU 291 of the vehicle ECU 290 of the vehicle control device 210 executes the routine shown in Fig. 36 at a predetermined calculation cycle. Therefore, at a predetermined timing, the vehicle CPU 291 starts the process from step 3600 in Fig. 36, and proceeds to step 3605 to determine whether the value of the warehousing route setting completion flag X20 is "0". The warehousing route setting completion flag X20 is a flag indicating whether the target warehousing route Rin_tgt has been set, and its value is set to "1" when the target warehousing route Rin_tgt has been set. Note that when the value of the warehousing route setting completion flag X20 is "0", the target warehousing route Rin_tgt has not been set.

[0239] If the vehicle CPU 291 determines "Yes" in step 3605, the process proceeds to step 3610, where it determines whether or not the designated parking space signal S14 has been received. If the vehicle CPU 291 determines "Yes" in step 3610, the process proceeds to step 3615, where it sets the target entrance route Rin_tgt. Next, the vehicle CPU 291 proceeds to step 3620, where it sets the value of the entrance route setting completion flag X20 to "1". Next, the vehicle CPU 291 proceeds to step 3695, where it temporarily ends this routine.

[0240] On the other hand, if the vehicle CPU 291 determines "No" in step 3610, it proceeds directly to step 3695 and temporarily ends this routine.

[0241] Also, if the vehicle CPU 291 determines "No" in step 3605, it proceeds directly to step 3695 and temporarily ends this routine.

[0242] Furthermore, the vehicle CPU 291 executes the routine shown in Fig. 37 at a predetermined calculation cycle. Therefore, at a predetermined timing, the vehicle CPU 291 starts processing from step 3700 in Fig. 37, and proceeds to step 3705 to determine whether the value of the entry route setting completion flag X20 is "1". If the vehicle CPU 291 determines "Yes" in step 3705, it proceeds to step 3710 to determine whether the continuous touch operation signal S16 has been received.

[0243] If the vehicle CPU 291 determines "Yes" in step 3710, the process proceeds to step 3715, where it controls the operation of the vehicle driving device 220 so that the host vehicle 200 travels along the target entrance route Rin_tgt. Next, the vehicle CPU 291 proceeds to step 3720, where it determines whether the host vehicle 200 has reached the target parking point Pin_tgt. That is, the vehicle CPU 291 determines whether the entrance of the host vehicle 200 into the designated parking section 31D has been completed.

[0244] If the vehicle CPU 291 determines "Yes" in step 3720, it executes a vehicle stop process, and further executes a vehicle stop maintenance process. Next, the vehicle CPU 291 proceeds to step 3730, where it stops the operation of the surrounding information detection device 260 and the vehicle driving device 220. Next, the vehicle CPU 291 proceeds to step 3735, where it wirelessly transmits an entry completion signal S18 to the outside. Next, the vehicle CPU 291 proceeds to step 3740, where it sets the value of the entry route setting completion flag X20 to "0". Next, the vehicle CPU 291 proceeds to step 3795, where it temporarily ends this routine.

[0245] On the other hand, if the vehicle CPU 291 determines "No" in step 3720, it proceeds directly to step 3795 and temporarily ends this routine.

[0246] On the other hand, if the vehicle CPU 291 determines "No" in step 3710, the process proceeds to step 3745, where it executes vehicle stop processing. The vehicle CPU 291 proceeds to step 3795, where it temporarily ends this routine.

[0247] Moreover, if the vehicle CPU 291 determines "No" in step 3705, it proceeds directly to step 3795 and temporarily ends this routine.

[0248] Furthermore, the vehicle CPU 291 executes the routine shown in Fig. 38 at a predetermined calculation cycle. Therefore, at a predetermined timing, the vehicle CPU 291 starts processing from step 3800 in Fig. 38, and proceeds to step 3805 to determine whether the value of the outgoing route setting completion flag X21 is "0". The outgoing route setting completion flag X21 is a flag indicating whether the target outgoing route Rout_tgt has been set, and its value is set to "1" when the target outgoing route Rout_tgt has been set. Note that when the value of the outgoing route setting completion flag X21 is "0", the target outgoing route Rout_tgt has not been set.

[0249] If the vehicle CPU 291 determines "Yes" in step 3805, the process proceeds to step 3810, where it determines whether or not the designated leaving direction signal S22 has been received. If the vehicle CPU 291 determines "Yes" in step 3810, the process proceeds to step 3815, where it sets the target leaving route Rout_tgt. Next, the vehicle CPU 291 proceeds to step 3820, where it sets the value of the leaving route setting completion flag X21 to "1". Next, the vehicle CPU 291 proceeds to step 3895, where it temporarily ends this routine.

[0250] On the other hand, if the vehicle CPU 291 determines "No" in step 3810, it proceeds directly to step 3895 and temporarily ends this routine.

[0251] Also, if the vehicle CPU 291 determines "No" in step 3805, it proceeds directly to step 3895 and temporarily ends this routine.

[0252] Furthermore, the vehicle CPU 291 is configured to execute the routine shown in Fig. 39 at a predetermined calculation cycle. Therefore, at a predetermined timing, the vehicle CPU 291 starts processing from step 3900 in Fig. 39, and proceeds to step 3905 to determine whether the value of the leaving route setting completion flag X21 is "1". If the vehicle CPU 291 determines "Yes" in step 3905, it proceeds to step 3910 to determine whether the continuous touch operation signal S26 has been received.

[0253] If the vehicle CPU 291 determines "Yes" in step 3910, the process proceeds to step 3915, where the vehicle CPU 291 controls the operation of the vehicle driving device 220 so that the host vehicle 200 travels along the target exit route Rout_tgt. Next, the vehicle CPU 291 proceeds to step 3920, where the vehicle CPU 291 determines whether the host vehicle 200 has reached the target exit point Pout_tgt. That is, the vehicle CPU 291 determines whether the exit of the host vehicle 200 from the designated parking section 31D has been completed.

[0254] If the vehicle CPU 291 judges "Yes" in step 3920, it executes a vehicle stop process, and further executes a vehicle stop maintenance process. Next, the vehicle CPU 291 advances the process to step 3930, and stops the operation of the surrounding information detection device 260 and the vehicle driving device 220. Next, the vehicle CPU 291 advances the process to step 3935, and wirelessly transmits an exit completion signal S28 to the outside. Next, the vehicle CPU 291 advances the process to step 3940, and sets the value of the exit route setting completion flag X21 to "0". Next, the vehicle CPU 291 advances the process to step 3995, and temporarily ends this routine.

[0255] On the other hand, if the vehicle CPU 291 determines "No" in step 3920, it proceeds directly to step 3995 and temporarily ends this routine.

[0256] On the other hand, if the vehicle CPU 291 determines "No" in step 3910, the process proceeds to step 3945, where it executes vehicle stop processing. The vehicle CPU 291 proceeds to step 3995, where it temporarily ends this routine.

[0257] Moreover, if the vehicle CPU 291 determines "No" in step 3905, it proceeds directly to step 3995 and temporarily ends this routine.

[0258] Furthermore, the vehicle CPU 291 is configured to execute the routine shown in Fig. 40 at a predetermined calculation cycle. Therefore, at a predetermined timing, the vehicle CPU 291 starts the process from step 4000 in Fig. 40, advances the process to step 4005, and determines whether or not the control end command signal S30 has been received.

[0259] If the vehicle CPU 291 determines "Yes" in step 4005, the process proceeds to step 4010, where it executes a stop process, and further executes a stop maintenance process. Next, the vehicle CPU 291 proceeds to step 4015, where it stops the operation of the surrounding information detection device 260 and the vehicle driving device 220. Next, the vehicle CPU 291 proceeds to step 4020, where it sets the values ​​of the entrance route setting completion flag X20 and the exit route setting completion flag X21 to "0", respectively. Next, the vehicle CPU 291 proceeds to step 4095, where it temporarily ends this routine.

[0260] On the other hand, if the vehicle CPU 291 determines "No" in step 4005, it proceeds directly to step 4095 and temporarily ends this routine.

[0261] Furthermore, the vehicle CPU 291 is configured to execute the routine shown in Fig. 41 or Fig. 43 at a predetermined calculation cycle. Therefore, when the vehicle CPU 291 executes the routine shown in Fig. 41, at a predetermined timing, the vehicle CPU 291 starts processing from step 4100 of the routine shown in Fig. 41, and proceeds to step 4105 to determine whether or not remote driving control is being executed.

[0262] If the vehicle CPU 291 determines "Yes" in step 4105, the process proceeds to step 4110, where it determines whether the host vehicle 200 is traveling (forward or backward). If the vehicle CPU 291 determines "Yes" in step 4110, the process proceeds to step 4115, where it transmits the traveling vibration request signal S41 to the outside via the vehicle transmitting / receiving device 280. Next, the vehicle CPU 291 proceeds to step 4120.

[0263] On the other hand, if the vehicle CPU 291 determines “No” in step 4110 , it advances the process directly to step 4120 .

[0264] When the vehicle CPU 291 advances the process to step 4120, it determines whether the shift state of the transmission device 224 has been changed from a forward drive state to a reverse drive state or from a reverse drive state to a forward drive state. If the vehicle CPU 291 determines "Yes" in step 4120, it advances the process to step 4125 and transmits a shift change vibration request signal S42 to the outside via the vehicle transmitting / receiving device 280. Next, the vehicle CPU 291 advances the process to step 4195 and temporarily ends the process of this routine.

[0265] On the other hand, if the vehicle CPU 291 determines "No" in step 4120, it proceeds directly to step 4195, and temporarily ends the processing of this routine.

[0266] Also, if the vehicle CPU 291 determines "No" in step 4105, it proceeds directly to step 4195 and temporarily ends the processing of this routine.

[0267] On the other hand, when the vehicle CPU 291 executes the routine shown in Fig. 41, the terminal CPU 191 executes the routine shown in Fig. 42 at a predetermined calculation period. Therefore, at a predetermined timing, the process starts from step 4200 of the routine shown in Fig. 42, and proceeds to step 4205 to determine whether the remote driving application is running.

[0268] If the terminal CPU 191 judges "Yes" in step 4205, the process proceeds to step 4210, where it judges whether or not the running vibration request signal S41 has been received. If the terminal CPU 191 judges "Yes" in step 4210, the process proceeds to step 4215, where it executes running vibration processing. This causes the operation terminal 100 to vibrate in the running vibration pattern. Next, the terminal CPU 191 proceeds to step 4220.

[0269] On the other hand, if the terminal CPU 191 determines “No” in step 4210 , it advances the process directly to step 4220 .

[0270] When the terminal CPU 191 advances the process to step 4220, it judges whether or not the shift change vibration request signal S42 has been received. When the terminal CPU 191 judges "Yes" in step 4220, it advances the process to step 4225 and executes the shift change vibration process. As a result, the operation terminal 100 is vibrated in the shift change vibration pattern. Next, the terminal CPU 191 advances the process to step 4295 and temporarily ends the process of this routine.

[0271] On the other hand, if the terminal CPU 191 determines "No" in step 4220, it advances the process directly to step 4295 and temporarily ends the process of this routine.

[0272] Also, if the terminal CPU 191 judges "No" in step 4205, it advances the process directly to step 4295 and temporarily ends the process of this routine.

[0273] Alternatively, when the vehicle CPU 291 is configured to execute the routine shown in FIG. 43, at a predetermined timing, the vehicle CPU 291 starts processing from step 4300 of the routine shown in FIG. 43, advances the processing to step 4305, and determines whether remote driving control is being executed.

[0274] If the vehicle CPU 291 determines "Yes" in step 4305, the process proceeds to step 4310, where it determines whether the host vehicle 200 is traveling (moving forward or backward). If the vehicle CPU 291 determines "Yes" in step 4310, the process proceeds to step 4315, where it determines whether an obstacle has been detected. If the vehicle CPU 291 determines "Yes" in step 4315, the process proceeds to step 4320, where it transmits an obstacle vibration request signal S44 to the outside via the vehicle transmitting / receiving device 280. Next, the vehicle CPU 291 proceeds to step 4340.

[0275] On the other hand, if the vehicle CPU 291 determines "No" in step 4315, the process proceeds to step 4325, where it determines whether the host vehicle 200 is being braked. If the vehicle CPU 291 determines "Yes" in step 4325, the process proceeds to step 4330, where it transmits a braking vibration request signal S43 to the outside via the vehicle transmitting / receiving device 280. Next, the vehicle CPU 291 proceeds to step 4340.

[0276] On the other hand, if the vehicle CPU 291 determines “No” in step 4325, the process proceeds to step 4335, and transmits the traveling vibration request signal S41 to the outside via the vehicle transmitting / receiving device 280. Next, the vehicle CPU 291 proceeds to step 4340.

[0277] Also, if the vehicle CPU 291 determines “No” in step 4310 , it advances the process directly to step 4340 .

[0278] When the vehicle CPU 291 advances the process to step 4340, it determines whether the shift state of the transmission device 224 has been changed from a forward drive state to a reverse drive state or from a reverse drive state to a forward drive state. If the vehicle CPU 291 determines "Yes" in step 4340, it advances the process to step 4345 and transmits a shift change vibration request signal S42 to the outside via the vehicle transmitting / receiving device 280. Next, the vehicle CPU 291 advances the process to step 4395 and temporarily ends the process of this routine.

[0279] On the other hand, if the vehicle CPU 291 determines "No" in step 4340, it proceeds directly to step 4395, and temporarily ends the processing of this routine.

[0280] Also, if the vehicle CPU 291 determines "No" in step 4305, it proceeds directly to step 4395 and temporarily ends the processing of this routine.

[0281] On the other hand, when the vehicle CPU 291 executes the routine shown in Fig. 43, the terminal CPU 191 executes the routine shown in Fig. 44 at a predetermined calculation cycle. Therefore, at a predetermined timing, the process starts from step 4400 of the routine shown in Fig. 44, and proceeds to step 4405 to determine whether the remote driving application is running.

[0282] If the terminal CPU 191 judges "Yes" in step 4405, the process proceeds to step 4410, where it judges whether or not an obstacle vibration request signal S44 has been received. If the terminal CPU 191 judges "Yes" in step 4410, the process proceeds to step 4415, where it executes obstacle vibration processing. This causes the operation terminal 100 to vibrate in the obstacle vibration pattern. Next, the terminal CPU 191 proceeds to step 4420.

[0283] On the other hand, if the terminal CPU 191 determines “No” in step 4410 , it advances the process directly to step 4420 .

[0284] When the process proceeds to step 4420, the terminal CPU 191 determines whether or not the braking vibration request signal S43 has been received. When the terminal CPU 191 determines "Yes" in step 4420, the process proceeds to step 4425, and executes braking vibration processing. As a result, the operation terminal 100 is vibrated with the braking vibration pattern. Next, the terminal CPU 191 proceeds to step 4430.

[0285] On the other hand, if the terminal CPU 191 determines “No” in step 4420 , it advances the process directly to step 4430 .

[0286] When the terminal CPU 191 proceeds to step 4430, it determines whether or not it has received a running vibration request signal S41. When the terminal CPU 191 determines "Yes" in step 4430, it proceeds to step 4435 and executes running vibration processing. As a result, the operation terminal 100 is vibrated with the running vibration pattern. Next, the terminal CPU 191 proceeds to step 4440.

[0287] On the other hand, if the terminal CPU 191 determines “No” in step 4430 , it advances the process directly to step 4440 .

[0288] When the terminal CPU 191 advances the process to step 4440, it judges whether or not the shift change vibration request signal S42 has been received. When the terminal CPU 191 judges "Yes" in step 4440, it advances the process to step 4445 and executes the shift change vibration process. As a result, the operation terminal 100 is vibrated with the shift change vibration pattern. Next, the terminal CPU 191 advances the process to step 4495 and temporarily ends the process of this routine.

[0289] On the other hand, if the terminal CPU 191 determines "No" in step 4440, it advances the process directly to step 4495 and temporarily ends the process of this routine.

[0290] Also, if the terminal CPU 191 determines "No" in step 4405, it advances the process directly to step 4495 and temporarily ends the process of this routine.

[0291] The specific operation of the remote driving system 10 has been described above.

[0292] The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the present invention. [Explanation of symbols]

[0293] 10... remote driving system, 100... operation terminal, 110... terminal control device, 120... terminal display, 130... vibration device, 140... sound device, 180... terminal wireless receiver / transmitter, 190... terminal ECU, 200... host vehicle, 210... vehicle control device, 220... vehicle driving device, 280... vehicle wireless receiver / transmitter, 290... vehicle ECU

Claims

1. A remote driving application that performs wireless communication between a vehicle control device of a vehicle itself and a terminal control device of an operation terminal, transmits a continuous touch operation signal to the vehicle control device in response to an operation on a display of the operation terminal in which a user of the operation terminal touches the display and slides the finger across the display, and causes the vehicle control device to execute remote driving control for autonomously driving the vehicle itself, the remote driving application being installed on the terminal control device, A remote driving application configured to generate vibrations in the operation terminal or output a notification sound from the operation terminal in accordance with a control state of the host vehicle when the vehicle control device is executing the remote driving control, the vehicle control device is configured to perform autonomous driving and braking of the host vehicle and autonomously change a shift state of a transmission device of the host vehicle through the remote driving control on the condition that the continuous touch operation signal is received; The remote driving application is configured to, when the vehicle control device is executing the remote driving control, autonomously change the shift state from a forward drive state in which the vehicle is driven forward to a reverse drive state in which the vehicle is driven backward, or change the shift state from the reverse drive state to the forward drive state in a state in which the vehicle control device autonomously brakes and stops the vehicle, generate a vibration having a different vibration pattern in the operation terminal or output a notification sound having a different output pattern from the operation terminal when a control state of the host vehicle other than a control state of the host vehicle in which the vehicle control device autonomously changes the shift state from the forward drive state to the reverse drive state occurs and when a control state of the host vehicle other than a control state of the host vehicle in which the vehicle control device autonomously changes the shift state from the reverse drive state to the forward drive state occurs. Remote driving application.

2. 2. The remote driving application according to claim 1, When the vehicle control device is executing the remote driving control and the vehicle control device is driving the vehicle autonomously, a vibration having a predetermined driving vibration pattern is generated in the operation terminal, or a notification sound having a predetermined driving notification sound output pattern is output from the operation terminal, When the vehicle control device is executing the remote driving control, when the vehicle control device autonomously brakes and stops the vehicle and autonomously changes the shift state from the forward drive state to the reverse drive state or when the vehicle control device autonomously changes the shift state from the reverse drive state to the forward drive state, a vibration having a shift change vibration pattern different from the driving vibration pattern is generated in the operation terminal, or a notification sound having a shift change notification sound output pattern different from the driving notification sound output pattern is output from the operation terminal. It is configured as follows: Remote driving application.

3. 2. The remote driving application according to claim 1, When the vehicle control device is executing the remote driving control and the vehicle control device is autonomously braking the host vehicle, a vibration having a predetermined braking vibration pattern is generated in the operation terminal, or a notification sound having a predetermined braking notification sound output pattern is output from the operation terminal, When the vehicle control device is executing the remote driving control, when the vehicle control device autonomously brakes and stops the host vehicle and autonomously changes the shift state from the forward drive state to the reverse drive state or when the vehicle control device autonomously changes the shift state from the reverse drive state to the forward drive state, a vibration having a shift change vibration pattern different from the braking vibration pattern is generated in the operation terminal, or a notification sound having a shift change notification sound output pattern different from the braking notification sound output pattern is output from the operation terminal. It is configured as follows: Remote driving application.

4. 2. The remote driving application according to claim 1, When an obstacle is detected around the vehicle while the vehicle control device is executing the remote driving control, the vehicle control device is configured to generate vibrations in the operation terminal having a vibration pattern different from the vibration pattern generated in the operation terminal according to the control state of the vehicle, or to output a notification sound from the operation terminal having an output pattern different from the output pattern for outputting a notification sound from the operation terminal according to the control state of the vehicle. Remote driving application.

5. A remote driving system that performs wireless communication between a vehicle control device of a vehicle itself and a terminal control device of an operation terminal, transmits a continuous touch operation signal to the vehicle control device in response to an operation on a display of the operation terminal in which a user of the operation terminal touches the display and slides the finger across the display, and causes the vehicle control device to execute remote driving control for autonomously driving the vehicle itself, In a remote driving system configured to generate vibrations in the operation terminal or output a notification sound from the operation terminal in accordance with a control state of the vehicle when the vehicle control device is executing the remote driving control, the vehicle control device is configured to perform autonomous driving and braking of the host vehicle and autonomously change a shift state of a transmission device of the host vehicle through the remote driving control on the condition that the continuous touch operation signal is received; The terminal control device is configured to generate a vibration having a different vibration pattern from a control state of the host vehicle other than a control state of the host vehicle in which the vehicle control device has autonomously changed the shift state from a forward drive state in which the host vehicle is driven forward to a reverse drive state in which the host vehicle is driven backward, or to output a notification sound having a different output pattern from the operation terminal when the vehicle control device autonomously brakes and stops the host vehicle while the vehicle control device is executing the remote driving control, and when a control state of the host vehicle other than a control state of the host vehicle in which the vehicle control device has autonomously changed the shift state from the forward drive state to the reverse drive state occurs and when a control state of the host vehicle in which the vehicle control device has autonomously changed the shift state from the reverse drive state to the forward drive state occurs. Remote driving system.

Images