Long distance travel application and long distance travel system

Abstract

A remote travel application and a remote travel system are provided, which enable a user of an operation terminal to easily recognize that a gear position state of a transmission device of a self vehicle is changed during execution of remote travel control for autonomously traveling the self vehicle in a remote manner. The remote travel application is installed in the operation terminal for performing the remote travel control for autonomously traveling the self vehicle in response to an operation of the operation terminal, in order to perform wireless communication between the self vehicle (200) and the operation terminal (100). The remote travel application causes the operation terminal to vibrate or outputs a notification sound from the operation terminal in a different mode when a control state of the self vehicle is in a state where the gear position state of the transmission device is changed, than when the control state is in a state other than the state.

Description

TECHNICAL FIELD

[0001] The present application relates to a remote travel application program and a remote travel system. BACKGROUND

[0002] A remote travel application program and a remote travel system are known, which are used to cause an ego vehicle to autonomously travel and stop in a parking area by operating a terminal (operation terminal) such as a mobile phone to remotely operate a control device of the ego vehicle (see, for example, Patent Literature 1).

[0003] The conventional remote travel application program and the remote travel system (hereinafter referred to as a remote travel application program and the like) vibrate the operation terminal to notify a user of the operation terminal of the presence of an obstacle when the remote travel application program and the like detect an obstacle that will hinder the travel of the ego vehicle when the ego vehicle autonomously travels. In addition, the conventional remote travel application program and the like vibrate the operation terminal to notify the user of the operation terminal of the stop of the ego vehicle when the ego vehicle autonomously stops and the start of the ego vehicle when the stopped ego vehicle autonomously starts.

[0004] PRIOR ART DOCUMENTS

[0005] Patent Literature 1: Japanese Patent Application Laid-Open No. 2015-516772 SUMMARY

[0006] The conventional remote travel application program and the like change the frequency of vibration or the like when the operation terminal is vibrated to notify the presence of an obstacle, the stop of the ego vehicle, and the start of the ego vehicle, so that the user can distinguish and recognize whether there is an obstacle, the ego vehicle stops, or the ego vehicle starts, from the vibration of the operation terminal.

[0007] However, the presence of an obstacle, the stop of the ego vehicle, or the start of the ego vehicle is an event that can be easily recognized by a user who is operating the operation terminal while looking at the ego vehicle, but, for example, the change in the gear state of a transmission device for the ego vehicle to make a U-turn is an event that cannot be easily recognized by the user even if the user is looking at the ego vehicle. Therefore, it is advantageous for the user to easily recognize an event that cannot be easily recognized by the user by visually observing the ego vehicle using the operation terminal. However, the conventional remote travel application program and the like do not use the operation terminal to easily recognize such an event that cannot be easily recognized by the user by visually observing the ego vehicle.

[0008] An object of the present application is to provide a remote travel application and a remote travel system that enable a user of an operation terminal to easily recognize that a gear position state of a transmission device of a self vehicle is changed during execution of remote travel control that causes the self vehicle to autonomously travel in a remote manner.

[0009] The remote travel application according to the present application is installed in a terminal control device of an operation terminal in order to perform wireless communication between a vehicle control device of a self vehicle and the terminal control device, and causes the vehicle control device to execute remote travel control that causes the self vehicle to autonomously travel in response to an operation of the operation terminal. In addition, the remote travel application according to the present application is configured to cause the operation terminal to generate vibration or output a notification sound from the operation terminal in accordance with a control state of the self vehicle when the vehicle control device executes the remote travel control. Furthermore, the remote travel application according to the present application is configured to cause the operation terminal to generate vibration of a vibration pattern that is different from a vibration pattern when the control state of the self vehicle is in a state other than a state in which a gear position state of a transmission device of the self vehicle is changed, or output a notification sound of an output pattern that is different from an output pattern when the control state of the self vehicle is in the state other than the state in which the gear position state of the transmission device of the self vehicle is changed, when the vehicle control device executes the remote travel control.

[0010] According to the present application, vibration of different vibration patterns or notification sounds of different output patterns are generated or output from the operation terminal when the gear position state of the transmission device of the self vehicle is changed and when the gear position state of the transmission device of the self vehicle is not changed during execution of the remote travel control, and thus the user of the operation terminal can easily recognize that the gear position state of the transmission device of the self vehicle is changed by the vibration or the notification sound of the operation terminal.

[0011] In particular, the remote travel application according to the present application is configured to cause the operation terminal to generate vibration of a predetermined travel vibration pattern or output a notification sound of a predetermined travel notification sound output pattern from the operation terminal when the control state of the own vehicle is in a state in which the own vehicle travels, in a case in which the vehicle control device is caused to execute the remote travel control. In this case, the remote travel application according to the present application is configured to cause the operation terminal to generate vibration of a gear change vibration pattern different from the travel vibration pattern or output a notification sound of a gear change notification sound output pattern different from the travel notification sound output pattern from the operation terminal when the control state of the own vehicle is in a state in which a gear position state of a transmission device of the own vehicle is changed, in a case in which the vehicle control device is caused to execute the remote travel control.

[0012] According to the present application, the vibration pattern or the output pattern is different when the own vehicle is traveling and when the gear position state of the transmission device of the own vehicle is changed during execution of the remote travel control, and thus a user of the operation terminal can easily recognize whether the own vehicle is traveling or the gear position state of the transmission device of the own vehicle is changed by vibration or a notification sound of the operation terminal.

[0013] Further, the remote travel application according to the present application can also be configured to cause the operation terminal to generate vibration of a predetermined braking vibration pattern or output a notification sound of a predetermined braking notification sound output pattern from the operation terminal when the control state of the own vehicle is in a state in which the own vehicle is braked, in a case in which the vehicle control device is caused to execute the remote travel control. In this case, the remote travel application according to the present application can further also be configured to cause the operation terminal to generate vibration of a gear change vibration pattern different from the braking vibration pattern or output a notification sound of a gear change notification sound output pattern different from the braking notification sound output pattern from the operation terminal when the control state of the own vehicle is in a state in which a gear position state of a transmission device of the own vehicle is changed, in a case in which the vehicle control device is caused to execute the remote travel control.

[0014] According to the present application, the vibration pattern or the output pattern is different when the own vehicle is braked and when the gear position state of the transmission device of the own vehicle is changed during execution of the remote travel control, and thus a user of the operation terminal can easily recognize whether the own vehicle is braked or the gear position state of the transmission device of the own vehicle is changed.

[0015] Further, the remote travel application according to the present application can be configured to cause the operation terminal to generate vibration or output a notification sound from the operation terminal in a vibration pattern different from a vibration pattern generated from the operation terminal in accordance with the control state of the own vehicle, in the case where an obstacle is detected in the vicinity of the own vehicle when causing the vehicle control device to execute the remote travel control.

[0016] According to the present application, in the case where an obstacle is detected during the execution of the remote travel control, vibration in a vibration pattern different from a vibration pattern corresponding to the control state of the own vehicle is generated from the operation terminal, or a notification sound in an output pattern different from an output pattern corresponding to the control state of the own vehicle is output from the operation terminal, so that the user of the operation terminal can easily recognize the control state of the own vehicle and the presence of the obstacle through the vibration or the notification sound of the operation terminal.

[0017] Further, the remote travel system according to the present application is a system in which a vehicle control device of an own vehicle and a terminal control device of an operation terminal perform wireless communication, and the vehicle control device is caused to execute remote travel control in which the own vehicle autonomously travels in response to an operation of the operation terminal. Further, the remote travel system according to the present application is configured to cause the operation terminal to generate vibration or output a notification sound in accordance with the control state of the own vehicle when causing the vehicle control device to execute the remote travel control. Further, the terminal control device is configured to cause the operation terminal to generate vibration in a vibration pattern different from a vibration pattern when the control state of the own vehicle is in a state in which a gear position state of a transmission device of the own vehicle is changed, or output a notification sound in an output pattern different from an output pattern when the control state of the own vehicle is in a state in which the gear position state of the transmission device of the own vehicle is changed, in the case where the vehicle control device is caused to execute the remote travel control.

[0018] According to the present application, for the same reasons as described above, the user of the operation terminal can easily recognize that the gear position state of the transmission device of the own vehicle is changed through the vibration or the notification sound of the operation terminal.

[0019] The components of the present application are not limited to those described in the embodiments of the present application described below with reference to the drawings. Other objects, other features, and other advantages of the present application will become more apparent from the description of the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 FIG. 1 is a diagram showing a remote travel system to which an embodiment of the present application is applied.

[0021] Figure 2 FIG. 2 is a diagram showing a remote travel system to which a modification of the embodiment of the present application is applied.

[0022] Figure 3 FIG. 3 is a diagram showing a remote travel system to which another modification of the embodiment of the present application is applied.

[0023] Figure 4 FIG. 4 is a diagram showing a terminal control device that constitutes the remote travel system to which the embodiment of the present application is applied, and an operation terminal and the like in which the terminal control device is mounted.

[0024] Figure 5 FIG. 5 is a diagram showing a vehicle control device that constitutes the remote travel system to which the embodiment of the present application is applied, and a vehicle (own vehicle) and the like in which the vehicle control device is mounted.

[0025] Figure 6 FIG. 6 is a diagram showing a scene in which the own vehicle stops near a parking zone.

[0026] Figure 7 FIG. 7 is a diagram showing an operation terminal in which an initial screen image is displayed on a terminal display.

[0027] Figure 8 FIG. 8 is a diagram showing an operation terminal in which a garage entry selection image is displayed on a terminal display.

[0028] Figure 9 FIG. 9 is a diagram showing an operation terminal in which a parking zone selection image is displayed on a terminal display.

[0029] Figure 10 FIG. 10 is a diagram showing a target garage entry path.

[0030] Figure 11 FIG. 11 is a diagram showing an operation terminal in which a garage entry operation image is displayed on a terminal display.

[0031] Figure 12 FIG. 12 is a diagram showing an operation terminal in which a garage entry operation image is displayed on a terminal display.

[0032] Figure 13 FIG. 13 is a diagram showing a scene in which the own vehicle proceeds while right-turning by remote garage entry control.

[0033] Figure 14 FIG. 14 is a diagram showing an operation terminal in which a garage entry operation image is displayed on a terminal display.

[0034] Figure 15is a view showing a scenario in which the own vehicle is backed up while making a left turn by remote garage entry control.

[0035] Figure 16 is a view showing a scenario in which the own vehicle is backed up straight by remote garage entry control.

[0036] Figure 17 is a view showing a scenario in which the garage entry of the own vehicle based on remote garage entry control is completed.

[0037] Figure 18 is a view showing an operation terminal in which a garage entry operation image is displayed on a terminal display.

[0038] Figure 19 is a view showing a scenario in which the own vehicle is parked in a parking area.

[0039] Figure 20 is a view showing an operation terminal in which a garage exit direction selection image is displayed on a terminal display.

[0040] Figure 21 is a view showing a target garage exit path.

[0041] Figure 22 is a view showing an operation terminal in which a garage exit operation image is displayed on a terminal display.

[0042] Figure 23 is a view showing an operation terminal in which a garage exit operation image is displayed on a terminal display.

[0043] Figure 24 is a view showing a scenario in which the own vehicle is advanced straight by remote garage exit control.

[0044] Figure 25 is a view showing a scenario in which the garage exit of the own vehicle based on remote garage exit control is completed.

[0045] Figure 26 is a view showing an operation terminal in which a garage exit operation image is displayed on a terminal display.

[0046] Figure 27 is a view showing a time chart showing vibration and the like of an operation terminal during a period in which remote travel control is executed.

[0047] Figure 28 is a view showing a time chart showing vibration and the like of an operation terminal during a period in which remote travel control is executed.

[0048] Figure 29 is a flowchart showing a routine executed by a terminal control device to which an embodiment of the present application relates.

[0049] Figure 30is a flowchart showing a routine executed by the terminal control device to which the embodiment of the present application relates.

[0050] Figure 31 is a flowchart showing a routine executed by the terminal control device to which the embodiment of the present application relates.

[0051] Figure 32 is a flowchart showing a routine executed by the terminal control device to which the embodiment of the present application relates.

[0052] Figure 33 is a flowchart showing a routine executed by the terminal control device to which the embodiment of the present application relates.

[0053] Figure 34 is a flowchart showing a routine executed by the terminal control device to which the embodiment of the present application relates.

[0054] Figure 35 is a flowchart showing a routine executed by the terminal control device to which the embodiment of the present application relates.

[0055] Figure 36 is a flowchart showing a routine executed by the vehicle control device to which the embodiment of the present application relates.

[0056] Figure 37 is a flowchart showing a routine executed by the vehicle control device to which the embodiment of the present application relates.

[0057] Figure 38 is a flowchart showing a routine executed by the vehicle control device to which the embodiment of the present application relates.

[0058] Figure 39 is a flowchart showing a routine executed by the vehicle control device to which the embodiment of the present application relates.

[0059] Figure 40 is a flowchart showing a routine executed by the vehicle control device to which the embodiment of the present application relates.

[0060] Figure 41 is a flowchart showing a routine executed by the vehicle control device to which the embodiment of the present application relates.

[0061] Figure 42 is a flowchart showing a routine executed by the terminal control device to which the embodiment of the present application relates.

[0062] Figure 43 is a flowchart showing a routine executed by the vehicle control device to which the embodiment of the present application relates.

[0063] Figure 44 is a flowchart showing a routine executed by the terminal control device to which the embodiment of the present application relates.

[0064] Label Explanation

[0065] 10. Remote driving system; 100. Operating terminal; 110. Terminal control device; 120. Terminal display; 130. Vibration device; 140. Audio device; 180. Terminal wireless transceiver; 190. Terminal ECU; 200. Vehicle itself; 210. Vehicle control device; 220. Vehicle driving device; 280. Vehicle wireless transceiver; 290. Vehicle ECU Detailed Implementation

[0066] Hereinafter, the remote driving application and remote driving system according to embodiments of the present invention will be described with reference to the accompanying drawings.

[0067] The remote driving application described below relates to an application installed on a terminal control device (remote entry / exit application) for remote driving control (remote entry / exit control) that enables wireless communication between the control device of its own vehicle (vehicle control device) and the control device of the operating terminal (terminal control device), allowing the vehicle control device to perform remote driving control (remote entry / exit control) in response to an operation on the operating terminal, such as driving and parking (entering) the vehicle into the parking area autonomously, or driving and exiting the parking area autonomously in response to an operation on the operating terminal.

[0068] In addition, the remote driving system described below is a system for wireless communication between the vehicle control device and the terminal control device, enabling the vehicle control device to perform the aforementioned remote driving control (remote entry and exit control) (remote entry and exit system).

[0069] However, the remote driving application and remote driving system (hereinafter referred to as "remote driving application, etc.") involved in this invention are not limited to such remote entry and exit applications and systems. As long as it is an application or system that performs remote driving control in response to operation of an operating terminal to enable the vehicle to drive autonomously from outside the vehicle, the remote driving application, etc. involved in this invention can be any application or system.

[0070] like Figure 1 As shown, the remote driving system 10 according to the embodiments of the present invention includes at least a control device (terminal control device 110) mounted on an operating terminal 100 and a control device (vehicle control device 210) mounted on its own vehicle 200. The remote driving application according to the embodiments of the present invention is installed on the terminal control device 110.

[0071] In the remote driving system 10, the terminal control device 110 and the vehicle control device 210 can also be configured to directly transmit and receive various signals wirelessly, but such asFigure 2 As shown, the remote travel system 10 can be configured to include at least the terminal control device 110, the vehicle control device 210, and the Internet 300, and to perform transmission and reception of various signals between the terminal control device 110 and the vehicle control device 210 via the Internet 300.

[0072] In addition, as shown, the remote travel system 10 can be configured to include at least the terminal control device 110, the vehicle control device 210, and the Internet 300, and to perform transmission and reception of various signals between the terminal control device 110 and the vehicle control device 210 via the Internet 300. Figure 3 As shown, the remote travel system 10 can be configured to include at least the terminal control device 110, the vehicle control device 210, the Internet 300, and a server 301 provided on the Internet 300, and to perform transmission and reception of various signals between the terminal control device 110 and the vehicle control device 210 via the Internet 300 and the server 301.

[0073] <Operation terminal>

[0074] In the present example, the operation terminal 100 is a so-called smartphone that a person can carry as a telephone, but as long as it is a terminal that a user such as the driver DR of the own vehicle 200 can take out to the outside of the own vehicle 200, it can be, for example, a so-called smart key that a person can carry or a terminal dedicated to remote entry and exit, and the like.

[0075] As shown, the operation terminal 100 is provided with a terminal control device 110. The terminal control device 110 is provided with a terminal ECU 190. Figure 4 ECU is an abbreviation of Electronic Control Unit. The terminal ECU 190 is provided with a microcomputer as a main part. The terminal ECU 190 includes a CPU (terminal CPU 191), a ROM, a RAM, a nonvolatile memory, and an interface, and the like. The terminal CPU 191 realizes various functions by executing instructions (programs, routines) stored in the ROM. In particular, a remote travel application program (hereinafter referred to as "remote travel application") for causing the ECU (vehicle ECU 290) of the own vehicle 200 to perform remote entry and exit control described later is installed in the terminal ECU 190.

[0076] In addition, in the case where the remote travel system 10 is configured as shown in FIG. 1, the remote travel system 10 can be configured to cause the program stored in the server 301 to assume a part of the functions assumed by the remote travel application installed in the terminal ECU 190.

[0077] In addition, in the case where the remote travel system 10 is configured as shown in FIG. 1, the remote travel system 10 can be configured to update the remote travel application installed in the terminal ECU 190 by the server 301. Figure 3 Figure 3

[0078] ​​In addition, the operation terminal 100 is provided with a terminal display 120, a vibration device 130, an acoustic device 140, and a terminal transceiver 180.

[0079] The terminal display 120 is a device that displays an image. In addition, the terminal display 120 is manufactured so that a specific property thereof changes according to contact of an object to the terminal display 120. In particular, in the present example, the terminal display 120 is manufactured so that a specific property thereof changes according to contact of a human finger to the terminal display 120.

[0080] The terminal display 120 is electrically connected to a terminal ECU 190. The terminal ECU 190 is capable of causing the terminal display 120 to display various images. In addition, the terminal ECU 190 is capable of detecting a change in the specific property of the terminal display 120 when an object contacts the terminal display 120, and determining a portion of the terminal display 120 that the object contacts based on the detected change.

[0081] Further, the terminal ECU 190 is capable of determining a portion of the terminal display 120 that an object contacts based on a change in the specific property of the terminal display 120, and therefore the terminal display 120 is a device that provides information for determining a portion that an object contacts (contact information providing device). In the present example, the contact information providing device is the terminal display 120, but is not limited thereto, and may, for example, be a device that does not have a function of displaying an image but is capable of providing information for determining a portion that an object contacts.

[0082] The vibration device 130 is a device that generates vibration, and is, for example, a vibrator. The vibration device 130 is electrically connected to the terminal ECU 190. The terminal ECU 190 is capable of causing the operation terminal 100 to vibrate by operating the vibration device 130.

[0083] The acoustic device 140 is a device that outputs a notification sound such as a buzzer sound, and is, for example, a buzzer. The acoustic device 140 is electrically connected to the terminal ECU 190. The terminal ECU 190 is capable of outputting a notification sound via the acoustic device 140.

[0084] The terminal transceiver 180 is a device that receives signals that come from the outside of the operation terminal 100 in a wireless manner, and transmits signals to the outside of the operation terminal 100 in a wireless manner. The terminal transceiver 180 is electrically connected to the terminal ECU 190. The terminal ECU 190 is capable of transmitting various signals to the outside of the operation terminal 100 in a wireless manner via the terminal transceiver 180. In addition, the terminal ECU 190 is capable of receiving various signals that the ECU (vehicle ECU 290) of the own vehicle 200 has transmitted to the outside of the own vehicle 200 in a wireless manner via the vehicle transceiver 280 via the terminal transceiver 180.

[0085] Thus, the terminal control device 110 is configured to be capable of wireless communication with the vehicle control device 210.

[0086] < Own vehicle >

[0087] As shown in FIG. 1, the own vehicle 200 is equipped with the vehicle control device 210. Figure 5

[0088] The vehicle control device 210 is a device that performs remote entry control and remote exit control as remote entry / exit control. The remote entry control is control to stop the own vehicle 200 in a specified parking area by making the own vehicle 200 automatically travel in response to a signal transmitted from the operation terminal 100 to the outside in a wireless manner and automatically stop at a position within the specified parking area. The remote exit control is control to make the own vehicle 200 exit from the specified parking area by making the own vehicle 200 automatically travel in response to a signal transmitted from the operation terminal 100 to the outside in a wireless manner and automatically stop at a position outside the specified parking area.

[0089] As shown in FIG. 2, the vehicle control device 210 is equipped with a vehicle ECU 290. The vehicle ECU 290 is equipped with a microcomputer as a main part. The vehicle ECU 290 includes a CPU (vehicle CPU 291), a ROM, a RAM, a nonvolatile memory, and an interface, and the like. The vehicle CPU 291 realizes various functions by executing instructions (programs, routines) stored in the ROM. In particular, the vehicle ECU 290 stores a program for performing remote entry / exit control. Figure 5

[0090] Further, in a case where the remote travel system 10 is configured as shown in FIG. 3, the remote travel system 10 can be configured such that the program stored in the server 301 assumes a part of the processing assumed by the program for remote entry / exit control stored in the vehicle ECU 290. In addition, in a case where the remote travel system 10 is configured as shown in FIG. 4, the remote travel system 10 can be configured such that the program for remote entry / exit control stored in the vehicle ECU 290 is updated by the server 301. Figure 3 Figure 3

[0091] < Vehicle travel device >

[0092] Further, the own vehicle 200 is equipped with a vehicle travel device 220. The vehicle travel device 220 is a device that performs driving, braking, steering, and gear shifting of the own vehicle 200, and in this example, is equipped with a driving device 221, a braking device 222, a steering device 223, and a transmission device 224. ​​​​

[0093] <Driving device>

[0094] The driving device 221 is a device that outputs a driving force provided to the own vehicle 200 in order to make the own vehicle 200 travel, such as an internal combustion engine and / or a motor. The driving device 221 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 is able to control the driving force output from the driving device 221 by controlling the operation of the driving device 221.

[0095] <Braking device>

[0096] The braking device 222 is a device that outputs a braking force provided to the own vehicle 200 in order to brake the own vehicle 200, such as an oil hydraulic brake device. The braking device 222 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 is able to control the braking force output from the braking device 222 by controlling the operation of the braking device 222.

[0097] <Steering device>

[0098] The steering device 223 is a device that outputs a steering force provided to the own vehicle 200 in order to steer the own vehicle 200, such as a power steering device. The steering device 223 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 is able to control the steering force output from the steering device 223 by controlling the operation of the steering device 223.

[0099] <Transmission device>

[0100] The transmission device 224 is a device that switches whether or not to transmit the driving force output from the driving device 221 to the driving wheels of the own vehicle 200, and / or switches whether to transmit the driving force to the driving wheels to make the own vehicle 200 travel forward or to make the own vehicle 200 travel backward. Further, the transmission device 224 is a device that locks a gear of the transmission device 224 by hooking a claw-like member (a parking lock lever) to the gear so as not to rotate, thereby keeping the own vehicle 200 in a stopped state. Thus, the transmission device 224 also functions as a parking keeping device that keeps the own vehicle 200 in a stopped state.

[0101] The transmission device 224 operates in any one of a state in which the driving force is transmitted to the driving wheels to make the own vehicle 200 travel forward (a forward drive state SD), a state in which the driving force is transmitted to the driving wheels to make the own vehicle 200 travel backward (a backward drive state SR), a state in which the driving force is not transmitted to the driving wheels of the own vehicle 200 (a neutral state SN), and a state in which the own vehicle 200 is kept in a stopped state (a parking state SP) (a gear state).

[0102] The transmission device 224 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 is capable of setting the gear state of the transmission device 224 to any one of the forward drive state SD, the reverse drive state SR, the neutral state SN, and the parking state SP by controlling the operation of the transmission device 224.

[0103] < Parking maintenance device >

[0104] Further, the own vehicle 200 is equipped with a parking maintenance device 230. The parking maintenance device 230 is a device for maintaining the own vehicle 200 in a stopped state, and in this example, is a parking brake device 231. The parking brake device 231 can be either an electric parking brake device or a manual parking brake device. The parking brake device 231 is a device for maintaining the own vehicle 200 in a stopped state by providing a braking force to the wheels of the stopped own vehicle 200. In particular, the parking brake device 231 is a device for maintaining the own vehicle 200 in a stopped state by pressing a brake pad against a brake disc provided to the wheels of the stopped own vehicle 200 to provide a braking force to the wheels. In this example, the parking brake device 231 is an electric parking brake device, and thus is electrically connected to the vehicle ECU 290. The vehicle ECU 290 is capable of maintaining the own vehicle 200 in a stopped state by operating the parking brake device 231.

[0105] < Sensors and the like >

[0106] Further, the own vehicle 200 is equipped with an accelerator pedal 241, an accelerator pedal operation amount sensor 242, a brake pedal 243, a brake pedal operation amount sensor 244, a steering wheel 245, a steering shaft 246, a steering angle sensor 247, a steering torque sensor 248, a remote in-out request operator 249, a vehicle speed detection device 250, a shift lever 251, a gear position sensor 252, a surrounding information detection device 260, and a vehicle transceiver device 280.

[0107] < Accelerator pedal operation amount sensor >

[0108] The accelerator pedal operation amount sensor 242 is a sensor for detecting the operation amount of the accelerator pedal 241. The accelerator pedal operation amount sensor 242 is electrically connected to the vehicle ECU 290. The accelerator pedal operation amount sensor 242 transmits information on the detected operation amount of the accelerator pedal 241 to the vehicle ECU 290. The vehicle ECU 290 obtains the operation amount of the accelerator pedal 241 as the accelerator pedal operation amount AP on the basis of the information.

[0109] The vehicle ECU 290 obtains a required drive force (required drive torque) by calculation on the basis of the accelerator pedal operation amount AP and the running speed of the own vehicle 200 (own vehicle speed) except for the case where the remote entry-exit control described later is executed. The vehicle ECU 290 controls the operation of the drive device 221 so as to output the required drive force. In addition, the vehicle ECU 290, in the case where the remote entry-exit control described later is executed, decides a drive force required for the own vehicle 200 to run as desired by the remote entry-exit control, and controls the operation of the drive device 221 so as to output the drive force.

[0110] <Brake pedal operation amount sensor>

[0111] The brake pedal operation amount sensor 244 is a sensor that detects the operation amount of the brake pedal 243. The brake pedal operation amount sensor 244 is electrically connected to the vehicle ECU 290. The brake pedal operation amount sensor 244 transmits information of the detected operation amount of the brake pedal 243 to the vehicle ECU 290. The vehicle ECU 290 obtains the operation amount of the brake pedal 243 as a brake pedal operation amount BP on the basis of the information.

[0112] The vehicle ECU 290 obtains a required brake force (required brake torque) by calculation from the brake pedal operation amount BP except for the case where the remote entry-exit control described later is executed. The vehicle ECU 290 controls the operation of the brake device 222 so as to output the required brake force. In addition, the vehicle ECU 290, in the case where the remote entry-exit control described later is executed, decides a brake force required for the own vehicle 200 to brake as desired by the remote entry-exit control, and controls the operation of the brake device 222 so as to output the brake force.

[0113] <Steering angle sensor>

[0114] The steering angle sensor 247 is a sensor that detects the rotation angle of the steering shaft 246 with respect to the neutral position, and is electrically connected to the vehicle ECU 290. The steering angle sensor 247 transmits information of the detected rotation angle of the steering shaft 246 to the vehicle ECU 290. The vehicle ECU 290 obtains the rotation angle of the steering shaft 246 as a steering angle Θ on the basis of the information.

[0115] <Steering torque sensor>

[0116] The steering torque sensor 248 is a sensor that detects a torque input by the driver DR of the own vehicle 200 to the steering shaft 246 via the steering wheel 245, and is electrically connected to the vehicle ECU 290. The steering torque sensor 248 transmits information of the detected torque to the vehicle ECU 290. The vehicle ECU 290 acquires a torque input by the driver DR to the steering shaft 246 via the steering wheel 245 (driver input torque) on the basis of the information.

[0117] The vehicle ECU 290 acquires a required steering force (required steering torque) on the basis of the steering angle θ, the driver input torque, and a running speed of the own vehicle 200 (own vehicle speed) except for a case where the remote garage control described later is executed, and controls the operation of the steering device 223 so that the required steering torque is output from the steering device 223. In addition, the vehicle ECU 290 decides a steering force required for the own vehicle 200 to run as desired by the remote garage control in the case where the remote garage control described later is executed, and controls the operation of the steering device 223 so that the steering force is output.

[0118] <Remote Garage Request Operator>

[0119] The remote garage request operator 249 is an operator for the driver DR to operate in order to request the vehicle ECU 290 to execute the remote garage control described later, for example, a switch. The remote garage request operator 249 is electrically connected to the vehicle ECU 290. The remote garage request operator 249 transmits a predetermined signal to the vehicle ECU 290 when operated by the driver DR. The vehicle ECU 290 determines that the execution of the remote garage control is requested when receiving the predetermined signal.

[0120] <Speed Detection Device>

[0121] The speed detection device 250 is a device that detects a running speed of the own vehicle 200, for example, a wheel speed sensor. The speed detection device 250 is electrically connected to the vehicle ECU 290. The speed detection device 250 transmits information of the detected running speed of the own vehicle 200 to the vehicle ECU 290. The vehicle ECU 290 acquires a running speed of the own vehicle 200 (own vehicle speed V) on the basis of the information.

[0122] <Range Sensor>

[0123] The shift position sensor 252 is a sensor that detects the set position of the shift lever 251. The shift lever 251 is a device that is operated by the driver DR of the own vehicle 200, and the set position of the shift lever 251 that the driver DR can set is the forward position (drive range D), the reverse position (reverse range R), the neutral position (neutral range N), and the parking position (park range P). The shift position sensor 252 is electrically connected to the vehicle ECU 290. The shift position sensor 252 transmits a signal indicating the detected set position of the shift lever 251 to the vehicle ECU 290.

[0124] The shift position sensor 252 transmits a signal indicating that the set position of the shift lever 251 is the drive range D to the vehicle ECU 290 when the shift lever 251 is set to the drive range D. The vehicle ECU 290 controls the shift state of the transmission device 224 so that the transmission device 224 becomes the forward drive state SD when receiving this signal.

[0125] In addition, the shift position sensor 252 transmits a signal indicating that the set position of the shift lever 251 is the reverse range R to the vehicle ECU 290 when the shift lever 251 is set to the reverse range R. The vehicle ECU 290 controls the shift state of the transmission device 224 so that the transmission device 224 becomes the reverse drive state SR when receiving this signal.

[0126] In addition, the shift position sensor 252 transmits a signal indicating that the set position of the shift lever 251 is the neutral range N to the vehicle ECU 290 when the shift lever 251 is set to the neutral range N. The vehicle ECU 290 controls the shift state of the transmission device 224 so that the transmission device 224 becomes the neutral state SN when receiving this signal.

[0127] In addition, the shift position sensor 252 transmits a signal indicating that the set position of the shift lever 251 is the park range P to the vehicle ECU 290 when the shift lever 251 is set to the park range P. The vehicle ECU 290 controls the shift state of the transmission device 224 so that the transmission device 224 becomes the park state SP when receiving this signal.

[0128] Further, the vehicle ECU 290 controls (performs shift change) the shift state of the transmission device 224 as necessary in order to travel the own vehicle 200 as desired by the remote entry and exit control described later.

[0129] <Peripheral Information Detection Device>

[0130] The peripheral information detection device 260 is a device that detects the peripheral information of the own vehicle 200, and in this example, is provided with the radio wave sensor 261 and the image sensor 262.

[0131] <radio wave sensor>

[0132] The radio wave sensor 261 is a sensor that uses radio waves to detect information about an object existing in the periphery of the host vehicle 200, and is at least one of, for example, a radar sensor (millimeter wave radar or the like), an acoustic wave sensor such as an ultrasonic sensor (clearance sonar), and an optical sensor such as a laser radar (LiDAR). The radio wave sensor 261 is electrically connected to the vehicle ECU 290. The radio wave sensor 261 transmits radio waves, and receives radio waves (reflected waves) reflected by an object. The radio wave sensor 261 transmits information about the transmitted radio waves and the received radio waves (reflected waves) to the vehicle ECU 290. In other words, the radio wave sensor 261 detects an object existing in the periphery of the host vehicle 200, and transmits information about the detected object to the vehicle ECU 290. The vehicle ECU 290 can acquire information (periphery detection information IS) about an object existing in the periphery of the host vehicle 200 on the basis of the information (radio wave information IR or radio wave data). The object detected using the radio wave sensor 261 is, for example, a vehicle, a wall, a bicycle, a person, or the like.

[0133] <image sensor>

[0134] The image sensor 262 is a sensor that captures the periphery of the host vehicle 200, and is, for example, a camera. The image sensor 262 is electrically connected to the vehicle ECU 290. The image sensor 262 captures the periphery of the host vehicle 200, and transmits information about the captured image to the vehicle ECU 290.

[0135] <vehicle transceiver>

[0136] The vehicle transceiver 280 is a device that receives signals coming from the outside of the host vehicle 200 in a wireless manner, and transmits signals to the outside of the host vehicle 200 in a wireless manner. The vehicle transceiver 280 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can transmit various signals to the outside of the host vehicle 200 in a wireless manner via the vehicle transceiver 280. In addition, the vehicle ECU 290 can receive various signals transmitted from the terminal ECU 190 to the outside of the operation terminal 100 in a wireless manner via the terminal transceiver 180 via the vehicle transceiver 280.

[0137] <outline of operation of remote travel system>

[0138] Next, an outline of the operation of the remote travel system 10 will be described.

[0139] When the own vehicle 200 is stopped and is required to perform the remote entry-exit control, the vehicle control device 210 becomes a state of receiving a wireless signal from the outside, and determining whether the wireless signal is a wireless signal transmitted from the registered operation terminal. The registered operation terminal is an operation terminal registered to the vehicle control device 210 as an operation terminal for causing the vehicle control device 210 to perform the remote entry-exit control.

[0140] When the driver DR goes out of the own vehicle 200 and performs an application start operation on the terminal display 120 of the operation terminal 100, the terminal control device 110 starts the remote travel application in response to the application start operation, and displays an initial screen image G10 on the terminal display 120 as shown in FIG. 1. Figure 7

[0141] The application start operation is a touch operation on the terminal display 120 for causing the remote travel application to start. In this example, the touch operation on the terminal display 120 is an operation of touching the terminal display 120 with a finger (contact operation). The initial screen image G10 includes a remote entry-exit start image G11 and an application end image G15. The remote entry-exit start image G11 is an image displaying the text "remote entry-exit start". The application end image G15 is an image displaying the text "end". The remote entry-exit start image portion P11 and the application end image portion P15 are portions that accept a touch operation by a user (terminal user UR) of the operation terminal 100. The remote entry-exit start image portion P11 is a portion of the terminal display 120 that displays the remote entry-exit start image G11. The application end image portion P15 is a portion of the terminal display 120 that displays the application end image G15.

[0142] The terminal control device 110 transmits an application start signal S10 to the outside in a wireless manner when starting the remote travel application. The application start signal S10 is a signal indicating information for determining whether the operation terminal 100 is a registered operation terminal (for identifying the operation terminal 100), such as an ID.

[0143] The vehicle control device 210 determines whether the operation terminal 100 indicated by the application start signal S10 is a registered operation terminal when receiving the application start signal S10. The registered operation terminal is an operation terminal registered as an operation terminal for causing the vehicle control device 210 to perform the remote entry-exit control. The operation terminal 100 is a registered operation terminal, and therefore the vehicle control device 210 determines that the operation terminal 100 is a registered operation terminal when receiving the application start signal S10. In this case, the vehicle control device 210 performs the remote entry-exit control based on various signals transmitted to the outside in a wireless manner from the operation terminal 100 thereafter.

[0144] ​When the vehicle control device 210 determines that the operation terminal 100 is a registered operation terminal, the vehicle control device 210 causes the surrounding information detection device 260 and the vehicle travel device 220 and the like to operate or perform preparation for operating the vehicle travel device.

[0145] Further, the vehicle control device 210 can also be configured to, when determining that the operation terminal 100 is a registered operation terminal, perform determination of the type of assistance that can be executed based on the surrounding detection information IS, and transmit the determination result to the outside in a wireless manner. For example, as described later, the vehicle control device 210 is configured to be able to execute remote garage entry control and remote garage exit control, but can also be configured to, depending on the situation around the own vehicle 200, transmit a signal indicating that only remote garage entry control can be executed to the outside in a wireless manner in a case where remote garage exit control cannot be executed and only remote garage entry control can be executed. In this case, the terminal control device 110, when receiving the signal, adjusts the image displayed on the terminal display 120 so that only remote garage entry control can be selected.

[0146] Further, the remote travel system 10 described below is a system that can only select control to cause the own vehicle 200 to back up and park side by side in the specified parking area 31D as control to cause the own vehicle 200 to enter the specified parking area 31D, but can also be a system that, on the basis of the control, can also select control to cause the own vehicle 200 to advance and park side by side in the specified parking area 31D, control to cause the own vehicle 200 to advance and park in line in the specified parking area 31D, and control to cause the own vehicle 200 to back up and park in line in the specified parking area 31D.

[0147] In a case where a touch operation is applied to the remote garage entry and exit start image portion P11, as shown in Figure 8 In the present example, the touch operation effective as a touch operation applied to the remote garage entry and exit start image portion P11 is a flick operation. The flick operation is one of the slide operations, and is one of the touch operations in which a finger touching the terminal display 120 is slid in a straight line in the same direction on the terminal display 120 (a touch operation in which the finger is slid).

[0148] The entry selection image G21 is an image that displays the word "entry." The exit selection image G22 is an image that displays the word "exit." The application end image G25 is an image that displays the word "end." The entry selection image portion P21, the exit selection image portion P22, and the application end image portion P25 are portions that accept a touch operation by the terminal user UR. The entry selection image portion P21 is a portion of the terminal display 120 that displays the entry selection image G21. The exit selection image portion P22 is a portion of the terminal display 120 that displays the exit selection image G22. The application end image portion P25 is a portion of the terminal display 120 that displays the application end image G25.

[0149] <Remote entry control>

[0150] The terminal control device 110 wirelessly transmits an entry selection signal S11 to the outside in a case where a touch operation is applied to the entry selection image portion P21. The entry selection signal S11 is a signal that indicates that a touch operation is applied to the entry selection image portion P21. In the present example, a tap operation is a touch operation that is effective as a touch operation applied to the entry selection image portion P21. The tap operation is one of touch operations in which the terminal display 120 is touched with a finger and then immediately released.

[0151] The vehicle control device 210 detects a zone in which the own vehicle 200 can be parked (parkable zone 31C) on the basis of the surrounding detection information IS when the entry selection signal S11 is received. For example, as shown in Fig. 6, in a case where the own vehicle 200 is stopped in the vicinity of three parking zones 311 to 313 in which no other vehicle is parked, the vehicle control device 210 detects those parking zones 311 to 313 as parkable zones 311C to 313C. Figure 6 Figure 6 The example shown in Fig. 6 is an example in which the parking zones 311 to 313 exist on the left side of the own vehicle 200, but in a case where one or more parking zones exist on the right side of the own vehicle 200, the vehicle control device 210 also detects those parking zones as parkable zones. Further, the reference numeral 400 in the figure indicates a zone line that zones the parking zones 31.

[0152] The vehicle control device 210 acquires the position of the own vehicle 200 with respect to the detected parkable zone 31C and the like when the parkable zone 31C is detected.

[0153] ​When the vehicle control device 210 detects a parking zone 31C and obtains the position of its own vehicle 200 relative to the parking zone 31C, it wirelessly transmits a designated parking zone signal S12 and a vehicle position signal S13 to the outside. The designated parking zone signal S12 is one of the signals indicating information related to the driving state of its own vehicle 200, specifically the parking zone 31C. The vehicle position signal S13 is another signal indicating information related to the driving state of its own vehicle 200, specifically the position of its own vehicle 200 relative to the parking zone 31C.

[0154] When the terminal control device 110 receives the designated parking zone signal S12 and the vehicle position signal S13, if Figure 9 As shown, the parking area selection image G30 is displayed on the terminal display 120. The parking area selection image G30 includes a parking area image G31, a vehicle image G32, a selection confirmation image G34, an application end image G35, and an entry / exit reselection image G36.

[0155] Parkable zone image G31 is an image showing the parkable zone 31C. Figure 9 It shows the relationship with Figure 6 The example shown corresponds to the example, therefore, Figure 9 The parking zone image G31 shown includes parking zone images G311 to G313, which respectively display parking zone 311C to 313C. Vehicle image G32 displays the vehicle itself 200. Selection confirmation image G34 displays the word "Confirm". Application completion image G35 displays the word "Complete". Re-selection for entry / exit image G36 displays the word "Re-selection for entry / exit".

[0156] Additionally, the parking zone image portion P31, the selection confirmation image portion P34, the application end image portion P35, and the entry / exit reselection image portion P36 are portions that accept touch operations from the terminal user UR. The parking zone image portion P31 is the portion of the terminal display 120 that displays the parking zone image G31. Figure 9 Showing with Figure 6 The example shown corresponds to the example, therefore, Figure 9The illustrated parkable zone image portion P31 includes three portions (parkable zone image portions P311 to P313) of the terminal display 120 that respectively display three parkable zone images G311 to G313. The selection determination image portion P34 is a portion of the terminal display 120 that displays the selection determination image G34. The application end image portion P35 is a portion of the terminal display 120 that displays the application end image G35. The entry / exit re-selection image portion P36 is a portion of the terminal display 120 that displays the entry / exit re-selection image G36.

[0157] The terminal control device 110 wirelessly transmits a specified parkable zone signal S14 (travel type determination signal) to the outside in a case where a touch operation is applied to the parkable zone image portion P31, and then a touch operation is applied to the selection determination image portion P34. In Figure 9 In the illustrated example, the terminal control device 110 wirelessly transmits the specified parkable zone signal S14 to the outside in a case where a touch operation is applied to any one of the three parkable zone image portions P311 to P313, and then a touch operation is applied to the selection determination image portion P34. The specified parkable zone signal S14 is a signal that indicates the parkable zone 31C specified by the terminal user UR applying a touch operation to the parkable zone image portion P31 to park the own vehicle 200. In addition, the specified parkable zone signal S14 is also a travel type determination signal that determines the type of travel of the own vehicle 200 based on the vehicle travel processing described later.

[0158] Further, in the present example, the touch operation effective as the touch operation applied to the parkable zone image portion P31 is a long press operation. The long press operation can also be referred to as a long tap operation, and is one of the touch operations in which the terminal display 120 is continuously touched for a relatively long time with a finger.

[0159] In addition, the parkable zone selection image G30 can also include a parking direction selection image that is an image for selecting whether to advance the own vehicle 200 to park in the specified parkable zone 31D or to reverse the own vehicle 200 to park in the specified parkable zone 31D. In this case, the terminal control device 110 wirelessly transmits the specified parkable zone signal S14 to the outside in a case where a touch operation is applied to the parkable zone image portion P31 and a touch operation is applied to the parking direction selection image, and then a touch operation is applied to the selection determination image portion P34.

[0160] The vehicle control device 210 sets the parkable zone 31C specified by the terminal user UR as the specified parkable zone 31D when the specified parkable zone signal S14 is received. In Figure 6In the example shown, the vehicle control device 210 sets the parking available zone 31 C (312C) specified by the terminal user UR among the parking available zones 311C to 313C to the specified parking zone 31D when receiving the specified parking zone signal S14.

[0161] The vehicle control device 210 starts the remote parking control when the specified parking zone 31D is set. The vehicle control device 210 sets the target parking-in path Rin_tgt as shown when starting the remote parking control. The target parking-in path Rin_tgt is a path along which the own vehicle 200 travels in order to park the own vehicle 200 in the specified parking zone 31D. Further, the vehicle control device 210 sets the target parking-in path Rin_tgt so that the own vehicle 200 travels along the target parking-in path Rin_tgt. Figure 10 Figure 10 The example shown in FIG. 6 shows the case where the parking available zone 312C is set to the specified parking zone 31D. Figure 6

[0162] The vehicle control device 210 transmits the parking-in remaining distance signal S15 to the outside in a wireless manner when the target parking-in path Rin_tgt is set. The parking-in remaining distance signal S15 is one of signals indicating information about the travel state of the own vehicle 200, and is a signal indicating the distance (parking-in remaining distance Din) between the specified parking zone 31D, which should be reached by the own vehicle 200 through the remote parking control, and the target parking-in point Pin_tgt. The vehicle control device 210 acquires the parking-in remaining distance Din on the basis of the surrounding detection information IS.

[0163] On the other hand, the terminal control device 110 displays the parking-in operation image G40 on the terminal display 120 when applying the touch operation to the parking available zone image portion P31 as shown in FIG. 7. Figure 11 Figure 11 The example shown in FIG. 6 shows the case where the specified parking zone 31D is the parking available zone 31C (312C) located on the left side of the own vehicle 200. Figure 10

[0164] ​​​​The entry operation image G40 includes a designated parking zone image G41, a vehicle image G42, an entry remaining distance image G43, an entry travel operation image G44, an application end image G45, and an entry / exit selection image G46. The designated parking zone image G41 is an image that displays the designated parking zone 31D. The vehicle image G42 is an image that displays the own vehicle 200. The entry remaining distance image G43 is an image that displays the entry remaining distance Din. The entry travel operation image G44 is an image that zones a region of a predetermined area. The application end image G45 is an image that displays the word "end". The entry / exit selection image G46 is an image that displays the word "entry / exit selection". In addition, the entry travel operation image portion P44, the application end image portion P45, and the entry / exit selection image portion P46 are portions that accept a touch operation by the terminal user UR. The entry travel operation image portion P44 is a portion of the terminal display 120 that displays the entry travel operation image G44. The application end image portion P45 is a portion of the terminal display 120 that displays the application end image G45. The entry / exit selection image portion P46 is a portion of the terminal display 120 that displays the entry / exit selection image G46.

[0165] When the entry remaining distance signal S15 is received, the terminal control device 110 displays the entry remaining distance image G43 that displays the entry remaining distance Din indicated by the entry remaining distance signal S15 on the terminal display 120.

[0166] Also, the terminal control device 110 wirelessly transmits a continuous touch operation signal S16 (a predetermined operation signal) to the outside during a predetermined operation is applied to the entry travel operation image portion P44. The continuous touch operation signal S16 is a signal that indicates that a predetermined operation is applied to the entry travel operation image portion P44. In this example, the predetermined operation applied to the entry travel operation image portion P44 is a touch operation, and the touch operation that is effective as the touch operation is a continuous touch operation. The continuous touch operation is one of the slide operations, and is an operation (a slide operation) in which the terminal user UR keeps a state in which a finger touches the entry travel operation image portion P44 constant, and slides the finger on the entry travel operation image portion P44 as indicated by a line L in the figure. Figure 12 Therefore, the terminal user UR touches the entry travel operation image portion P44 with the finger to perform a touch operation, but in a case in which the finger stops on the entry travel operation image portion P44 without performing a continuous touch operation, the terminal control device 110 does not transmit the continuous touch operation signal S16.

[0167] Further, the terminal control device 110 can also be configured to transmit, to the outside, a signal indicating the position of the in-warehouse travel operation image portion P44 to which the touch operation is applied, in a wireless manner, instead of the continuous touch operation signal S16. In this case, the vehicle control device 210 that receives this signal determines whether the continuous touch operation is being applied to the in-warehouse travel operation image portion P44.

[0168] The vehicle control device 210, when receiving the continuous touch operation signal S16, performs a vehicle travel process in which, after controlling the operation of the transmission device 224 so that the transmission device 224 becomes the forward drive state SD, the operation of the vehicle travel device 220 is controlled so that the own vehicle 200 advances while turning right along the target in-warehouse path Rin_tgt, and is stopped at the time point when a predetermined distance is advanced. Figure 13

[0169] The vehicle control device 210 performs a vehicle travel process so that the own vehicle 200 travels along the target in-warehouse path Rin_tgt during the period when the continuous touch operation signal S16 is received.

[0170] Further, the vehicle control device 210 continuously transmits, to the outside, the vehicle position signal S13, the in-warehouse remaining distance signal S15, and the travel direction signal S17 in a wireless manner during the period from when the reception of the continuous touch operation signal S16 is started until when the parking of the own vehicle 200 to the specified parking zone 31D is completed. The travel direction signal S17 is one of the signals indicating information related to the travel state of the own vehicle 200, and is a signal indicating the travel direction of the own vehicle 200.

[0171] The terminal control device 110, when receiving the vehicle position signal S13, displays, as shown in Figure 14 Figure 14 The vehicle image G42 shown in Figure 13

[0172] Further, the terminal control device 110, when receiving the in-warehouse remaining distance signal S15, displays, as shown in Figure 14

[0173] Further, the terminal control device 110, when receiving the travel direction signal S17, displays, as shown in Figure 14 ​​​​As shown, the travel direction image G47, which indicates the travel direction of the vehicle 200 as represented by the travel direction signal S17, is displayed in the portion near the vehicle image G42 on the terminal display 120.

[0174] Furthermore, when the terminal user UR removes their finger from the parking maneuver image portion P44, the terminal control device 110 stops wirelessly transmitting the continuous touch operation signal S16. In this case, the vehicle control device 210 stops receiving the continuous touch operation signal S16. When the vehicle control device 210 stops receiving the continuous touch operation signal S16, it performs a parking process that temporarily stops the vehicle 200 using the braking device 222.

[0175] Vehicle control device 210 when Figure 13 When the vehicle 200 stops while turning right and moving forward as shown, the following vehicle driving process is performed: After changing the gear state of the transmission device 224 from forward drive state SD to reverse drive state SR in order to turn the vehicle 200 around, as shown... Figure 15 As shown, the operation of the vehicle driving device 220 is controlled so that the vehicle 200 reverses while turning left, and the steering angle θ is gradually reduced. When the forward and backward direction of the vehicle 200 becomes parallel to the longitudinal direction of the designated parking area 31D, the steering angle θ becomes zero.

[0176] Then, the vehicle control unit 210 controls the operation of the vehicle driving device 220 to achieve the following: Figure 16 The vehicle is driven in a manner that causes it to reverse directly, as shown in the diagram.

[0177] Furthermore, in this example, while the vehicle control device 210 is driving its own vehicle 200 via remote parking control, it controls the driving force output from the drive device 221 and the braking force output from the braking device 222 so that its own vehicle speed V is below a predetermined speed limit Vlimit.

[0178] And, as Figure 17 As shown, when its own vehicle 200 arrives at the target parking location Pin_tgt, the vehicle control device 210 stops its own vehicle 200 through a parking process. Furthermore, it maintains the vehicle 200 in a stopped state through a parking hold process, stopping the operation of devices such as the surrounding information detection device 260 and the vehicle driving device 220, thus ending the remote parking control. Therefore, the parking of its own vehicle 200 in the designated parking area 31D is completed.

[0179] Further, in the present example, the parking processing is processing of stopping the own vehicle 200 by the brake device 222. In addition, the parking hold processing is processing of holding the own vehicle 200 in a stopped state by operating the parking brake device 231 or controlling the operation of the transmission device 224 so that the transmission device 224 becomes a parking state SP to hold the own vehicle 200 in a stopped state.

[0180] Further, the vehicle control device 210 wirelessly transmits a parking completion signal S18 to the outside in a case where the parking (stopping) of the own vehicle 200 has been completed. The parking completion signal S18 is a signal indicating that the parking (stopping) of the own vehicle 200 has been completed.

[0181] Further, the vehicle control device 210 detects an obstacle on the basis of the surrounding detection information IS during the period in which the own vehicle 200 is running by the remote parking control, stops the own vehicle 200 by the parking processing in a case where it is necessary to stop the own vehicle 200 due to the own vehicle 200 becoming about to contact with the obstacle or the like, resets the target parking path Rintgt (sets the target parking path Rintgt again), and restarts the running of the own vehicle 200.

[0182] The terminal control device 110, when receiving the parking completion signal S18, displays a parking completion image G49 on the terminal display 120 as shown in FIG. 19. The parking completion image G49 is an image indicating that the parking of the own vehicle 200 has been completed. Figure 18

[0183] <Remote Unparking Control>

[0184] The terminal control device 110 wirelessly transmits an unparking selection signal S21 to the outside in a case where a touch operation is applied to the unparking selection image portion P22. The unparking selection signal S21 is a signal indicating that a touch operation is applied to the unparking selection image portion P22. In the present example, the touch operation to be adopted as a valid touch operation applied to the unparking selection image portion P22 is a tap operation.

[0185] The vehicle control device 210, when receiving the unparking selection signal S21, acquires the position of the own vehicle 200 with respect to the stopping area (the current stopping area 31N) in which the own vehicle 200 is parked and the like on the basis of the surrounding detection information IS. For example, in a case where the own vehicle 200 is parked as shown in FIG. 18, the stopping area 31 is the current stopping area 31N, and the vehicle control device 210 acquires the position of the own vehicle 200 with respect to the current stopping area 31N and the like. Figure 19

[0186] ​​The vehicle control device 210 externally transmits a vehicle position signal S23 in a wireless manner when acquiring the position of the own vehicle 200 with respect to the current parking area 31N or the like. The vehicle position signal S23 is a signal indicating information about the position of the own vehicle 200 with respect to the current parking area 31N or the like.

[0187] The terminal control device 110, when receiving the vehicle position signal S23, displays the departure direction selection image G50 on the terminal display 120 as shown in Figure 20 The departure direction selection image G50 includes a current parking area image G51, a vehicle image G52, a departure direction image G53, a selection determination image G54, an application end image G55, and an entry / exit reselection image G56.

[0188] The current parking area image G51 is an image that displays the current parking area 31N. The vehicle image G52 is an image that displays the own vehicle 200. The departure direction image G53 is an image that displays a direction in which the own vehicle 200 can depart (departure direction). Figure 20 Examples corresponding to the examples shown in Figure 19 Thus, the departure direction image G53 shown in Figure 20 includes an image that displays a forward direction (forward direction image G531) and an image that displays a reverse direction (reverse direction image G532). The selection determination image G54 is an image that displays the word "OK". The application end image G55 is an image that displays the word "End". The entry / exit reselection image G56 is an image that displays the word "Entry / Exit Re-selection".

[0189] In addition, the departure direction image portion P53, the selection determination image portion P54, the application end image portion P55, and the entry / exit reselection image portion P56 are portions that accept a touch operation by the terminal user UR. The departure direction image portion P53 is a portion of the terminal display 120 that displays the departure direction image G53. Figure 20 Examples corresponding to the examples shown in Figure 19 Thus, the departure direction image portion P53 shown in Figure 20 includes two portions (forward direction image portion P531 and reverse direction image portion P532) of the terminal display 120 that respectively display the forward direction image G531 and the reverse direction image G532. The selection determination image portion P54 is a portion of the terminal display 120 that displays the selection determination image G54. The application end image portion P55 is a portion of the terminal display 120 that displays the application end image G55. The entry / exit reselection image portion P56 is a portion of the terminal display 120 that displays the entry / exit reselection image G56.

[0190] The terminal control device 110 wirelessly transmits a specified exit direction signal S22 to the outside in a case where a touch operation is applied to the exitable direction image portion P53, and then a touch operation is applied to the selection determination image portion P54. The specified exit direction signal S22 is a signal that indicates the exitable direction in which the own vehicle 200 is to exit, which is specified by the terminal user UR applying a touch operation to the exitable direction image portion P53. In addition, the specified exit direction signal S22 is also a travel type determination signal that determines the type of travel of the own vehicle 200 based on the vehicle travel processing described later. Figure 20 In the example illustrated in FIG. 6, the terminal control device 110 wirelessly transmits the specified exit direction signal S22 to the outside in a case where a touch operation is applied to either one of the forward direction image portion P531 and the reverse direction image portion P532, and then a touch operation is applied to the selection determination image portion P54. The specified exit direction signal S22 is a signal that indicates the exitable direction in which the own vehicle 200 is to exit, which is specified by the terminal user UR applying a touch operation to the exitable direction image portion P53. In addition, the specified exit direction signal S22 is also a travel type determination signal that determines the type of travel of the own vehicle 200 based on the vehicle travel processing described later.

[0191] Further, in the present example, the touch operation that is effective as a touch operation applied to the exitable direction image portion P53 is a long press operation.

[0192] The vehicle control device 210 sets the exitable direction specified by the terminal user UR as the specified exit direction when the specified exit direction signal S22 is received. In Figure 19 In the example illustrated in FIG. 6, the vehicle control device 210 sets the exitable direction (the forward direction) specified by the terminal user UR as the specified exit direction when the specified exit direction signal S22 is received.

[0193] The vehicle control device 210 starts the remote exit control when the specified exit direction is set. The vehicle control device 210 sets the target exit path Rout_tgt when the remote exit control is started, as illustrated in Figure 21 The target exit path Rout_tgt is a path along which the own vehicle 200 is to travel in order to cause the own vehicle 200 to exit from the current parking area 31N in the specified exit direction. Figure 21 An example in which the forward direction is set as the specified exit direction is illustrated in Figure 19 An example in which the forward direction is set as the specified exit direction is illustrated in

[0194] The vehicle control device 210 transmits the exit remaining distance signal S25 to the outside in a wireless manner when the target exit route Rout_tgt is set. The exit remaining distance signal S25 is a signal that indicates the distance (exit remaining distance Dout) between the current parking zone 31N outside the point (target exit point Pout_tgt) to which the own vehicle 200 is to be brought by the remote exit control. The vehicle control device 210 acquires the exit remaining distance Dout on the basis of the surrounding detection information IS.

[0195] On the other hand, the terminal control device 110 displays the exit operation image G60 on the terminal display 120 when a touch operation is applied to the exit direction image portion P53, as shown in Figure 22 Figure 22 An example corresponding to the example shown in Figure 21 is shown. An example in which the specified exit direction is the forward direction is shown.

[0196] The exit operation image G60 includes a current parking zone image G61, a vehicle image G62, an exit remaining distance image G63, an exit travel operation image G64, an application end image G65, an entry and exit reselection image G66, and an exit direction image G67. The current parking zone image G61 is an image that displays the current parking zone 31N. The vehicle image G62 is an image that displays the own vehicle 200. The exit remaining distance image G63 is an image that displays the exit remaining distance Dout. The exit travel operation image G64 is an image that divides a region of a predetermined area. The application end image G65 is an image that displays the word "end". The entry and exit reselection image G66 is an image that displays the word "entry and exit reselection". The exit direction image G67 is an image that displays the specified exit direction.

[0197] In addition, the exit travel operation image portion P64, the application end image portion P65, and the entry and exit reselection image portion P66 are portions that accept a touch operation by the terminal user UR. The exit travel operation image portion P64 is a portion of the terminal display 120 that displays the exit travel operation image G64. The application end image portion P65 is a portion of the terminal display 120 that displays the application end image G65. The entry and exit reselection image portion P66 is a portion of the terminal display 120 that displays the entry and exit reselection image G66.

[0198] The terminal control device 110 displays the exit remaining distance image G63 that displays the exit remaining distance Dout indicated by the exit remaining distance signal S25 on the terminal display 120 when the exit remaining distance signal S25 is received.

[0199] ​Furthermore, during the period when the terminal control device 110 applies a touch operation to the outbound driving operation image portion P64, it wirelessly transmits a continuous touch operation signal S26 (predetermined operation signal) to the outside. The continuous touch operation signal S26 indicates that a predetermined operation has been applied to the outbound driving operation image portion P64. In this example, the predetermined operation applied to the outbound driving operation image portion P64 is a touch operation, and the effective touch operation as this touch operation is a continuous touch operation. A continuous touch operation is one type of swiping operation, and it is an operation (swiping operation) where the terminal user UR maintains the state of touching the outbound driving operation image portion P64 with their finger, such as... Figure 23 As shown by line L, the finger slides on the outbound driving operation image portion P64. Therefore, if the terminal user UR touches the outbound driving operation image portion P64 to perform a touch operation, but the finger stops on the outbound driving operation image portion P64 and no continuous touch operation is performed, the terminal control device 110 does not send a continuous touch operation signal S26.

[0200] Alternatively, the terminal control device 110 may be configured to wirelessly transmit a signal indicating the position of the outbound driving operation image portion P64 to which a touch operation has been applied, instead of the continuous touch operation signal S26. In this case, the vehicle control device 210, upon receiving the signal, determines whether a continuous touch operation is being applied to the outbound driving operation image portion P64.

[0201] When the vehicle control unit 210 receives the continuous touch operation signal S26, it performs the following vehicle driving process: after setting the gear state of the transmission unit 224 to the forward drive state SD, as follows... Figure 24 As shown, the operation of the vehicle driving device 220 is controlled so that the vehicle 200 moves along the target exit path Rout_tgt, and stops when the vehicle 200 reaches the target exit location Pout_tgt.

[0202] During the period of receiving continuous touch operation signals S26, the vehicle control device 210 performs vehicle driving processing to make its own vehicle 200 drive along the target outbound path Rout_tgt.

[0203] In addition, during the period from the start of receiving continuous touch operation signals S26 until the vehicle 200 completes its exit from the current parking area 31N, the vehicle control device 210 continuously transmits vehicle position signals S23 and remaining exit distance signals S25 wirelessly to the outside.

[0204] When the terminal control device 110 receives the vehicle position signal S23, if... Figure 23As shown, the vehicle image G62 is displayed in the portion of the terminal display 120 corresponding to the position of the current parking zone 31N indicated by the vehicle position signal S23 with respect to the own vehicle 200. Figure 23 As shown, the vehicle image G62 is displayed in the portion of the terminal display 120 corresponding to the position of the current parking zone 31N indicated by the vehicle position signal S23 with respect to the own vehicle 200. Figure 24 As shown, the vehicle image G62 is displayed in the portion of the terminal display 120 corresponding to the position of the current parking zone 31N indicated by the vehicle position signal S23 with respect to the own vehicle 200.

[0205] Further, the terminal control device 110, upon receiving the egress remaining distance signal S25, displays, as shown in Figure 23 the terminal display 120, an egress remaining distance image G63 indicating the egress remaining distance Dout indicated by the egress remaining distance signal S25.

[0206] Further, the terminal control device 110, in the case where the terminal user UR removes the finger from the egress travel operation image portion P64, stops the wireless transmission of the continuous touch operation signal S26 to the outside. In this case, the vehicle control device 210 becomes not to receive the continuous touch operation signal S26. The vehicle control device 210, upon becoming not to receive the continuous touch operation signal S26, executes a stop processing of temporarily stopping the own vehicle 200 by the brake device 222.

[0207] Further, in the present example, the vehicle control device 210, during the travel of the own vehicle 200 by the remote egress control, controls the driving force output from the drive device 221 and the braking force output from the brake device 222 so that the own vehicle speed V becomes equal to or lower than the predetermined limit vehicle speed Vlimit.

[0208] Further, as shown in Figure 25 the terminal display 120, an egress remaining distance image G63 indicating the egress remaining distance Dout indicated by the egress remaining distance signal S25.

[0209] Further, the vehicle control device 210, in the case where the egress of the own vehicle 200 has been completed, wirelessly transmits an egress completion signal S28 to the outside. The egress completion signal S28 is a signal indicating that the egress of the own vehicle 200 has been completed.

[0210] The terminal control device 110, upon receiving the egress completion signal S28, displays, as shown in Figure 26 the terminal display 120, an egress completion image G69. The egress completion image G69 is an image indicating that the egress of the own vehicle 200 has been completed.

[0211] Furthermore, the vehicle control device 210 is configured such that, during the execution of remote outbound control, if it is determined that the vehicle 200 needs to turn back in order to allow its own vehicle 200 to leave the warehouse, the gear state of the transmission device 224 is changed to allow the vehicle 200 to turn back.

[0212] <Application End, etc.>

[0213] Furthermore, when the terminal control device 110 applies a touch operation to the application end image portion P15, etc. (i.e., application end image portions P15, P25, P35, P45, P55, or P65), it terminates the remote driving application and wirelessly sends a control end command signal S30 to the outside. The control end command signal S30 is a signal that instructs the termination of remote entry and exit control.

[0214] In addition, when the terminal control device 110 applies a touch operation to the in / out selection image portion P36 (i.e., the in / out selection image portions P36, P46, P56 or P66), it displays the in / out selection image G20 on the terminal display 120 and wirelessly sends a control end command signal S30 to the outside.

[0215] When the vehicle control device 210 receives the control end command signal S30, it stops its own vehicle 200 through parking processing. Furthermore, it keeps its own vehicle 200 in a stopped state through parking holding processing, stops the operation of devices such as the surrounding information detection device 260 and the vehicle driving device 220, and ends the remote parking control when performing remote parking control and ends the remote parking control when performing remote parking control.

[0216] Furthermore, in the event that a large number of screen changes are performed continuously during the activation of the remote driving application, the terminal control device 110 may also be configured to display an image on the terminal display 120 that specifies the portion of the terminal display 120 to which a touch operation should be applied in order to return the screen to the previous screen.

[0217] <Vibration Treatment>

[0218] As explained above, when the terminal user UR makes his / her own vehicle 200 drive autonomously through remote driving control (remote entry and exit control), he / she continuously applies touch operation to the terminal display 120 of the operating terminal 100 with his / her finger. However, in order to ensure the driving safety of his / her own vehicle 200, the terminal user UR is also required not to apply touch operation to the terminal display 120 while looking at the terminal display 120 at the same time, but to apply touch operation to the terminal display 120 while confirming the driving status of his / her own vehicle 200 with his / her eyes.

[0219] Therefore, if the terminal user UR can be provided with information about the control state of the own vehicle 200 from the operation terminal 100, the terminal user UR can obtain more information about the control state of the own vehicle 200, and thus it is preferable for the terminal user UR.

[0220] Therefore, the vehicle ECU 290 is configured to, when the own vehicle 200 autonomously travels by the remote travel control (when the own vehicle 200 autonomously advances and when the own vehicle 200 autonomously retreats), externally transmit, via the vehicle transceiver 280, a signal (travel vibration request signal S41) that requests the vibration device 130 to operate so that the operation terminal 100 vibrates in a predetermined vibration pattern (travel vibration pattern). In other words, the vehicle ECU 290 is configured to, during the period in which the remote travel control is executed, when the control state of the own vehicle 200 is in a state in which the own vehicle 200 travels, externally transmit, via the vehicle transceiver 280, the travel vibration request signal S41.

[0221] On the other hand, the terminal ECU 190 is configured to, when the travel vibration request signal S41 is received via the terminal transceiver 180, execute a vibration process (travel vibration process) that causes the vibration device 130 to operate so that the travel vibration pattern is generated. In other words, the terminal ECU 190 is configured to, during the period in which the remote travel control is executed, when the control state of the own vehicle 200 is in a state in which the own vehicle 200 travels, execute the travel vibration process.

[0222] Thus, the terminal user UR can recognize that the own vehicle 200 is traveling on the basis of the vibration of the operation terminal 100 in addition to visually recognizing that the own vehicle 200 is traveling.

[0223] In the present example, the vibration pattern is defined by the duration of one vibration (vibration duration), the number of vibrations in one vibration process (vibration number), the time interval between vibrations in one vibration process (vibration time interval), the intensity of vibration (vibration intensity), and the frequency of vibration (vibration frequency). Therefore, by making one or more of these vibration duration, vibration number, vibration time interval, vibration intensity, and vibration frequency different, it is possible to set the vibration pattern to different patterns.

[0224] In the present example, the operation terminal 100 vibrates in the same vibration pattern when the own vehicle 200 advances and when the own vehicle 200 retreats by the remote travel control, but the vibration pattern can be made different when the own vehicle 200 advances and when the own vehicle 200 retreats.

[0225] Further, the vehicle ECU 290 is configured to transmit, to the outside via the vehicle transceiver 280, a signal (travel notification sound request signal S51) that requests the output of a notification sound from the audio device 140 in a predetermined notification sound output mode (travel notification sound output mode) when the self vehicle 200 is autonomously traveling by the remote travel control (when the self vehicle 200 is autonomously advancing and when the self vehicle 200 is autonomously retreating) on the basis of or instead of the transmission of the travel vibration request signal S41 to the outside. In this case, the terminal ECU 190 is configured to execute a notification sound process (travel notification sound process) that outputs the notification sound of the travel notification sound output mode from the audio device 140 when the travel notification sound request signal S51 is received via the terminal transceiver 180.

[0226] Further, the vehicle ECU 290 is configured not to transmit, to the outside, a signal (vibration request signal S40) that requests the vibration implemented by the vibration device 130 when the self vehicle 200 is stopped by the remote travel control. In other words, the vehicle ECU 290 is configured not to transmit the vibration request signal S40 to the outside when the control state of the self vehicle 200 is in a state where the self vehicle 200 is stopped during the execution of the remote travel control.

[0227] On the other hand, the terminal ECU 190 stops the operation of the vibration device 130 when the vibration request signal S40 is not received. That is, the terminal ECU 190 does not perform any vibration process. In other words, the terminal ECU 190 does not perform any vibration process when the control state of the self vehicle 200 is in a state where the self vehicle 200 is stopped during the execution of the remote travel control.

[0228] Further, the vehicle ECU 290 is configured to transmit, to the outside via the vehicle transceiver 280, a signal (travel notification sound request signal S51) that requests the output of a notification sound from the audio device 140 in a predetermined notification sound output mode (travel notification sound output mode) when the self vehicle 200 is autonomously traveling by the remote travel control (when the self vehicle 200 is autonomously advancing and when the self vehicle 200 is autonomously retreating) on the basis of or instead of the transmission of the travel vibration request signal S41 to the outside. In this case, the terminal ECU 190 is configured to execute a notification sound process (travel notification sound process) that outputs the notification sound of the travel notification sound output mode from the audio device 140 when the travel notification sound request signal S51 is received via the terminal transceiver 180.

[0229] Further, the gear change vibration pattern is set to a different pattern from the travel vibration pattern. In addition, the vehicle ECU 290 can also be configured to transmit the gear change vibration request signal S42 to the outside via the vehicle transceiver 280 also when changing the gear state of the transmission device 224 from the forward drive state SD to the reverse drive state SR or from the reverse drive state SR to the forward drive state SD for a purpose other than performing the U-turn of the own vehicle 200.

[0230] In addition, the vehicle ECU 290 can also be configured to transmit the gear change vibration request signal S42 to the outside via the vehicle transceiver 280 also when changing the gear state of the transmission device 224 from the forward drive state SD to the reverse drive state SR or from the reverse drive state SR to the forward drive state SD for a purpose other than performing the U-turn of the own vehicle 200.

[0231] On the other hand, the terminal ECU 190 is configured to execute, when the gear change vibration request signal S42 is received via the terminal transceiver 180, a vibration process (gear change vibration process) of causing the vibration device 130 to operate so that the gear change vibration pattern is generated. In other words, the terminal ECU 190 is configured to execute the gear change vibration process when the control state of the own vehicle 200 is in a state of changing the gear state of the transmission device 224 during the execution of the remote travel control.

[0232] Thus, the terminal user UR can recognize the condition that the gear state of the transmission device 224 is changed for the purpose of performing the U-turn of the own vehicle 200 or the like, by the vibration of operating the terminal 100, although it is difficult to recognize by the visual method.

[0233] Further, the vehicle ECU 290 can also be configured to transmit, to the outside via the vehicle transceiver 280, a signal (gear change notification sound request signal S52) that requests the output of a notification sound from the audio device 140 in a predetermined notification sound output pattern (gear change notification sound output pattern) on the basis of or instead of transmitting the gear change vibration request signal S42 to the outside when changing the gear state of the transmission device 224. In this case, the terminal ECU 190 is configured to execute, when the gear change notification sound request signal S52 is received via the terminal transceiver 180, a notification sound process (gear change notification sound process) of outputting the notification sound of the gear change notification sound output pattern from the audio device 140. In this case, the gear change notification sound output pattern is a different pattern from the travel notification sound output pattern.

[0234] In addition, the notification sound output mode is defined by the output duration of the notification sound at a time (notification sound output duration), the number of times of output of the notification sound in one notification sound processing (notification sound output number), the time interval between the output of the notification sound in one notification sound processing and the output of the notification sound (notification sound output time interval), the intensity of the notification sound (notification sound intensity), and the frequency or pitch of the notification sound (notification sound frequency). Therefore, by making one or more of these notification sound output duration, notification sound output number, notification sound output time interval, notification sound intensity, and notification sound frequency different, the notification sound output mode can be set to different modes.

[0235] According to the remote travel system described above, for example, during execution of the remote travel control, as shown in Figure 27 vibration of the operation terminal 100 is generated. Figure 27 The following example is shown: during execution of the remote travel control, at time t270, the forward travel of the own vehicle 200 is started, at time t271, the own vehicle 200 is temporarily stopped for the purpose of U-turn of the own vehicle 200, at time t272, the gear state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR, and at time t273, the reverse travel of the own vehicle 200 is started.

[0236] In the example shown in Figure 27 When the forward travel of the own vehicle 200 is started at time t270, travel vibration processing is started, and by the vibration device 130 of the operation terminal 100, vibration of the travel vibration mode is generated in the operation terminal 100.

[0237] Then, when the own vehicle 200 is stopped at time t271, the vibration processing (travel vibration processing) is stopped.

[0238] Then, when the gear state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR at time t272, gear change vibration processing is executed, and by the vibration device 130 of the operation terminal 100, vibration of the gear change vibration mode is generated in the operation terminal 100.

[0239] Then, when the reverse travel of the own vehicle 200 is started at time t273, travel vibration processing is started, and by the vibration device 130 of the operation terminal 100, vibration of the travel vibration mode is generated in the operation terminal 100.

[0240] Further, in the case where the braking of the own vehicle 200 is performed by the remote travel control, it is preferable for the terminal user UR if the terminal user UR can be notified of the situation.

[0241] Thus, the vehicle ECU 290 can also be configured to, in a case where the own vehicle 200 is braked by the remote travel control, externally transmit, via the vehicle transceiver 280, a signal (brake vibration request signal S43) that requests the vibration device 130 to be operated to vibrate the operation terminal 100 in a predetermined vibration pattern (brake vibration pattern). In other words, the vehicle ECU 290 can also be configured to, during the period in which the remote travel control is executed, externally transmit, via the vehicle transceiver 280, the brake vibration request signal S43 when the control state of the own vehicle 200 is in a state of braking the own vehicle 200.

[0242] On the other hand, in this case, the terminal ECU 190 is configured to, when the brake vibration request signal S43 is received via the terminal transceiver 180, execute a vibration process (brake vibration process) that causes the vibration device 130 to be operated to generate vibration in the brake vibration pattern. In other words, the terminal ECU 190 is configured to, during the period in which the remote travel control is executed, execute the brake vibration process when the control state of the own vehicle 200 is in a state of braking the own vehicle 200. In this case, the brake vibration pattern is set to a pattern that is different from both the travel vibration pattern and the gear change vibration pattern.

[0243] Thus, the terminal user UR can recognize that the own vehicle 200 is being braked from the vibration of the operation terminal 100 even when it is difficult to recognize by visual observation that the own vehicle 200 is being braked.

[0244] Further, the vehicle ECU 290 can also be configured to, in a case where the own vehicle 200 is braked by the remote travel control, externally transmit, via the vehicle transceiver 280, a signal (brake vibration request signal S43) that requests the vibration device 130 to be operated to vibrate the operation terminal 100 in a predetermined vibration pattern (brake vibration pattern). In other words, the vehicle ECU 290 can also be configured to, during the period in which the remote travel control is executed, externally transmit, via the vehicle transceiver 280, the brake vibration request signal S43 when the control state of the own vehicle 200 is in a state of braking the own vehicle 200. In this case, the brake vibration pattern is set to a pattern that is different from both the travel vibration pattern and the gear change vibration pattern.

[0245] In addition, when the own vehicle 200 is traveling by the remote travel control, in a case where an obstacle (or an obstacle that seems to be an obstacle) such as a person that hinders the travel of the own vehicle 200 is detected on the basis of the surrounding detection information IS, it is preferable for the terminal user UR if the situation can be notified to the terminal user UR.

[0246] Therefore, the vehicle ECU 290 can also be configured such that, when an obstacle is detected while the vehicle 200 is being driven by remote driving control, it sends a signal (obstacle vibration request signal S44) to the outside via the vehicle transceiver 280 requesting the vibration device 130 to operate so that the operating terminal 100 vibrates in a predetermined vibration mode (obstacle vibration mode).

[0247] On the other hand, in this case, the terminal ECU 190 is configured to perform vibration processing (obstacle vibration processing) when it receives the obstacle vibration request signal S44 via the terminal transceiver 180, thereby activating the vibration device 130 to generate vibration in the obstacle vibration mode.

[0248] In this case, the obstacle vibration mode is set to a mode different from the driving vibration mode, gear shifting vibration mode, and braking vibration mode. Additionally, when an obstacle is detected, braking of the vehicle 200 is initiated; however, in this example, obstacle vibration processing is performed instead of braking vibration processing.

[0249] Therefore, in addition to visually identifying the presence of obstacles, the terminal user UR can also identify the presence of obstacles based on the vibration of the operating terminal 100.

[0250] Furthermore, the vehicle ECU 290 can also be configured such that, when an obstacle is detected while the vehicle 200 is being driven via remote driving control, it sends a signal (obstacle notification sound request signal S54) via the vehicle transceiver 280 requesting the audio device 140 to output a notification sound in a predetermined notification sound output mode (obstacle notification sound output mode), either in addition to or instead of sending an obstacle vibration request signal S44. In this case, the terminal ECU 190 is configured to, upon receiving the obstacle notification sound request signal S54 via the terminal transceiver 180, perform notification sound processing (obstacle notification sound processing) to output the notification sound in the obstacle notification sound output mode from the audio device 140. In this case, the obstacle notification sound output mode is different from the driving notification sound output mode, the gear change notification sound output mode, and the braking notification sound output mode.

[0251] According to this remote driving system, for example, during the execution of remote driving control, such as Figure 28 The operation terminal 100 vibrates as shown. Figure 28The following example is shown: during the execution of the remote travel control, at time t280, the forward travel of the own vehicle 200 is started, at time t281, the braking of the own vehicle 200 is started for the purpose of the U-turn of the own vehicle 200, at time t282, the own vehicle 200 is temporarily stopped, at time t283, the gear state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR, at time t284, the reverse travel of the own vehicle 200 is started, at time t285, the braking of the own vehicle 200 is started in accordance with the detection of the obstacle, at time t286, the own vehicle 200 is stopped, at time t287, the reverse travel of the own vehicle 200 is started in accordance with the change to the absence of the obstacle.

[0252] In Figure 28 In the example shown, when the forward travel of the own vehicle 200 is started at time t280, the travel vibration process is started, and the vibration of the travel vibration pattern is generated at the operation terminal 100 by operating the vibration device 130 of the operation terminal 100.

[0253] Then, when the braking of the own vehicle 200 is started at time t281, the travel vibration process is stopped, and the braking vibration process is started, and the vibration of the braking vibration pattern is generated at the operation terminal 100 by operating the vibration device 130 of the operation terminal 100.

[0254] Then, when the own vehicle 200 is stopped at time t282, the braking vibration process is stopped.

[0255] Then, when the gear state of the transmission device 224 is changed from the forward drive state SD to the reverse drive state SR at time t283, the gear change vibration process is executed, and the vibration of the gear change vibration pattern is generated at the operation terminal 100 by operating the vibration device 130 of the operation terminal 100.

[0256] Then, when the reverse travel of the own vehicle 200 is started at time t284, the travel vibration process is started, and the vibration of the travel vibration pattern is generated at the operation terminal 100 by operating the vibration device 130 of the operation terminal 100.

[0257] Then, when the braking of the own vehicle 200 is started in accordance with the detection of the obstacle at time t285, the travel vibration process is stopped, and the obstacle vibration process is started, and the vibration of the obstacle vibration pattern is generated at the operation terminal 100 by operating the vibration device 130 of the operation terminal 100. Then, the own vehicle 200 is stopped at time t286.

[0258] Then, when the reverse travel of the own vehicle 200 is started in accordance with the change to the absence of the obstacle at time t287, the obstacle vibration process is stopped, and the travel vibration process is started, and the vibration of the travel vibration pattern is generated at the operation terminal 100 by operating the vibration device 130 of the operation terminal 100.

[0259] The above describes the remote travel system 10 that performs travel vibration processing when the host vehicle 200 is made to travel forward or backward by remote travel control, performs gear change vibration processing when the host vehicle 200 is made to turn around, and performs obstacle vibration processing on the basis of those vibration processing when an obstacle is detected, but the remote travel system 10 can also be configured so that, in the case where it has information that is preferably notified to the terminal user UR by vibration in addition to the information such as the travel state (forward and backward) of the host vehicle 200, the turn around of the host vehicle 200, and the detection of an obstacle, in order to notify the information, the operation terminal 100 is made to vibrate in a vibration pattern different from the vibration pattern used to notify other information by the vibration device 130.

[0260] The above is a summary of the operation of the remote travel system 10. According to the remote travel system 10, the vibration pattern of the operation terminal 100 is different respectively when the host vehicle 200 travels by remote travel control, when the host vehicle 200 is made to brake, when the gear state of the transmission device 224 is changed, and when an obstacle is detected, so the terminal user UR can recognize the control state of the host vehicle 200, the presence of an obstacle from the vibration of the operation terminal 100 while looking at the host vehicle 200 without looking at the terminal display 120. In other words, the vibration pattern is different according to the control state of the host vehicle 200, the situation around the host vehicle 200, so the terminal user UR can recognize the control state of the host vehicle 200, the situation around the host vehicle 200 from the vibration of the operation terminal 100 while looking at the host vehicle 200 without looking at the terminal display 120.

[0261] <Detailed Operation of Remote Travel System>

[0262] Next, the detailed operation of the remote travel system 10 is described. The terminal CPU 191 of the terminal ECU 190 of the terminal control device 110 executes the routine shown in FIG. 29 at a predetermined operation cycle. Therefore, when it becomes a predetermined timing, the terminal CPU 191 starts processing from step 2900 of FIG. 29, makes the processing proceed to step 2905, and determines whether or not the value of the initial screen image display flag X10 is "0". The initial screen image display flag X10 is a flag indicating whether or not the initial screen image G10 is displayed on the terminal display 120, and the value is set to "1" when the initial screen image G10 is displayed on the terminal display 120. Further, in the case where the value of the initial screen image display flag X10 is "0", the initial screen image G10 is not displayed on the terminal display 120. Figure 29 Figure 29

[0263] ​​When the terminal CPU 191 determines "Yes" in step 2905, it causes the processing to proceed to step 2910, and determines whether an application start operation has been performed. That is, the terminal CPU 191 determines whether a predetermined touch operation for starting the remote travel application has been applied to the terminal display 120. When the terminal CPU 191 determines "Yes" in step 2910, it causes the processing to proceed to step 2915, and starts the remote travel application and displays the initial screen image G10. Next, the terminal CPU 191 causes the processing to proceed to step 2920, and wirelessly transmits an application start signal S10 to the outside. Next, the terminal CPU 191 causes the processing to proceed to step 2925, and sets the value of the initial screen image display flag X10 to "1". Next, the terminal CPU 191 causes the processing to proceed to step 2995, and temporarily ends the routine.

[0264] On the other hand, when the terminal CPU 191 determines "No" in step 2910, it causes the processing to directly proceed to step 2995, and temporarily ends the routine.

[0265] In addition, when the terminal CPU 191 determines "No" in step 2905, it causes the processing to proceed to step 2930, and determines whether an application end operation has been performed. That is, the terminal CPU 191 determines whether a tap operation has been applied to the application end image portion P15 or the like. When the terminal CPU 191 determines "Yes" in step 2930, it causes the processing to proceed to step 2935, and ends the remote travel application. Next, the terminal CPU 191 causes the processing to proceed to step 2940, and wirelessly transmits a control end instruction signal S30 to the outside. Next, the terminal CPU 191 causes the processing to proceed to step 2945, and sets the values of the initial screen image display flag X10, the entrance / exit selection image display flag X11, the parking area selection image display flag X12, the exit direction selection image display flag X13, the entrance operation image display flag X14, and the exit operation image display flag X15 to "0", respectively. Next, the terminal CPU 191 causes the processing to proceed to step 2995, and temporarily ends the routine.

[0266] On the other hand, when the terminal CPU 191 determines "No" in step 2930, it causes the processing to directly proceed to step 2995, and temporarily ends the routine.

[0267] Further, the terminal CPU 191 executes the routine illustrated in FIG. 29 at a predetermined operation cycle. Therefore, when it becomes a predetermined timing, the terminal CPU 191 causes the processing to proceed to step 2905, and determines whether the remote travel application is being executed. When the terminal CPU 191 determines "Yes" in step 2905, it causes the processing to proceed to step 2907, and determines whether the remote travel application is in the initial screen image G10. When the terminal CPU 191 determines "Yes" in step 2907, it causes the processing to proceed to step 2909, and determines whether the remote travel application is in the entrance operation image G20. When the terminal CPU 191 determines "Yes" in step 2909, it causes the processing to proceed to step 2910, and determines whether an application start operation has been performed. Figure 30 Figure 30 ​Step 3000 begins processing, which proceeds to step 3005, where it is determined whether the value of flag X10 in the initial screen image display is "1". If the terminal CPU 191 determines "yes" in step 3005, the processing proceeds to step 3010, where it is determined whether the value of flag X11 in the inbound / outbound selection image display is "0". Flag X11 in the inbound / outbound selection image display indicates whether the inbound / outbound selection image G20 is displayed on the terminal display 120, and its value is set to "1" when the inbound / outbound selection image G20 is displayed on the terminal display 120. Furthermore, if the value of flag X11 in the inbound / outbound selection image display is "0", the inbound / outbound selection image G20 is not displayed on the terminal display 120.

[0268] If the terminal CPU 191 determines "yes" in step 3010, it proceeds to step 3015 to determine whether a remote inbound / outbound operation has been initiated. Specifically, the terminal CPU 191 determines whether a touch operation has been applied to the remote inbound / outbound start image portion P11. If the terminal CPU 191 determines "yes" in step 3015, it proceeds to step 3020, displaying the inbound / outbound selection image G20 on the terminal display 120. Next, the terminal CPU 191 proceeds to step 3025, setting the value of the flag X11 in the inbound / outbound selection image display to "1". Then, the terminal CPU 191 proceeds to step 3095, temporarily terminating this routine.

[0269] On the other hand, if the terminal CPU191 determines "no" in step 3015, the process will directly proceed to step 3095, temporarily ending the routine.

[0270] In addition, if the terminal CPU191 determines "no" in step 3005 or step 3010, the process will directly proceed to step 3095, temporarily ending this routine.

[0271] Furthermore, the terminal CPU191 executes operations in predetermined computation cycles. Figure 31 The example shown. Therefore, when the scheduled time is reached, the terminal CPU191 starts from... Figure 31The process starts in step 3100, and the process advances to step 3105, where it is determined whether the value of flag X11 in the in-out selection image display is "1". If the terminal CPU 191 determines YES in step 3105, the process advances to step 3110, where it is determined whether the value of flag X12 in the parking area selection image display is "0". Flag X12 in the parking area selection image display is a flag indicating whether the parking area selection image G30 is displayed on the terminal display 120, and its value is set to "1" when the parking area selection image G30 is displayed on the terminal display 120. Further, if the value of flag X12 in the parking area selection image display is "0", the parking area selection image G30 is not displayed on the terminal display 120.

[0272] If the terminal CPU 191 determines YES in step 3110, the process advances to step 3115, where it is determined whether an in selection operation has been performed. That is, the terminal CPU 191 determines whether a touch operation has been applied to the in selection image portion P21. If the terminal CPU 191 determines YES in step 3115, the process advances to step 3120, where the parking area selection image G30 is displayed on the terminal display 120. Next, the terminal CPU 191 advances the process to step 3125, where the value of flag X12 in the parking area selection image display is set to "1". Next, the terminal CPU 191 advances the process to step 3130.

[0273] On the other hand, if the terminal CPU 191 determines NO in step 3115, the process advances directly to step 3130.

[0274] When the terminal CPU 191 advances the process to step 3130, it is determined whether the value of flag X13 in the out direction selection image display is "0". Flag X13 in the out direction selection image display is a flag indicating whether the out direction selection image G50 is displayed on the terminal display 120, and its value is set to "1" when the out direction selection image G50 is displayed on the terminal display 120. Further, if the value of flag X13 in the out direction selection image display is "0", the out direction selection image G50 is not displayed on the terminal display 120.

[0275] If the terminal CPU 191 determines "yes" in step 3130, it proceeds to step 3135 to determine whether an outbound selection operation has been applied to the terminal display 120. That is, the terminal CPU 191 determines whether a touch operation has been applied to the outbound selection image portion P22. If the terminal CPU 191 determines "yes" in step 3135, it proceeds to step 3140 to display the outbound direction selection image G50 on the terminal display 120. Next, the terminal CPU 191 proceeds to step 3145 to set the value of the flag X13 in the outbound direction selection image display to "1". Then, the terminal CPU 191 proceeds to step 3195, temporarily terminating this routine.

[0276] On the other hand, if the terminal CPU191 determines "no" in step 3135, the processing will directly proceed to step 3195, temporarily ending the routine.

[0277] In addition, if the terminal CPU191 determines "no" in step 3105, step 3110 or step 3130, the process will directly proceed to step 3195, temporarily ending this routine.

[0278] Furthermore, the terminal CPU191 executes operations in predetermined computation cycles. Figure 32 The example shown. Therefore, when the scheduled time is reached, the terminal CPU191 starts from... Figure 32 Step 3200 begins processing, which proceeds to step 3205, where it is determined whether the value of flag X12 in the parking area selection image display is "1". If the terminal CPU 191 determines "yes" in step 3205, the processing proceeds to step 3210, where it is determined whether the value of flag X14 in the parking operation image display is "0". Flag X14 in the parking operation image display indicates whether the parking operation image G40 is displayed on the terminal display 120, and its value is set to "1" when the parking operation image G40 is displayed on the terminal display 120. Furthermore, if the value of flag X14 in the parking operation image display is "0", the parking operation image G40 is not displayed on the terminal display 120.

[0279] If the terminal CPU 191 determines "yes" in step 3210, the process proceeds to step 3215, whereby it determines whether a parking zone selection operation has been applied to the terminal display 120. That is, the terminal CPU 191 determines whether a touch operation has been applied to the parking zone image portion P31. If the terminal CPU 191 determines "yes" in step 3215, the process proceeds to step 3220, whereby it determines whether a confirmation operation has been applied to the terminal display 120. That is, the terminal CPU 191 determines whether a touch operation has been applied to the selection confirmation image portion P34.

[0280] When the terminal CPU 191 determines "Yes" in step 3220, it causes the process to proceed to step 3225, and displays the storage operation image G40. Next, the terminal CPU 191 causes the process to proceed to step 3230, and wirelessly transmits the designated parking area signal S12 to the outside. Also at this time, the terminal CPU 191 displays the vehicle image G42 on the portion of the terminal display 120 that corresponds to the position of the own vehicle 200 indicated by the vehicle position signal S13, displays the storage remaining distance image G43 that displays the storage remaining distance Din indicated by the storage remaining distance signal S15 on the terminal display 120, and displays the travel direction image G47 that displays the travel direction of the own vehicle 200 indicated by the travel direction signal S17 on the terminal display 120.

[0281] Next, the terminal CPU 191 causes the process to proceed to step 3235, and sets the value of the storage operation image display flag X14 to "1". Next, the terminal CPU 191 causes the process to proceed to step 3240.

[0282] On the other hand, when the terminal CPU 191 determines "No" in step 3220, it causes the process to proceed directly to step 3240.

[0283] Also, when the terminal CPU 191 determines "No" in step 3215, it also causes the process to proceed directly to step 3240.

[0284] When the terminal CPU 191 causes the process to proceed to step 3240, it determines whether or not a storage and retrieval re-selection operation has been applied to the terminal display 120. That is, the terminal CPU 191 determines whether or not a touch operation has been applied to the storage and retrieval re-selection image portion P36. When the terminal CPU 191 determines "Yes" in step 3240, it causes the process to proceed to step 3245, and displays the storage and retrieval selection image G20 on the terminal display 120. Next, the terminal CPU 191 causes the process to proceed to step 3250, and sets the values of the storage and retrieval selection image display flag X11, the parking area selection image display flag X12, and the storage operation image display flag X14 to "0". Next, the terminal CPU 191 causes the process to proceed to step 3295, and temporarily ends this routine.

[0285] On the other hand, when the terminal CPU 191 determines "No" in step 3240, it causes the process to proceed directly to step 3295, and temporarily ends this routine.

[0286] Also, when the terminal CPU 191 determines "No" in step 3205 or step 3210, it also causes the process to proceed directly to step 3295, and temporarily ends this routine.

[0287] Furthermore, the terminal CPU191 executes operations in predetermined computation cycles. Figure 33 The example shown. Therefore, when the scheduled time is reached, the terminal CPU191 starts from... Figure 33 Step 3300 begins processing, which proceeds to step 3305, where it is determined whether the value of flag X13 in the outbound direction selection image display is "1". If the terminal CPU 191 determines "yes" in step 3305, the processing proceeds to step 3310, where it is determined whether the value of flag X15 in the outbound operation image display is "0". Flag X15 in the outbound operation image display indicates whether the outbound operation image G60 is displayed on the terminal display 120. In this case, it is set to "1" when the outbound operation image G60 is displayed on the terminal display 120. Furthermore, if the value of flag X15 in the outbound operation image display is "0", the outbound operation image G60 is not displayed on the terminal display 120.

[0288] If the terminal CPU 191 determines "yes" in step 3310, the process proceeds to step 3315, whereby it determines whether an outbound direction selection operation has been applied to the terminal display 120. That is, the terminal CPU 191 determines whether a touch operation has been applied to the outbound direction image portion P53. If the terminal CPU 191 determines "yes" in step 3315, the process proceeds to step 3320, whereby it determines whether a confirmation operation has been applied to the terminal display 120. That is, the terminal CPU 191 determines whether a touch operation has been applied to the selection confirmation image portion P54.

[0289] If the terminal CPU 191 determines "yes" in step 3320, it proceeds to step 3325 and displays the outbound operation image G60 on the terminal display 120. Next, the terminal CPU 191 proceeds to step 3330 and wirelessly transmits the specified outbound direction signal S22. Furthermore, at this time, the terminal CPU 191 displays the vehicle image G62 on the portion of the terminal display 120 corresponding to the position of its own vehicle 200 indicated by the vehicle position signal S23, and displays the remaining outbound distance image G63 on the terminal display 120, which displays the remaining outbound distance Dout indicated by the remaining outbound distance signal S25.

[0290] Next, the terminal CPU 191 causes the processing to proceed to step 3335, setting the value of flag X15 in the outbound operation image display to "1". Then, the terminal CPU 191 causes the processing to proceed to step 3340.

[0291] On the other hand, if the terminal CPU191 determines "no" in step 3320, the processing will proceed directly to step 3340.

[0292] In addition, if the terminal CPU191 determines "no" in step 3315, the process will directly proceed to step 3340.

[0293] When the terminal CPU 191 initiates step 3340, it determines whether an inbound / outbound reselection operation has been applied to the terminal display 120. Specifically, the terminal CPU 191 determines whether a touch operation has been applied to the inbound / outbound reselection image portion P56. If the terminal CPU 191 determines "yes" in step 3340, it initiates step 3345, displaying the inbound / outbound selection image G20 on the terminal display 120. Next, the terminal CPU 191 initiates step 3350, setting the values ​​of flag X11 in the inbound / outbound selection image display, flag X13 in the outbound direction selection image display, and flag X15 in the outbound operation image display to "0". Then, the terminal CPU 191 initiates step 3395, temporarily terminating this routine.

[0294] On the other hand, if the terminal CPU191 determines "no" in step 3340, the process will directly proceed to step 3395, temporarily ending the routine.

[0295] In addition, if the terminal CPU191 determines "no" in step 3305 or step 3310, the process will directly proceed to step 3395, temporarily ending this routine.

[0296] Furthermore, the terminal CPU191 executes operations in predetermined computation cycles. Figure 34 The example shown. Therefore, when the scheduled time is reached, the terminal CPU191 starts from... Figure 34 Step 3400 begins processing, which proceeds to step 3405 to determine whether the value of flag X14 in the inbound operation image display is "1". If the terminal CPU 191 determines "yes" in step 3405, the processing proceeds to step 3410 to determine whether a driving operation has been performed. That is, the terminal CPU 191 determines whether a touch operation has been applied to the inbound driving operation image portion P44. If the terminal CPU 191 determines "yes" in step 3410, the processing proceeds to step 3415 to wirelessly transmit a continuous touch operation signal S16 to the outside. Next, the terminal CPU 191 proceeds to step 3420.

[0297] On the other hand, if the terminal CPU191 determines "no" in step 3410, the processing will proceed directly to step 3420.

[0298] When the terminal CPU 191 initiates step 3420, it determines whether the vehicle 200 has completed its entry into the designated parking area 31D. Specifically, the terminal CPU 191 determines whether it has received the entry completion signal S18. If the terminal CPU 191 determines "yes" in step 3420, it initiates step 3425 and displays the entry completion image G49 on the terminal display 120. Next, the terminal CPU 191 initiates step 3430.

[0299] On the other hand, if the terminal CPU191 determines "no" in step 3420, the processing will proceed directly to step 3430.

[0300] When the terminal CPU 191 initiates step 3430, it determines whether an entry / exit reselection operation has been applied to the terminal display 120. Specifically, the terminal CPU 191 determines whether a touch operation has been applied to the entry / exit reselection image portion P46. If the terminal CPU 191 determines "yes" in step 3430, it initiates step 3435, displaying the entry / exit selection image G20 on the terminal display 120. Next, the terminal CPU 191 initiates step 3440, setting the values ​​of flag X12 in the parking area selection image display and flag X14 in the entry operation image display to "0" respectively. Then, the terminal CPU 191 initiates step 3495, temporarily terminating this routine.

[0301] On the other hand, if the terminal CPU191 determines "no" in step 3430, the process will directly proceed to step 3495, temporarily ending the routine.

[0302] In addition, if the terminal CPU191 determines "no" in step 3405, the process will directly proceed to step 3495, temporarily ending this routine.

[0303] Furthermore, the terminal CPU191 executes operations in predetermined computation cycles. Figure 35 The example shown. Therefore, when the scheduled time is reached, the terminal CPU191 starts from... Figure 35 Step 3500 begins processing, which proceeds to step 3505 to determine if the value of flag X15 in the outbound operation image display is "1". If the terminal CPU 191 determines "yes" in step 3505, the processing proceeds to step 3510 to determine whether a driving operation has been performed. That is, the terminal CPU 191 determines whether a touch operation has been applied to the outbound driving operation image portion P64. If the terminal CPU 191 determines "yes" in step 3510, the processing proceeds to step 3515, wirelessly transmitting a continuous touch operation signal S26 to the outside. Next, the terminal CPU 191 proceeds to step 3520.

[0304] On the other hand, in a case where the terminal CPU 191 determines "NO" in step 3510, the process is made to proceed directly to step 3520.

[0305] When the process is made to proceed to step 3520 by the terminal CPU 191, it determines whether the egress from the current parking area 31N of the own vehicle 200 has been completed. That is, the terminal CPU 191 determines whether the egress completion signal S28 is received. In a case where the terminal CPU 191 determines "YES" in step 3520, the process is made to proceed to step 3525, and the egress completion image G69 is displayed on the terminal display 120. Then, the terminal CPU 191 makes the process proceed to step 3530.

[0306] On the other hand, in a case where the terminal CPU 191 determines "NO" in step 3520, the process is made to proceed directly to step 3530.

[0307] When the process is made to proceed to step 3530 by the terminal CPU 191, it determines whether the entry / egress re-selection operation is applied to the terminal display 120. That is, the terminal CPU 191 determines whether the touch operation is applied to the entry / egress re-selection image portion P66. In a case where the terminal CPU 191 determines "YES" in step 3530, the process is made to proceed to step 3535, and the entry / egress selection image G20 is displayed on the terminal display 120. Then, the terminal CPU 191 makes the process proceed to step 3540, and the values of the egress direction selection image display flag X13 and the egress operation image display flag X15 are set to "0", respectively. Then, the terminal CPU 191 makes the process proceed to step 3595, and temporarily ends the present routine.

[0308] On the other hand, in a case where the terminal CPU 191 determines "NO" in step 3530, the process is made to proceed directly to step 3595, and the present routine is temporarily ended.

[0309] In addition, in a case where the terminal CPU 191 determines "NO" in step 3505, the process is also made to proceed directly to step 3595, and the present routine is temporarily ended.

[0310] On the other hand, the vehicle CPU 291 of the vehicle ECU 290 of the vehicle control device 210 executes the routine shown in FIG. 36 at a predetermined operation cycle. Therefore, when it becomes a predetermined timing, the vehicle CPU 291 makes the process proceed to step 3600. Figure 36 Figure 36 ​The routine shown in FIG. 36 is started by the vehicle CPU 291, and the process is advanced to step 3605, where it is determined whether the value of the storage path setting completion flag X20 is "0". The storage path setting completion flag X20 is a flag indicating whether the target storage path Rmt_tgt has been set, and its value is set to "1" when the target storage path Rmt_tgt is set. Further, when the value of the storage path setting completion flag X20 is "0", the target storage path Rmt_tgt is not set.

[0311] When the vehicle CPU 291 determines "Yes" in step 3605, the process is advanced to step 3610, where it is determined whether the designated parking area signal S14 has been received. When the vehicle CPU 291 determines "Yes" in step 3610, the process is advanced to step 3615, where the target storage path Rmt_tgt is set. Next, the vehicle CPU 291 advances the process to step 3620, where the value of the storage path setting completion flag X20 is set to "1". Next, the vehicle CPU 291 advances the process to step 3695, where the routine is temporarily ended.

[0312] On the other hand, when the vehicle CPU 291 determines "No" in step 3610, the process is directly advanced to step 3695, where the routine is temporarily ended.

[0313] Further, when the vehicle CPU 291 determines "No" in step 3605, the process is also directly advanced to step 3695, where the routine is temporarily ended.

[0314] Further, the vehicle CPU 291 executes the routine shown in FIG. 37 at a predetermined operation cycle. Therefore, when it becomes the predetermined timing, the vehicle CPU 291 starts the process from step 3700 of FIG. 37, and advances the process to step 3705, where it is determined whether the value of the storage path setting completion flag X20 is "1". When the vehicle CPU 291 determines "Yes" in step 3705, the process is advanced to step 3710, where it is determined whether the continuous touch operation signal S16 has been received. Figure 37 Figure 37 When the vehicle CPU 291 determines "Yes" in step 3710, the process is advanced to step 3715, where the operation of the vehicle travel device 220 is controlled so that the host vehicle 200 travels along the target storage path Rmt_tgt. Next, the vehicle CPU 291 advances the process to step 3720, where it is determined whether the host vehicle 200 has reached the target parking place Pin_tgt. That is, the vehicle CPU 291 determines whether the host vehicle 200 has completed storage into the designated parking area 31D.

[0315] When the vehicle CPU 291 determines "Yes" in step 3710, the process is advanced to step 3715, where the operation of the vehicle travel device 220 is controlled so that the host vehicle 200 travels along the target storage path Rmt_tgt. Next, the vehicle CPU 291 advances the process to step 3720, where it is determined whether the host vehicle 200 has reached the target parking place Pin_tgt. That is, the vehicle CPU 291 determines whether the host vehicle 200 has completed storage into the designated parking area 31D.

[0316] ​If the vehicle CPU 291 determines "yes" in step 3720, it proceeds to step 3725 to perform parking processing, and further performs parking holding processing. Next, the vehicle CPU 291 proceeds to step 3730 to stop the operation of the surrounding information detection device 260 and the vehicle driving device 220. Next, the vehicle CPU 291 proceeds to step 3735 to wirelessly send a parking completion signal S18 to the outside. Next, the vehicle CPU 291 proceeds to step 3740 to set the value of the parking path setting completion flag X20 to "0". Next, the vehicle CPU 291 proceeds to step 3795 to temporarily terminate this routine.

[0317] On the other hand, if the vehicle CPU 291 determines "no" in step 3720, the process will directly proceed to step 3795, temporarily ending the current routine.

[0318] Additionally, if the vehicle CPU 291 determines "no" in step 3710, it proceeds to step 3745 to perform a parking procedure. Then, the vehicle CPU 291 proceeds to step 3795, temporarily terminating the current routine.

[0319] In addition, if the vehicle CPU291 determines "no" in step 3705, the process will directly proceed to step 3795, temporarily ending this routine.

[0320] Furthermore, the vehicle CPU291 executes in predetermined computation cycles. Figure 38 The example shown. Therefore, when the predetermined timing is reached, the vehicle CPU291 from Figure 38 Step 3800 begins processing, leading to step 3805, where it is determined whether the value of the outbound path setting completion flag X21 is "0". The outbound path setting completion flag X21 indicates whether the target outbound path Rout_tgt has been set; its value is set to "1" when the target outbound path Rout_tgt has been set. Furthermore, if the value of the outbound path setting completion flag X21 is "0", the target outbound path Rout_tgt has not been set.

[0321] If the vehicle CPU 291 determines "yes" in step 3805, it proceeds to step 3810 to determine whether the specified outbound direction signal S22 has been received. If the vehicle CPU 291 determines "yes" in step 3810, it proceeds to step 3815 to set the target outbound path Rout_tgt. Next, the vehicle CPU 291 proceeds to step 3820 to set the value of the outbound path setting completion flag X21 to "1". Then, the vehicle CPU 291 proceeds to step 3895, temporarily terminating this routine.

[0322] On the other hand, if the vehicle CPU 291 determines "no" in step 3810, the process will directly proceed to step 3895, temporarily ending the current routine.

[0323] In addition, if the vehicle CPU291 determines "no" in step 3805, the process will directly proceed to step 3895, temporarily ending this routine.

[0324] Furthermore, the vehicle CPU291 executes in predetermined computation cycles. Figure 39 The example shown. Therefore, when the predetermined timing is reached, the vehicle CPU291 from Figure 39 Step 3900 begins processing, which proceeds to step 3905, where it is determined whether the value of the outbound path setting completion flag X21 is "1". If the vehicle CPU 291 determines "yes" in step 3905, the processing proceeds to step 3910, where it is determined whether a continuous touch operation signal S26 has been received.

[0325] If the vehicle CPU 291 determines "yes" in step 3910, it proceeds to step 3915 to control the operation of the vehicle driving device 220 so that the vehicle 200 travels along the target outbound path Rout_tgt. Next, the vehicle CPU 291 proceeds to step 3920 to determine whether the vehicle 200 has reached the target outbound location Pout_tgt. That is, the vehicle CPU 291 determines whether the vehicle 200 has completed its outbound journey from the designated parking area 31D.

[0326] If the vehicle CPU 291 determines "yes" in step 3920, it proceeds to step 3925 to perform parking processing, and further, to perform parking holding processing. Next, the vehicle CPU 291 proceeds to step 3930 to stop the operation of the surrounding information detection device 260 and the vehicle driving device 220. Next, the vehicle CPU 291 proceeds to step 3935 to wirelessly send an exit completion signal S28 to the outside. Next, the vehicle CPU 291 proceeds to step 3940 to set the value of the exit path setting completion flag X21 to "0". Next, the vehicle CPU 291 proceeds to step 3995 to temporarily terminate this routine.

[0327] On the other hand, if the vehicle CPU 291 determines "no" in step 3920, the process will directly proceed to step 3995, temporarily ending the current routine.

[0328] Additionally, if the vehicle CPU 291 determines "no" in step 3910, it proceeds to step 3945 to perform a parking procedure. Then, the vehicle CPU 291 proceeds to step 3995, temporarily terminating the current routine.

[0329] In addition, in the case where the vehicle CPU 291 determines "NO" in step 3905, the process is caused to proceed directly to step 3995, and the present routine is temporarily ended.

[0330] Further, the vehicle CPU 291 executes the routine shown in Figure 40 at a predetermined operation cycle. Therefore, when it becomes a predetermined timing, the vehicle CPU 291 starts the process from step 4000 of the routine shown in Figure 40 , causes the process to proceed to step 4005, and determines whether or not the control end instruction signal S30 is received.

[0331] In the case where the vehicle CPU 291 determines "YES" in step 4005, the process is caused to proceed to step 4010, and a parking process is executed, and further, a parked state maintaining process is executed. Next, the vehicle CPU 291 causes the process to proceed to step 4015, and stops the operation of the surrounding information detection device 260 and the vehicle travel device 220. Next, the vehicle CPU 291 causes the process to proceed to step 4020, and sets the values of the in-warehouse path setting completion flag X20 and the out-of-warehouse path setting completion flag X21 to "0", respectively. Next, the vehicle CPU 291 causes the process to proceed to step 4095, and temporarily ends the present routine.

[0332] On the other hand, in the case where the vehicle CPU 291 determines "NO" in step 4005, the process is caused to proceed directly to step 4095, and the present routine is temporarily ended.

[0333] Further, the vehicle CPU 291 executes the routine shown in Figure 41 or Figure 43 at a predetermined operation cycle. Therefore, in the case where the vehicle CPU 291 executes the routine shown in Figure 41 , when it becomes a predetermined timing, the vehicle CPU 291 starts the process from step 4100 of the routine shown in Figure 41 , causes the process to proceed to step 4105, and determines whether or not it is in a period during which remote travel control is executed.

[0334] In the case where the vehicle CPU 291 determines "YES" in step 4105, the process is caused to proceed to step 4110, and it is determined whether or not the own vehicle 200 is in travel (in forward travel or in reverse travel). In the case where the vehicle CPU 291 determines "YES" in step 4110, the process is caused to proceed to step 4115, and a travel vibration request signal S41 is transmitted to the outside via the vehicle transceiver 280. Next, the vehicle CPU 291 causes the process to proceed to step 4120.

[0335] On the other hand, in the case where the vehicle CPU 291 determines "NO" in step 4110, the process is caused to proceed directly to step 4120.

[0336] When the vehicle CPU 291 initiates step 4120, it determines whether the gear position of the transmission device 224 has changed from forward drive to reverse drive or vice versa. If the vehicle CPU 291 determines "yes" in step 4120, it initiates step 4125 and sends a gear change vibration request signal S42 to the outside via the vehicle transceiver 280. Then, the vehicle CPU 291 initiates step 4195, temporarily terminating the current routine.

[0337] On the other hand, if the vehicle CPU 291 determines "no" in step 4120, the processing will proceed directly to step 4195, temporarily ending the processing of this routine.

[0338] In addition, if the vehicle CPU291 determines "no" in step 4105, the process will directly proceed to step 4195, temporarily ending the processing of this routine.

[0339] On the other hand, the terminal CPU191 executes the process of the vehicle CPU291. Figure 41 In the case of the example shown, it is executed in a predetermined computation cycle. Figure 42 The example shown. Therefore, when the scheduled time is reached, the terminal CPU191 starts from... Figure 42 The routine shown begins processing at step 4200, which leads to step 4205, where it is determined whether the remote driving application has been started.

[0340] If the terminal CPU 191 determines "yes" in step 4205, it proceeds to step 4210 to determine whether a driving vibration request signal S41 has been received. If the terminal CPU 191 determines "yes" in step 4210, it proceeds to step 4215 to perform driving vibration processing. Thus, the operating terminal 100 vibrates in driving vibration mode. Next, the terminal CPU 191 proceeds to step 4220.

[0341] On the other hand, if the terminal CPU191 determines "no" in step 4210, the processing will proceed directly to step 4220.

[0342] When the terminal CPU 191 initiates step 4220, it determines whether a gear shift vibration request signal S42 has been received. If the terminal CPU 191 determines "yes" in step 4220, it initiates step 4225 and executes the gear shift vibration processing. Thus, the operation terminal 100 vibrates in gear shift vibration mode. Next, the terminal CPU 191 initiates step 4295, temporarily terminating the processing of this routine.

[0343] On the other hand, in a case where the terminal CPU 191 determines "NO" in step 4220, the process is made to proceed directly to step 4295, and the process of the present routine is temporarily ended.

[0344] In addition, in a case where the terminal CPU 191 determines "NO" in step 4205, the process is also made to proceed directly to step 4295, and the process of the present routine is temporarily ended.

[0345] Alternatively, in a case where the vehicle CPU 291 executes the routine illustrated in FIG. 43, when it becomes a predetermined timing, the vehicle CPU 291 starts the process from step 4300 of the routine illustrated in FIG. 43, and makes the process proceed to step 4305, and determines whether it is in a period during which remote travel control is executed. Figure 43 Figure 43 In a case where the vehicle CPU 291 determines "YES" in step 4305, the process is made to proceed to step 4310, and it is determined whether the own vehicle 200 is in travel (in forward travel or in reverse travel). In a case where the vehicle CPU 291 determines "YES" in step 4310, the process is made to proceed to step 4315, and it is determined whether an obstacle is detected. In a case where the vehicle CPU 291 determines "YES" in step 4315, the process is made to proceed to step 4320, and an obstacle vibration request signal S44 is transmitted to the outside via the vehicle transceiver 280. Subsequently, the vehicle CPU 291 makes the process proceed to step 4340.

[0346] On the other hand, in a case where the vehicle CPU 291 determines "NO" in step 4315, the process is made to proceed to step 4325, and it is determined whether braking is being performed on the own vehicle 200. In a case where the vehicle CPU 291 determines "YES" in step 4325, the process is made to proceed to step 4330, and a braking vibration request signal S43 is transmitted to the outside via the vehicle transceiver 280. Subsequently, the vehicle CPU 291 makes the process proceed to step 4340.

[0347] On the other hand, in a case where the vehicle CPU 291 determines "NO" in step 4325, the process is made to proceed to step 4335, and a travel vibration request signal S41 is transmitted to the outside via the vehicle transceiver 280. Subsequently, the vehicle CPU 291 makes the process proceed to step 4340.

[0348] In addition, in a case where the vehicle CPU 291 determines "NO" in step 4310, the process is made to proceed directly to step 4340.

[0349] In addition, in a case where the vehicle CPU 291 determines "NO" in step 4310, the process is made to proceed directly to step 4340.

[0350] ​When the vehicle CPU 291 initiates step 4340, it determines whether the gear position of the transmission device 224 has changed from forward drive to reverse drive or vice versa. If the vehicle CPU 291 determines "yes" in step 4340, it initiates step 4345 and sends a gear change vibration request signal S42 to the outside via the vehicle transceiver 280. Then, the vehicle CPU 291 initiates step 4395, temporarily terminating the current routine.

[0351] On the other hand, if the vehicle CPU 291 determines "no" in step 4340, the processing will proceed directly to step 4395, temporarily terminating the processing of this routine.

[0352] In addition, if the vehicle CPU291 determines "no" in step 4305, the process will directly proceed to step 4395, temporarily ending the processing of this routine.

[0353] On the other hand, the terminal CPU191 executes the process of the vehicle CPU291. Figure 43 In the case of the example shown, it is executed in a predetermined computation cycle. Figure 44 The example shown. Therefore, when the scheduled time is reached, the terminal CPU191 starts from... Figure 44 The routine shown begins processing at step 4400, which leads to step 4405, where it is determined whether the remote driving application has been started.

[0354] If the terminal CPU 191 determines "yes" in step 4405, it proceeds to step 4410 to determine whether an obstacle vibration request signal S44 has been received. If the terminal CPU 191 determines "yes" in step 4410, it proceeds to step 4415 to perform obstacle vibration processing. Thus, the operation terminal 100 vibrates in obstacle vibration mode. Next, the terminal CPU 191 proceeds to step 4420.

[0355] On the other hand, if the terminal CPU191 determines "no" in step 4410, the processing will proceed directly to step 4420.

[0356] When the terminal CPU 191 enters step 4420, it determines whether a braking vibration request signal S43 has been received. If the terminal CPU 191 determines "yes" in step 4420, it proceeds to step 4425 to perform braking vibration processing. Thus, the operating terminal 100 vibrates in braking vibration mode. Next, the terminal CPU 191 proceeds to step 4430.

[0357] On the other hand, in a case where the terminal CPU 191 determines "No" in step 4420, the process is made to proceed directly to step 4430.

[0358] When the terminal CPU 191 proceeds to step 4430, it determines whether or not the travel vibration request signal S41 has been received. In a case where the terminal CPU 191 determines "Yes" in step 4430, the process proceeds to step 4435, and travel vibration processing is executed. Thus, the operation terminal 100 vibrates in the travel vibration mode. Next, the terminal CPU 191 makes the process proceed to step 4440.

[0359] On the other hand, in a case where the terminal CPU 191 determines "No" in step 4430, the process is made to proceed directly to step 4440.

[0360] When the terminal CPU 191 makes the process proceed to step 4440, it determines whether or not the gear change vibration request signal S42 has been received. In a case where the terminal CPU 191 determines "Yes" in step 4440, the process proceeds to step 4445, and gear change vibration processing is executed. Thus, the operation terminal 100 vibrates in the gear change vibration mode. Next, the terminal CPU 191 makes the process proceed to step 4495, and temporarily ends the process of the present routine.

[0361] On the other hand, in a case where the terminal CPU 191 determines "No" in step 4440, the process is made to proceed directly to step 4495, and the process of the present routine is temporarily ended.

[0362] In addition, in a case where the terminal CPU 191 determines "No" in step 4405, the process is also made to proceed directly to step 4495, and the process of the present routine is temporarily ended.

[0363] The above is the specific operation of the remote travel system 10.

[0364] Furthermore, the present application is not limited to the above-described embodiments, and various modifications can be made within the scope of the present application.

Claims

1. A computer program product comprising a remote driving application, the remote driving application being installed on the terminal control device for wireless communication between a vehicle control device of a vehicle and a terminal control device of an operating terminal, and for causing the vehicle control device to perform remote driving control to enable the vehicle to drive autonomously in response to an operation on the operating terminal. The remote driving application is configured such that, when the vehicle control device performs the remote driving control, the operating terminal vibrates or outputs a notification tone based on the control status of the vehicle itself. In the remote driving application, when the vehicle control device performs the remote driving control and the control state of the vehicle itself is in a state that changes the gear state of the vehicle's transmission, the operating terminal generates a vibration with a different vibration mode than when the control state of the vehicle itself is in a state other than when the gear state of the vehicle's transmission is changed, or outputs a notification tone with a different output mode than when the control state of the vehicle itself is in a state other than when the gear state of the vehicle's transmission is changed.

2. The computer program product according to claim 1, wherein the remote driving application is configured as follows: When the vehicle control device performs the remote driving control and the vehicle's control state is in a state of driving, the operating terminal generates a vibration of a predetermined driving vibration mode or outputs a notification tone of a predetermined driving notification tone output mode from the operating terminal. When the vehicle control device performs the remote driving control and the control state of the vehicle is in a state of changing the gear state of the vehicle's transmission device, the operating terminal generates a vibration of a gear change vibration mode different from the driving vibration mode, or outputs a notification tone of a gear change notification tone output mode different from the driving notification tone output mode from the operating terminal.

3. The computer program product according to claim 1, wherein the remote driving application is configured as follows: When the vehicle control device performs the remote driving control and the vehicle is in a braking state, the operating terminal generates a predetermined braking vibration pattern or outputs a predetermined braking notification tone. When the vehicle control device performs the remote driving control and the control state of the vehicle itself is in a state of changing the gear state of the vehicle's transmission device, the operation terminal generates a vibration of a gear change vibration mode different from the braking vibration mode, or outputs a notification tone of a gear change notification tone output mode different from the braking notification tone output mode from the operation terminal.

4. The computer program product according to claim 1, wherein the remote driving application is configured as follows: When the vehicle control device detects an obstacle around its own vehicle while performing the remote driving control, it causes the operating terminal to generate a vibration mode different from the vibration mode generated by the operating terminal according to the control state of its own vehicle, or outputs a notification tone from the operating terminal in a different mode than the output tone output from the operating terminal according to the control state of its own vehicle.

5. A remote driving system for wirelessly communicating between a vehicle control device of its own vehicle and a terminal control device of an operating terminal, and for causing the vehicle control device to perform remote driving control to enable the own vehicle to drive autonomously in response to an operation of the operating terminal. The remote driving system is configured such that, when the vehicle control device performs the remote driving control, the operating terminal vibrates or outputs a notification tone based on the control status of the vehicle itself. In the remote driving system, the terminal control device is configured such that, when the vehicle control device performs the remote driving control and the control state of the vehicle itself is in a state that changes the gear position of the vehicle's transmission, the operating terminal generates a vibration with a different vibration mode than when the control state of the vehicle itself is in a state other than when changing the gear position of the vehicle's transmission, or outputs a notification tone with a different output mode than when the control state of the vehicle itself is in a state other than when changing the gear position of the vehicle's transmission.

Images