Remote driving system and remote driving application software

Abstract

To provide remote driving application software that allows easy touch operation for remote driving control. [Solution] The remote driving system includes an operation terminal 100 having a touch-sensitive portion P34 that has the function of detecting touch by a human finger, and a vehicle control means configured to execute remote driving control to automatically drive the vehicle to a target location Pin_tgt, the control means being configured to execute the remote driving control when a control execution command requesting execution of the remote driving control is received from the operation terminal. When a finger moves on the touch-sensitive portion while remaining in contact with the touch-sensitive portion, the operation terminal transmits the control execution command if the movement of the finger on the touch-sensitive portion satisfies predetermined touch operation conditions. The predetermined touch operation conditions do not include a condition that the trajectory of the finger movement on the touch-sensitive portion follows a trajectory of a specific, defined shape.

Description

[Technical field]

[0001] The present invention relates to a remote driving system and remote driving application software for remotely driving a vehicle using a portable operation terminal. [Background technology]

[0002] A remote parking system is known in which a user operates a portable operating terminal to automatically drive a vehicle and park it in a designated parking space without the driver having to perform any driving operation (hereinafter referred to as "remote parking control") (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-130978 A Summary of the Invention

[0004] In recent years, operation terminals equipped with displays that have the function of detecting the part of the display touched by the user's finger have become widespread. By performing an operation in which the user touches the display of such an operation terminal with his / her finger (hereinafter referred to as a "touch operation"), the remote driving system can perform remote parking control.

[0005] In this case, a predetermined condition is generally imposed on the touch operation by the user to perform the remote parking control. For example, if the predetermined condition is that the moving trajectory of the finger on the display is a trajectory of a specific prescribed shape, the user needs to move the finger on the display along the trajectory of the specific prescribed shape. This makes it very cumbersome for the user to perform the touch operation to perform the remote parking control.

[0006] The present invention has been made to address the above-mentioned problems. That is, one of the objects of the present invention is to provide a remote driving system that allows easy touch operations required for remote driving control, including remote parking control that allows a vehicle to automatically drive without a driver's operation.

[0008] The remote driving system according to the present invention includes an operation terminal having a touch sensing part that has a function of sensing touch by a human finger, and a vehicle. The vehicle detects the movement of a user's finger on the touch sensing part of the operation terminal. and satisfying a predetermined touch operation condition that reduces the possibility that the user will mistakenly perform remote driving control to move the vehicle to a target location. While the random trajectory is freely performed, Note receiving a control execution command for requesting execution of a remote driving control; and detecting the movement of the user's finger on the touch sensing portion of the operation terminal, which is moved by the remote driving control. The predetermined touch operation condition is satisfied If the movement is not freely performed along a random trajectory, the movement by the remote travel control is not performed.

[0011] In the remote driving system according to the present invention, the trajectory of the finger movement for remote driving control is a random trajectory. Touch-sensing part of The remote driving control can be performed by freely moving a finger on the touch screen. Therefore, the touch operation required for the user to perform the remote driving control is simple.

[0017] The predetermined touch operation condition is: The random trajectory but , e.g., a trajectory that includes at least one bend Including the condition .

[0018] If a user mistakenly operates the device Touch-sensing part of When a user touches the touch screen with a finger, the possibility that the trace of the touched finger includes a bend is small. Therefore, if the random trace includes at least one bend, the user may mistakenly touch the touch screen with a finger. Touch-sensing part ofWhen touching the touch screen with a finger, the probability that the random trajectory includes at least one bend is small. Therefore, the possibility that the user will erroneously control the remote driving can be reduced.

[0019] also, The predetermined touch operation condition is: The random trajectory but For example, the trajectory includes a trajectory of the finger that moves a predetermined distance or more within a predetermined time. Including the condition .

[0020] If a user mistakenly operates the device Touch-sensing part of When a user touches the touching area with a finger, the distance that the touching finger moves within a predetermined time is short. Therefore, if the random trajectory includes a trajectory of a finger that moves a predetermined distance or more within a predetermined time, the user may mistakenly touch the operation terminal Touch-sensing part of When the user touches the touch panel with a finger, the possibility that the random trajectory will include a trajectory of a finger that moves a predetermined distance or more within a predetermined time is small. Therefore, the possibility that the user will erroneously perform remote driving control can be reduced.

[0021] also, The predetermined touch operation condition is: The random trajectory but For example, the trajectory includes the trajectory of the finger moving continuously at a speed equal to or faster than a predetermined speed for a predetermined period of time or more. Including the condition .

[0022] If a user mistakenly operates the device Touch-sensing part of When a user touches the touch panel with a finger, the possibility that the speed of the finger movement will continue to be equal to or faster than a predetermined speed for a predetermined period of time or more is small. Therefore, if the random trajectory includes a trajectory of a finger moving continuously at a speed equal to or faster than a predetermined speed for a predetermined period of time or more, the user may mistakenly touch the touch panel with a finger. Touch-sensing part of When the user touches the touch panel with a finger, the possibility that the random trajectory will include a trajectory of a finger moving continuously for a predetermined period of time or more at a speed equal to or faster than a predetermined speed is small. Therefore, the possibility that the user will erroneously perform remote driving control can be reduced.

[0023] In the remote driving system according to the present invention, the operation terminal may be configured to include a display for displaying an image. In this case, the operation terminal may include: The touch sensing portion The display may be configured to display an image partitioning a specified area as an information image in a lower region of the display, and an image representing information regarding the driving of the vehicle under the remote driving control in an upper region of the display.

[0025] By displaying information about the vehicle's driving under remote driving control on the display of the operation terminal in this manner, the user can perform remote driving control more safely.

[0026] Furthermore, in the remote traveling system according to the present invention, In the end , the vehicle Both A sensor arranged on the vehicle to detect obstacles to travel Sa The sensor image is an image that represents Statue The sensor image may be configured to be displayed as a part of the information image in a predetermined display form. In this case, the predetermined display form is at least one of a form in which the sensor image is displayed in a color different from a color of an image other than the sensor image and a form in which the sensor image is displayed by blinking.

[0028] In this way, when a target object that is an obstacle to the vehicle's travel is detected, the sensor image is displayed on the operation terminal in a color different from the color of images other than the sensor image, or is displayed on the operation terminal in a blinking manner, so that the user can easily recognize the presence of the target object that is an obstacle to the vehicle's travel. This allows the user to perform remote travel control more safely.

[0029] Furthermore, in the remote traveling system according to the present invention, In the end , the vehicle Both From the above target Place A picture showing distance in Statue , may be configured to be displayed as part of the information image.

[0031] In this way, an image showing the distance from the vehicle to the destination is displayed as a part of the information image, so that the user can know the distance from the vehicle to the destination.

[0032] Furthermore, in the remote traveling system according to the present invention, In the end , the touch sensing unit minutes above The movement of the finger at Trajectory The above Random trajectories If not If, The control execution command may be configured not to be issued. vehicle If the control execution command is not sent from the operation terminal, If so, stop It can be configured as follows.

[0035] The trajectory of finger movement is not a random trajectory. In this case, it is highly likely that the user has no intention of driving the vehicle by remote driving control. Therefore, in this case, it is preferable to stop the vehicle. The trajectory of finger movement is not a random trajectory. By configuring the vehicle so that a control execution command is not sent from the operation terminal in such a case, it is possible to stop the vehicle when it is preferable to stop the vehicle.

[0036] Further, the remote driving control may, for example, determine a parking space as the target location. Place and And set the car Both Control of moving the vehicle to the parking space and controlling a location outside the parking space as the target location. Place and and moving the vehicle parked in the parking space to the target location.

[0037] This allows a user to park a vehicle in a parking space, or move a vehicle parked in a parking space out of the parking space, by using remote driving control.

[0038] Furthermore, in the remote traveling system according to the present invention, End and The above vehicle may be configured to communicate with each other via wireless communication.

[0040] In this way, the operation terminal vehicle By configuring the vehicle and the control device to communicate with each other via wireless communication, a user can directly control the vehicle to automatically drive by remote driving control using an operation terminal.

[0041] Furthermore, in the remote traveling system according to the present invention, In the end , Internet On Server located at And The device may be configured to communicate via wireless communication.

[0043] By configuring the operation terminal to be able to communicate with the server via wireless communication in this manner, the operation terminal can utilize the server, for example, when causing the vehicle to travel automatically through remote travel control.

[0044] Furthermore, in the remote driving system according to the present invention, vehicle The Internet On Server located at And The device may be configured to communicate via wireless communication.

[0045] Like this vehicle By configuring it to communicate with the server ,example For example, a server can be used to automatically drive a vehicle through remote driving control.

[0046] Books Other objects, features and attendant advantages of the present invention will become readily apparent from the following description of the embodiments of the present invention which is given with reference to the accompanying drawings. [Brief description of the drawings]

[0047] [Figure 1] FIG. 1 is a diagram showing a remote driving system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an operation terminal and its ECU etc. included in the remote driving system shown in FIG. [Diagram 3] FIG. 3 is a diagram showing a vehicle and its ECU etc. included in the remote driving system shown in FIG. [Figure 4] FIG. 4 is a diagram showing a vehicle to which the remote driving system according to the embodiment of the present invention is applied. [Diagram 5] FIG. 5 is a diagram showing an operation terminal on which an entry / exit selection image is displayed. [Figure 6] FIG. 6 is a diagram showing an operation terminal on whose display a parking space selection image is displayed. [Figure 7] FIG. 7 is a diagram showing an operation terminal on which a storage touch operation image is displayed. [Figure 8] FIG. 8 is a diagram showing an operation terminal on which a storage touch operation image is displayed. [Figure 9] FIG. 9 is a diagram showing the movement trajectory of the user's finger on the entry touch operation portion. [Figure 10] FIG. 10 is a diagram showing an operation terminal on which a storage touch operation image is displayed. [Figure 11] FIG. 11 is a diagram showing an operation terminal on which a storage touch operation image is displayed. [Figure 12] FIG. 12 is a diagram showing an operation terminal on which an image for selecting a leaving direction is displayed. [Figure 13] FIG. 13 is a diagram showing an operation terminal on which a delivery touch operation image is displayed. [Figure 14] FIG. 14 is a diagram showing an operation terminal on which a delivery touch operation image is displayed. [Figure 15]FIG. 15 is a diagram showing an operation terminal on which a delivery touch operation image is displayed. [Figure 16] FIG. 16 is a diagram showing a remote driving system according to a modified example of the embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing a remote driving system according to another modified example of the embodiment of the present invention. [Figure 18] FIG. 18 is a flowchart showing a routine executed by the CPU of the ECU of the operation terminal shown in FIG. [Figure 19] FIG. 19 is a flowchart showing a routine executed by the CPU of the operation terminal ECU shown in FIG. [Figure 20] FIG. 20 is a flowchart showing a routine executed by the CPU of the ECU of the operation terminal shown in FIG. [Figure 21] FIG. 21 is a flowchart showing a routine executed by the CPU of the ECU of the vehicle shown in FIG. [Figure 22] FIG. 22 is a flowchart showing a routine executed by the CPU of the ECU of the vehicle shown in FIG. [Diagram 23] FIG. 23 is a flowchart showing a routine executed by the CPU of the ECU of the vehicle shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0048] Hereinafter, a remote traveling system according to an embodiment of the present invention will be described with reference to the drawings. Hereinafter, the remote traveling system will be described using a remote parking entry / exit system, which is one of the remote traveling systems, as an example. The remote parking entry / exit system is a system in which a user operates a portable operation terminal to remotely and automatically drive a vehicle to park it in a specified parking space without the driver operating the vehicle, or remotely and automatically drive a vehicle parked in a parking space to move it out of the parking space.

[0049] In the following description, "entering" means parking a vehicle in a parking space, and "leaving" means taking a vehicle out of the parking space. The remote traveling system according to the present invention includes, in addition to the remote entering and leaving system, a system for remotely moving a vehicle to a predetermined location, for example.

[0050] As shown in FIG. 1, a remote driving system 10 according to an embodiment of the present invention (hereinafter referred to as "this driving system 10") includes an operation terminal 100 and a vehicle 200 (particularly, an ECU 290 of the vehicle 200).

[0051] In this example, the operation terminal 100 is a smartphone, which is one type of portable telephone, but it may also be a device such as a smart key or a dedicated terminal equipped with a display having a function of detecting touch by a human finger. Also, the part having a function of detecting touch by a human finger does not necessarily have to be a display.

[0052] <Operation terminal> As shown in FIG. 2, the operation terminal 100 includes a display 110, a terminal wireless transceiver 130, and an ECU 190 (hereinafter referred to as "terminal ECU 190").

[0053] The display 110 displays various images, which will be described later. The display 110 is electrically connected to the terminal ECU 190. The terminal ECU 190 can control the images displayed on the display 110.

[0054] The terminal wireless transceiver 130 is electrically connected to the terminal ECU 190. The terminal wireless transceiver 130 transmits various signals transmitted by the terminal ECU 190 to the outside of the operation terminal 100. Furthermore, the terminal wireless transceiver 130 receives various signals transmitted by an ECU 290 of the vehicle 200 (described later) to the outside of the vehicle 200 via a vehicle wireless transceiver 279, and transmits the signals to the terminal ECU 190.

[0055] The terminal ECU 190 includes a microcomputer as a main part. The microcomputer includes a CPU 191, a ROM, a RAM, a non-volatile memory, an interface, etc. The CPU 191 is configured to realize various functions by executing instructions (programs, routines) stored in the ROM.

[0056] The terminal ECU 190 has installed therein remote driving application software (hereinafter referred to as a "remote driving application") for causing the ECU 290 of the vehicle 200 to perform remote driving control, which will be described in detail later.

[0057] <Vehicle configuration> As shown in FIG. 3, vehicle 200 is equipped with ECU 290 (hereinafter referred to as "vehicle ECU 290"). ECU is an abbreviation for electronic control unit. Vehicle ECU 290 includes a microcomputer as a main part. The microcomputer includes a CPU 291, ROM, RAM, non-volatile memory, an interface, etc. CPU 291 is configured to realize various functions by executing instructions (programs, routines) stored in the ROM.

[0058] As shown in FIG. 3, the vehicle 200 includes an internal combustion engine 210, a first motor generator 221, a second motor generator 222, a power control unit 223, a battery 224, a braking device 230, a power steering device 240, and a door lock mechanism 250.

[0059] In this example, the vehicle 200 is a so-called plug-in hybrid vehicle (PHV) that runs using power output from at least one of an internal combustion engine 210, a first motor generator 221, and a second motor generator 222 as driving force and can charge the battery 224 with electricity from an external power source.

[0060] However, the traveling system 10 can also be applied to a vehicle equipped with only an internal combustion engine as a driving force source that supplies the vehicle 200 with driving force for traveling the vehicle 200. The traveling system 10 can also be applied to a so-called hybrid vehicle (HV) equipped with an internal combustion engine and a motor as driving force sources, and to a so-called electric vehicle (EV) equipped with only a motor as a driving force source without an internal combustion engine.

[0061] The traveling system 10 is also applicable to a so-called fuel cell vehicle (FCV) that has a motor as a driving force source and uses electricity generated by a fuel cell to drive the motor. The traveling system 10 is also applicable to a so-called in-wheel motor type vehicle that has a motor as a driving force source and has a motor arranged for each wheel, and each wheel is rotated by the motor arranged corresponding to the wheel.

[0062] <Internal combustion engine> The internal combustion engine 210 is a driving force source that provides the vehicle 200 with driving force for running the vehicle 200, and in this example is a well-known compression ignition type multi-cylinder internal combustion engine (a so-called diesel engine). Note that the internal combustion engine 210 may also be a well-known spark ignition type multi-cylinder internal combustion engine (a so-called gasoline engine).

[0063] The internal combustion engine 210 includes a plurality of combustion chambers (not shown), fuel injection valves 211 that inject fuel into each of the combustion chambers, a fuel injection valve actuator 212 that controls the operation of the fuel injection valves 211, and the like.

[0064] The fuel injection valve actuator 212 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can control the amount of fuel injected from the fuel injection valve 211 (hereinafter referred to as the "fuel injection amount") by controlling the operation of the fuel injection valve actuator 212. The vehicle ECU 290 can control the torque generated by the internal combustion engine 210 (hereinafter referred to as the "engine torque") by controlling the fuel injection amount. The larger the fuel injection amount, the larger the engine torque. The engine torque is transmitted to the driving wheels of the vehicle 200 via a transmission (not shown) and a drive shaft (not shown).

[0065] A power output shaft of the internal combustion engine 210 is connected to a power split mechanism (not shown) which will be described later. The power generated by the internal combustion engine 210 is input to the power split mechanism.

[0066] <Motor generators, etc.> The first motor generator 221 and the second motor generator 222 are each a permanent magnet type synchronous motor, and are electrically connected to a battery 224 via a power control unit 223 .

[0067] The power control unit 223 (hereinafter referred to as "PCU 223") includes an inverter, a boost converter, a DC / DC converter, and the like.

[0068] A rotating shaft of the first motor generator 221 (hereinafter referred to as "first MG 221") is connected to a power split mechanism (not shown). The first MG 221 is mainly used as a generator. When the first MG 221 is used as a generator, its rotating shaft is rotated by an external force such as the traveling energy of the vehicle 200 or the power output by the internal combustion engine 210, thereby generating electric power. The generated electric power is charged into the battery 224 via the PCU 223. The first MG 221 is also used as an electric motor. When the first MG 221 is used as an electric motor, the first MG 221 is driven by electric power supplied from the battery 224 via the PCU 223.

[0069] A rotating shaft of the second motor generator 222 (hereinafter referred to as the "second MG 222") is also connected to a power split mechanism (not shown). The second MG 222 is mainly used as an electric motor. When the second MG 222 is used as an electric motor, the second MG 222 is driven by electric power supplied from the battery 224 via the PCU 223. The second MG 222 is also used as a generator. When the second MG 222 is used as a generator, the rotating shaft of the second MG 222 is rotated by the above-mentioned external force, and the second MG 222 generates electric power. The generated electric power is charged to the battery 224 via the PCU 223.

[0070] The power split mechanism is, for example, a planetary gear mechanism. When the first MG 221 is used as a generator and the second MG 222 is used as an electric motor, the power split mechanism splits the power input from the internal combustion engine 210 to the power split mechanism at a predetermined ratio, and transmits the power to the drive wheels of the vehicle 200 as power for running the vehicle 200 and transmits the power to the first MG 221 as power for driving the first MG 221 as a generator. Furthermore, the power split mechanism transmits the power input from the second MG 222 to the power split mechanism to the drive wheels of the vehicle 200. The power split mechanism itself is known (for example, see JP 2013-177026 A, etc.).

[0071] 3, the PCU 223 is electrically connected to the vehicle ECU 290. The operation of the PCU 223 is controlled by the vehicle ECU 290. The vehicle ECU 290 can control the driving of the first MG 221 and the driving of the second MG 222 by controlling the operation of the PCU 223.

[0072] <Brake device> The brake device 230 includes friction brake mechanisms 231 provided corresponding to each wheel of the vehicle 200, friction brake actuators 232 provided corresponding to each friction brake mechanism 231, and hydraulic oil passages (not shown) provided corresponding to each friction brake actuator 232, etc.

[0073] Each friction brake actuator 232 is connected to a brake caliper (not shown) of the corresponding friction brake mechanism 231 via a corresponding hydraulic oil passage. Each friction brake actuator 232 is configured to supply hydraulic oil pressurized by a master cylinder (not shown) to the corresponding friction brake mechanism 231 (in this example, particularly to the brake caliper of the corresponding friction brake mechanism 231) via the corresponding hydraulic oil passage.

[0074] When hydraulic oil is supplied to each friction brake mechanism 231, a brake pad (not shown) of a brake caliper of each friction brake mechanism 231 is pressed against a brake disc (not shown), thereby applying a braking force to each wheel.

[0075] Each friction brake actuator 232 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can control the pressure of hydraulic oil (hereinafter referred to as "brake hydraulic pressure") supplied to each friction brake mechanism 231 by controlling the operation of each friction brake actuator 232. The greater the brake hydraulic pressure applied to each friction brake mechanism 231, the greater the braking force applied to each wheel.

[0076] The braking device 230 further includes a parking brake 233 and a parking brake actuator 234 .

[0077] The parking brake actuator 234 is connected to the parking brake 233. When the parking brake actuator 234 activates the parking brake 233, each wheel of the vehicle 200 is locked.

[0078] The parking brake actuator 234 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can control the operation of the parking brake 233 by controlling the operation of the parking brake actuator 234.

[0079] <Power steering device> The power steering device 240 includes a steering motor 241 and a motor driver 242. The motor driver 242 is connected to the steering motor 241. The steering motor 241 generates torque by electric power supplied from the motor driver 242, and can apply this torque to a steering shaft 243.

[0080] Motor driver 242 is electrically connected to vehicle ECU 290. Vehicle ECU 290 controls the operation of motor driver 242 to control the torque applied from steering motor 241 to steering shaft 243.

[0081] The door lock mechanism 250 is a mechanism for locking each door of the vehicle 200. The door lock mechanism 250 is electrically connected to the vehicle ECU 290. The vehicle ECU 290 can control the operation of the door lock mechanism 250. When the vehicle ECU 290 activates the door lock mechanism 250, the door lock mechanism 250 locks each door of the vehicle 200. On the other hand, when the vehicle ECU 290 releases the operation of the door lock mechanism 250, the door lock mechanism 250 releases the lock of each door.

[0082] <Sensors, etc.> Furthermore, the vehicle 200 is equipped with an accelerator pedal operation amount sensor 271, a brake pedal operation amount sensor 272, a steering angle sensor 273, a steering torque sensor 274, a wheel speed sensor 275, a radar sensor 276, four cameras 277, twelve clearance sonars 278, and a vehicle wireless transceiver 279.

[0083] Accelerator pedal operation amount sensor 271 is electrically connected to vehicle ECU 290. Accelerator pedal operation amount sensor 271 detects the operation amount of accelerator pedal 281, and transmits a signal indicating the detected operation amount to vehicle ECU 290. Vehicle ECU 290 obtains the operation amount of accelerator pedal 281 as "accelerator pedal operation amount AP" based on the signal.

[0084] Brake pedal operation amount sensor 272 is electrically connected to vehicle ECU 290. Brake pedal operation amount sensor 272 detects the operation amount of brake pedal 282, and transmits a signal indicating the detected operation amount to vehicle ECU 290. Vehicle ECU 290 obtains the operation amount of brake pedal 282 as a "brake pedal operation amount BP" based on the signal.

[0085] Steering angle sensor 273 is connected to vehicle ECU 290. Steering angle sensor 273 detects the steering angles of the left and right front wheels, which are steered wheels of vehicle 200, and transmits a signal indicating the detected steering angles to vehicle ECU 290. Vehicle ECU 290 acquires the steering angles of the left and right front wheels of vehicle 200 as "steering angle θst" based on the signal.

[0086] The steering torque sensor 274 is electrically connected to the vehicle ECU 290. The steering torque sensor 274 detects the torque applied to the steering shaft 243 due to the operation of the steering wheel 244 by the driver, and transmits a signal representing the detected torque to the vehicle ECU 290. Based on the signal, the vehicle ECU 290 obtains the torque applied to the steering shaft 243 as a "driver steering torque TQdriver."

[0087] Wheel speed sensor 275 is electrically connected to vehicle ECU 290. Wheel speed sensor 275 is provided corresponding to each wheel of vehicle 200, detects the rotation speed of each wheel, and transmits a signal representing the detected rotation speed to vehicle ECU 290. Vehicle ECU 290 obtains the rotation speed of each wheel based on the signals, and obtains speed SPD of vehicle 200 based on the obtained rotation speeds.

[0088] As shown in FIG. 4, one of the four cameras 277 is attached to the front 200F of the vehicle 200 on the widthwise center line Lx of the vehicle 200 so as to capture the scenery ahead of the vehicle 200. Another camera 277 is attached to the rear 200R of the vehicle 200 on the widthwise center line Lx so as to capture the scenery behind the vehicle 200. Still another camera 277 is attached in front of the longitudinal center line Ly of the vehicle 200 near the left side mirror 200ML of the vehicle 200 so as to capture the scenery on the left side of the vehicle 200. Still another camera 277 is attached in front of the longitudinal center line Ly of the vehicle 200 near the right side mirror 200MR of the vehicle 200 so as to capture the scenery on the right side of the vehicle 200.

[0089] Camera 277 is electrically connected to vehicle ECU 290. Camera 277 transmits image data acquired by photographing the scenery around vehicle 200 to vehicle ECU 290. Vehicle ECU 290 recognizes (acquires) the presence or absence of targets (e.g., other vehicles, pillars, pedestrians, etc.) present around vehicle 200 and the relative relationship between vehicle 200 and the targets, etc., based on the image data.

[0090] Four of the twelve clearance sonars 278 are mounted on the front 200F of the vehicle 200. Another four of the twelve clearance sonars 278 are mounted on the rearmost 200R of the vehicle 200. Another two of the twelve clearance sonars 278 are mounted on the left side 200SL of the vehicle 200. The remaining two of the twelve clearance sonars 278 are mounted on the right side 200SR of the vehicle 200.

[0091] One of the four clearance sonars 278 attached to the front 200F of the vehicle 200 is attached to the left of the widthwise center line Lx of the vehicle 200 and near the widthwise center line Lx. Another clearance sonar 278 is attached to the left of the widthwise center line Lx of the vehicle 200 and near the left end of the front 200F of the vehicle 200 away from the widthwise center line Lx. Yet another clearance sonar 278 is attached to the right of the widthwise center line Lx of the vehicle 200 and near the widthwise center line Lx. Another clearance sonar 278 is attached to the right of the widthwise center line Lx of the vehicle 200 and near the right end of the front 200F of the vehicle 200 away from the widthwise center line Lx.

[0092] One of the four clearance sonars 278 attached to the rearmost portion 200R of the vehicle 200 is attached to the left of the widthwise center line Lx of the vehicle 200 and near the widthwise center line Lx. Another clearance sonar 278 is attached to the left of the widthwise center line Lx of the vehicle 200 and near the left end of the rearmost portion 200R of the vehicle 200 away from the widthwise center line Lx. Yet another clearance sonar 278 is attached to the right of the widthwise center line Lx of the vehicle 200 and near the widthwise center line Lx. Another clearance sonar 278 is attached to the right of the widthwise center line Lx of the vehicle 200 and near the right end of the rearmost portion 200R of the vehicle 200 away from the widthwise center line Lx.

[0093] One of the two clearance sonars 278 attached to the left side 200SL of the vehicle 200 is attached to the left side 200SL in front of and relatively far from the longitudinal center line Ly of the vehicle 200. The remaining clearance sonar 278 is attached to the left side 200SL in rear of and relatively far from the longitudinal center line Ly of the vehicle 200.

[0094] One of the two clearance sonars 278 attached to the right side 200SR of the vehicle 200 is attached to the right side 200SR in front of and relatively far from the longitudinal center line Ly of the vehicle 200. The remaining clearance sonar 278 is attached to the right side 200SR behind and relatively far from the longitudinal center line Ly of the vehicle 200.

[0095] The clearance sonar 278 is electrically connected to the vehicle ECU 290. The clearance sonar 278 emits ultrasonic waves and receives ultrasonic waves reflected by a target (i.e., reflected waves). The clearance sonar 278 transmits a signal to the vehicle ECU 290 indicating the time from when the ultrasonic waves are emitted until the clearance sonar 278 detects the reflected waves. The vehicle ECU 290 obtains the distance from the point on the vehicle 200 where each clearance sonar 278 is attached to the target based on the signal.

[0096] In this example, the twelve clearance sonars 278 are attached to the vehicle 200 with the direction in which each clearance sonar 278 emits ultrasound adjusted so that the ultrasound emitted by the clearance sonars 278 can detect targets all around the vehicle 200.

[0097] The vehicle wireless transceiver 279 is electrically connected to the vehicle ECU 290. The vehicle wireless transceiver 279 transmits various signals transmitted by the vehicle ECU 290 to the outside of the vehicle 200. Furthermore, the vehicle wireless transceiver 279 receives various signals transmitted by the terminal ECU 190 to the outside of the operation terminal 100 via the terminal wireless transceiver 130 and transmits them to the vehicle ECU 290.

[0098] <System startup switch> The system activation switch 280 is a switch that is operated by the driver of the vehicle 200. The system activation switch 280 is electrically connected to the vehicle ECU 290.

[0099] When the system activation switch 280 is set to the ON position by the driver, the vehicle ECU 290 starts supplying power to the camera 277 etc., and prepares to start the internal combustion engine 210 and to drive the first MG 221 and the second MG 222. That is, when the system activation switch 280 is set to the ON position by the driver, the vehicle ECU 290 operates or prepares to operate vehicle driving equipment such as the camera 277, the internal combustion engine 210, the first MG 221, and the second MG 222, which are necessary for the vehicle 200 to travel.

[0100] On the other hand, when the system activation switch 280 is set to the OFF position by the driver, the vehicle ECU 290 stops the power supply to the camera 277 etc., the operation of the internal combustion engine 210, and the driving of the first MG 221 and the second MG 222. In other words, when the system activation switch 280 is set to the OFF position by the driver, the operation of vehicle driving equipment such as the camera 277, the internal combustion engine 210, the first MG 221, the second MG 222, etc., required for driving the vehicle 200, is stopped.

[0101] <Overview of the operation of this system> Next, an outline of the operation of the traveling system 10 will be described. The traveling system 10 is a system that performs remote traveling control in which a person outside the vehicle 200 remotely drives and moves the vehicle 200 from a place where the vehicle is stopped to a predetermined place using the operation terminal 100.

[0102] The remote driving control includes remote entrance control and remote leaving control. The remote entrance control is a control in which the user of the operation terminal 100 remotely drives and moves the vehicle 200 from the place where the vehicle is stopped to a parking space desired by the user of the operation terminal 100 using the operation terminal 100. The remote leaving control is a control in which the user remotely drives and moves the vehicle 200 parked in a parking space from the parking space to a specified place using the operation terminal 100.

[0103] By having the traveling system 10 perform remote entry control or remote exit control, a user of the operation terminal 100 can remotely move the vehicle 200 from the location where it is stopped to a specified location from outside the vehicle 200 without operating the steering wheel 244, accelerator pedal 281, brake pedal 282, etc.

[0104] In the following, the operation of the present traveling system 10 will be explained using as an example a case where a user operates the operation terminal 100 outside the vehicle 200 to cause the present traveling system 10 to perform remote entry control or remote exit control, but the user can also operate the operation terminal 100 inside the vehicle 200 to cause the present traveling system 10 to perform remote entry control or remote exit control.

[0105] In the following description, a "touch operation" refers to an operation in which the user of operation terminal 100 touches display 110 of operation terminal 100 with a finger.

[0106] When a user performs a touch operation to start the remote driving application, the terminal ECU 190 starts the remote driving application and displays an entry / exit selection image G10 on the display 110 as shown in Fig. 5. The entry / exit selection image G10 is an image for allowing the user to select whether to enter the vehicle 200 into a parking space (i.e., to enter the parking space) or to take the vehicle 200 out of the parking space (i.e., to exit the parking space).

[0107] The entry / exit selection image G10 includes an entry selection image G11, an exit selection image G12, and an application exit image G15. The entry selection image G11 is an image displaying the word "enter" and a line surrounding the word. The exit selection image G12 is an image displaying the word "exit" and a line surrounding the word. The application exit image G15 is an image displaying the word "exit" and a line surrounding the word.

[0108] The entry selection image G11 is displayed in an area approximately at the center of the display 110, and the exit selection image G12 is displayed in an area immediately below the entry selection image G11 in an area approximately at the center of the display 110. The application termination image G15 is displayed in the lower right corner area of ​​the display 110.

[0109] Furthermore, when the user performs a touch operation to start the remote driving application, the terminal ECU 190 transmits an application start signal S10 including a signal representing information such as an ID for identifying the operation terminal 100 to the outside.

[0110] When the vehicle ECU 290 receives the application start-up signal S10, the vehicle ECU 290 determines whether or not the operation terminal 100 represented by the application start-up signal S10 is a registered operation terminal. A registered operation terminal is an operation terminal registered in the vehicle ECU 290 as an operation terminal that causes the vehicle ECU 290 to perform remote driving control.

[0111] Since the operation terminal 100 is a registered operation terminal, the vehicle ECU 290 determines that the operation terminal 100 is a registered operation terminal when receiving the application start signal S10 transmitted from the terminal ECU 190. In this case, the vehicle ECU 290 performs remote driving control based on various signals transmitted from the operation terminal 100 to the outside.

[0112] When the vehicle ECU 290 determines that the operation terminal 100 is a registered operation terminal, it operates or prepares to operate the vehicle running devices such as the camera 277, the internal combustion engine 210, the first MG 221, and the second MG 222.

[0113] <Remote Stock Control> When the user performs a touch operation of touching the entry selection image portion P11 with his / her finger, the terminal ECU 190 transmits an entry selection signal S11 to the outside. The entry selection image portion P11 is a portion P11 of the display 110 that corresponds to the entry selection image G11. The entry selection signal S11 is a signal indicating that the user has performed a touch operation of touching the entry selection image portion P11 with his / her finger.

[0114] When the vehicle ECU 290 receives the entrance selection signal S11 transmitted by the operation terminal 100, it transmits a parking space information signal S12 and a vehicle position information signal S13 to the outside. The parking space information signal S12 is a signal that represents information about a space in which the vehicle 200 can be parked based on image data transmitted from the camera 277 by operating the camera 277. The vehicle position information signal S13 is a signal that represents information such as the position of the vehicle 200 with respect to the space in which the vehicle 200 can be parked.

[0115] When the terminal ECU 190 receives the parking space information signal S12 and the vehicle position information signal S13 transmitted by the vehicle ECU 290, it displays a parking space selection image G20 on the display 110. The parking space selection image G20 is an image that allows the user to select a desired space as a space for parking the vehicle 200.

[0116] The parking space selection image G20 includes a candidate parking space image G21, a vehicle image G22, an application termination image G25, and an initial screen image G26. The candidate parking space image G21 is an image displaying the letter "P" and lines dividing the space in which the vehicle 200 can be parked. The vehicle image G22 is an image displaying the vehicle 200 itself. The application termination image G25 is the same image as the application termination image G15 described above. The initial screen image G26 is an image displaying the letter "initial screen" and lines surrounding the letter.

[0117] The candidate parking space images G21 are displayed vertically in three rows in an area approximately in the center of the display 110, and the vehicle image G22 is displayed in an area of ​​the display 110 to the right of the candidate parking space images G21. The application termination image G25 is displayed in the lower right corner area of ​​the display 110, and the initial screen image G26 is displayed in the lower left corner area of ​​the display 110.

[0118] The locations on the display 110 where the candidate parking space image G21 and the vehicle image G22 are displayed are determined based on the parking space information signal S12 and the vehicle position information signal S13 received by the vehicle ECU 290.

[0119] When a user performs a touch operation of touching any of the parking space image portions P21 with a finger, the terminal ECU 190 transmits a parking space signal S14 to the outside. The parking space image portion P21 is a portion P21 of the display 110 that corresponds to the candidate parking space image G21. The parking space signal S14 is a signal that represents the parking space image portion P21 that the user touched with a finger.

[0120] When the vehicle ECU 290 receives the parking space signal S14 transmitted by the terminal ECU 190, the vehicle ECU 290 starts remote entry control and calculates an entry route. The entry route is a route along which the vehicle 200 travels to park (i.e., enter) the vehicle 200 in the parking space Pin_tgt indicated by the parking space signal S14, i.e., the target parking space Pin_tgt (or a target entry point Pin_tgt that is a target location to which the vehicle 200 is moved and stopped).

[0121] When the vehicle ECU 290 completes the calculation of the entrance route, it transmits an entrance remaining distance signal S15 to the outside. The entrance remaining distance signal S15 is a signal that indicates the distance between the target parking space Pin_tgt and the current position of the vehicle 200.

[0122] The vehicle ECU 290 obtains the distance between the target parking space Pin_tgt and the current position of the vehicle 200 by calculation using the image data transmitted from the camera 277.

[0123] On the other hand, when the user performs a touch operation of touching any of the parking space image portions P21 with a finger, the terminal ECU 190 displays an entry touch operation image G30 on the display 110, as shown in Fig. 7. The entry touch operation image G30 is an image for prompting the user to perform a touch operation to enter the vehicle 200 into the parking space Pin_tgt (i.e., the target parking space Pin_tgt) corresponding to the parking space image portion P21 selected by the user through the touch operation.

[0124] The entry touch operation image G30 includes a target parking space image G31, a vehicle image G32, a remaining entry distance image G33, a touch operation area image G34, an application termination image G35, and an initial screen image G36.

[0125] The target parking space image G31 is an image that displays the target parking space Pin_tgt. The vehicle image G32 is the same image as the vehicle image G22 described above. The remaining entry distance image G33 is an image that displays characters that indicate the distance between the current position of the vehicle 200 and the target parking space Pin_tgt and a line that surrounds the characters. The touch operation area image G34 is an image that partitions an area of ​​a predetermined area of ​​the display 110. The application termination image G35 is the same image as the application termination image G25 described above. The initial screen image G36 is the same image as the initial screen image G26 described above.

[0126] The target parking space image G31 is displayed in a substantially central area of ​​the upper half area 110A of the display 110. The vehicle image G32 is displayed in an area of ​​the display 110 to the right of the target parking space image G31. The remaining entry distance image G33 is displayed in the uppermost area of ​​the upper half area 110A of the display 110. Therefore, in this example, the upper half area 110A of the display 110 is used as an area for displaying information for providing the user with information regarding the entry of the vehicle 200 into the target parking space Pin_tgt (i.e., an information display area).

[0127] The terminal ECU 190 receives the remaining entry distance signal S15 transmitted by the vehicle ECU 290, and acquires the distance indicated by the remaining entry distance signal S15. The terminal ECU 190 displays the acquired distance on the remaining entry distance image G33.

[0128] The touch operation area image G34 is displayed in the lower half area 110B of the display 110. In this example, the area of ​​the touch operation area image G34 (hereinafter referred to as the "predetermined area") occupies most of the lower half area 110B of the display 110.

[0129] However, the area of ​​the touch operation area image G34 may be appropriately set to an area that allows a user to easily move his / her finger while touching the portion P34 of the display 110 corresponding to the touch operation area image G34 (hereinafter referred to as the "touch operation portion P34").

[0130] The application termination image G35 is displayed in the lower right corner of the display 110. The initial screen image G36 is displayed in the lower left corner of the display 110.

[0131] 8, while the user keeps touching the touch operation portion P34 with his / her finger and moves the finger on the touch operation portion P34 so as to satisfy a predetermined touch operation condition Cin (hereinafter referred to as "entry touch operation condition Cin"), the terminal ECU 190 transmits a control execution command signal S16 to the outside. The control execution command signal S16 is a signal indicating that a touch operation that satisfies the entry touch operation condition Cin has been performed.

[0132] The entry touch operation condition Cin is a condition C1in that when the user's finger moves on the touch operation part P34 while touching the touch operation part P34, the movement trajectory of the finger on the touch operation part P34 during a predetermined time T1in includes a predetermined number of bends N1in or more.

[0133] In this example, when the movement vector of the finger on the touch operation part P34 changes by a predetermined angle Ath or more, it is determined that one bending has occurred. For example, when a user moves a finger on the touch operation part P34 in a certain direction and then moves the finger in the opposite direction (i.e., when the user reverses the direction of moving the finger), the angle by which the movement vector of the finger on the touch operation part P34 has changed is approximately 180 degrees. Therefore, if the predetermined angle Ath is set to, for example, 150 degrees, when the user reverses the direction of moving the finger as described above, the movement vector of the finger changes by the predetermined angle Ath or more, and it is determined that one bending has occurred.

[0134] In this way, the entry touch operation condition Cin does not include the user's finger moving in a specific, prescribed path on the touch operation part P34 while touching the touch operation part P34. The specific, prescribed path is, for example, a straight path L1 in the left-right direction shown in Fig. 9(A), a straight path L2 in the up-down direction shown in Fig. 9(B), a straight path L3 in the diagonal direction shown in Fig. 9(C), a circular path L4 shown in Fig. 9(D), or the like.

[0135] Therefore, in this example, the entry touch operation condition Cin is satisfied even if the trajectory traveled by the user on the touch operation part P34 while touching the touch operation part P34 with his / her finger is "a random trajectory including straight lines in the up-down direction, straight lines in the left-right direction, straight lines in the diagonal direction, and curved lines, as well as a combination of at least two of these lines."

[0136] The predetermined time T1in and the predetermined number of times N1in may be set to a time and a number of times appropriate for determining that the user intends to cause the vehicle ECU 290 to perform the remote warehousing control described below. For example, the predetermined number of times N1in may be set to 1 time, and the time when the user is expected to move his / her finger on the touch operation portion P34 along a path including at least one bend when the user intends to cause the vehicle ECU 290 to perform the remote warehousing control may be set as the predetermined time T1in.

[0137] Furthermore, the entry touch operation condition Cin may be a condition C2in that, when the user's finger moves on the touch operation part P34 while touching the touch operation part P34, the moving distance Dtouch of the finger on the touch operation part P34 in a predetermined time T2in is equal to or greater than a predetermined distance D2in. The predetermined time T2in and the predetermined distance D2in may be set to a time and distance suitable for determining that the user intends to cause the vehicle ECU 290 to perform the remote entry control described below. For example, first, a distance suitable for determining that the user intends to cause the vehicle ECU 290 to perform the remote entry control may be set as the predetermined distance D2in, and a time required for the user to move the finger by the predetermined distance D2in on the touch operation part P34 when the user intends to cause the vehicle ECU 290 to perform the remote entry control may be set as the predetermined time T2in.

[0138] Alternatively, the entry touch operation condition Cin may be a condition C3in that, when the user's finger is moved on the touch operation part P34 while touching the touch operation part P34, the time T during which the finger movement speed Vtouch on the touch operation part P34 continues to be equal to or greater than a predetermined speed V3in is equal to or greater than a predetermined time T3in. The predetermined time T3in and the predetermined speed V3in may be set to a time and speed suitable for determining that the user intends to cause the vehicle ECU 290 to perform the remote entry control described below. For example, first, a time suitable for determining that the user intends to cause the vehicle ECU 290 to perform the remote entry control may be set as the predetermined time T3in, and a speed predicted as the speed at which the user will move his / her finger on the touch operation part P34 when the user intends to cause the vehicle ECU 290 to perform the remote entry control may be set as the predetermined speed V3in.

[0139] In addition, any two or all of the above-mentioned conditions C1in, C2in, and C3in may be set as the entry touch operation conditions Cin.

[0140] After completing the calculation of the entry route, while receiving the control execution command signal S16, the vehicle ECU 290 controls the operation of the fuel injection valve actuator 212, the friction brake actuator 232 and the motor driver 242 so that the vehicle 200 travels along the entry route and moves to the target parking space Pin_tgt.

[0141] Furthermore, during execution of the remote entry control, the vehicle ECU 290 continues to transmit to the outside the vehicle position information signal S13, the remaining entry distance signal S15, and a signal S17 indicating the traveling direction of the vehicle 200 (hereinafter referred to as the "traveling direction signal S17").

[0142] The terminal ECU 190 receives the vehicle position information signal S13 transmitted by the vehicle ECU 290, and changes the position of the vehicle image G32 displayed on the display 110 so that the position of the vehicle image G32 corresponds to the position of the vehicle 200 relative to the target parking space Pin_tgt represented by the vehicle position information signal S13, as shown in FIG. 10.

[0143] Furthermore, the terminal ECU 190 receives the remaining entry distance signal S15 transmitted by the vehicle ECU 290, and displays the distance indicated by the remaining entry distance signal S15 on the remaining entry distance image G33.

[0144] Furthermore, the terminal ECU 190 receives a traveling direction signal S17 transmitted by the vehicle ECU 290, and displays an image G37 (hereinafter referred to as the "traveling direction image G37") representing the traveling direction of the vehicle 200 represented by the traveling direction signal S17 in a portion of the display 110 near the vehicle image G32.

[0145] Furthermore, during execution of the remote entry control, if the vehicle ECU 290 detects a target existing within the predetermined distance Dth based on a signal from any of the clearance sonars 278, the vehicle ECU 290 transmits a sonar signal S18 to the outside. The sonar signal S18 is a signal representing the clearance sonar 278 that transmitted a signal representing a target existing within the predetermined distance Dth.

[0146] When the terminal ECU 190 receives the sonar signal S18, it displays the sonar image G38 in a portion of the display 110 near the vehicle image G32, as shown in Fig. 10. The sonar image G38 is an image that represents a portion of the vehicle image G32 that corresponds to the clearance sonar 278 represented by the sonar signal S18.

[0147] The operation terminal 100 is configured to display the sonar image G38 in a color (e.g., red) that can reliably notify the user that a target exists within a range within a predetermined distance Dth from the vehicle 200. In particular, the operation terminal 100 is configured to display the sonar image G38 in a color different from the color of images other than the sonar image G38.

[0148] The operation terminal 100 may be configured to display the sonar image G38 by blinking it. The operation terminal 100 may be configured to emit an alarm sound in addition to displaying the sonar image G38. The operation terminal 100 may be configured to emit an alarm sound when the distance between the vehicle 200 and the target becomes equal to or shorter than a distance Dshort that is shorter than a predetermined distance Dth while the sonar image G38 is being displayed.

[0149] When the entry touch operation condition Cin is no longer satisfied, for example because the user removes his / her finger from the touch operation portion P34, the terminal ECU 190 stops transmitting the control execution command signal S16 to the outside.

[0150] When the vehicle ECU 290 stops receiving the control execution command signal S16, the vehicle ECU 290 causes the friction brake mechanism 231 to stop the vehicle 200.

[0151] When the entry of vehicle 200 is completed by vehicle 200 arriving at target parking space Pin_tgt, vehicle ECU 290 locks each wheel of vehicle 200 using parking brake 233, locks the doors of vehicle 200 using door lock mechanism 250, and stops operation of vehicle driving equipment such as camera 277, internal combustion engine 210, first MG 221, and second MG 222 required for driving of vehicle 200, thereby ending remote entry control.

[0152] Furthermore, when the entry of vehicle 200 is completed, vehicle ECU 290 transmits to the outside a signal S19 indicating that the entry of vehicle 200 is completed (hereinafter referred to as "entry completion signal S19").

[0153] When the terminal ECU 190 receives the entry completion signal S19 transmitted by the vehicle ECU 290, it displays an entry completion image G39 on the display 110 as shown in Fig. 11. The entry completion image G39 is an image indicating that the entry of the vehicle 200 has been completed. The entry completion image G39 is displayed in the uppermost area of ​​the display 110.

[0154] In this traveling system 10, the entry touch operation condition Cin is a condition C1in that when a user's finger moves on the touch operation part P34 while touching the touch operation part P34, the movement trajectory of the finger on the touch operation part P34 during a predetermined time T1in includes a predetermined number of bends N1in or more.

[0155] Therefore, if the user accidentally touches the touch operation part P34 with his / her finger, the possibility that the trajectory of the finger that touched the touch operation part P34 satisfies the above-mentioned entry touch operation condition Cin is extremely small. Therefore, if the user accidentally touches the touch operation part P34 with his / her finger, the possibility that the remote entry control is performed is extremely small. Therefore, according to the present traveling system 10, the remote entry control can be performed only when it is certain that the user intends to enter the vehicle 200 into the target parking space Pin_tgt.

[0156] Furthermore, if there is a problem with the contact sensing function of the display 110 and the terminal ECU 190 recognizes that the touch operation part P34 is being touched even though the user's finger is not touching the touch operation part P34, the possibility that the contact locus recognized by the terminal ECU 190 satisfies the above-described storage touch operation condition Cin is extremely small. Therefore, the possibility that remote storage control is performed when the terminal ECU 190 recognizes that the touch operation part P34 is being touched even though the user's finger is not touching the touch operation part P34 is extremely small. Also from this point, according to the present driving system 10, remote storage control can be performed only when it is certain that the user intends to store the vehicle 200 in the target parking space Pin_tgt.

[0157] Furthermore, as described above, the storage touch operation condition Cin does not include the case where the user's finger moves on the touch operation part P34 while touching the touch operation part P34 and draws a specific specified locus (see FIG. 9). Therefore, the user can store the vehicle 200 in the target parking space Pin_tgt by freely moving the finger on the touch operation part P34 in a random locus. In other words, there is no restriction on how the finger moves when the user touches the touch operation part P34 with the finger in order to store the vehicle 200 in the target parking space Pin_tgt. For this reason, the user can cause the vehicle ECU 290 to perform remote storage control by a simple touch operation. Also, even when the user has to touch the touch operation part P34 with the finger of the hand holding the operation terminal 100, the user can easily perform a touch operation that satisfies the storage touch operation condition Cin.

[0158] Also, since there is no restriction on how the finger moves when the user touches the touch operation part P34 with the finger in order to store the vehicle 200 in the target parking space Pin_tgt, the user can perform a touch operation that satisfies the storage touch operation condition Cin even when the user looks away from the display 110 of the operation terminal 100. Therefore, it is easy for the user to directly visually confirm the running of the vehicle 200 and the situation around the vehicle 200.

[0159] <Remote shipping control> On the other hand, when the user performs a touch operation of touching the leaving selection image portion P12 with his / her finger, the terminal ECU 190 transmits an leaving selection signal S20 to the outside. The leaving selection image portion P12 is the portion P12 of the display 110 corresponding to the leaving selection image G12. The leaving selection signal S20 is a signal indicating that the user has performed a touch operation of touching the leaving selection image portion P12 with his / her finger.

[0160] When the vehicle ECU 290 receives the leaving selection signal S20 transmitted by the operation terminal 100, it transmits a vehicle position information signal S21 to the outside. The vehicle position information signal S21 is a signal that indicates information such as the position of the vehicle 200 relative to the space in which the vehicle 200 is parked, based on image data transmitted from the camera 277 by operating the camera 277.

[0161] When the terminal ECU 190 receives the vehicle position information signal S21 transmitted by the vehicle ECU 290, it displays an exit direction selection image G40 on the display 110, as shown in Fig. 12. The exit direction selection image G40 is an image for allowing the user to select the direction in which to exit from the parking space Pnow in which the vehicle 200 is currently parked (hereinafter referred to as the "current parking space Pnow").

[0162] The exit direction selection image G40 includes a current parking space image G41, a vehicle image G42, a candidate exit direction image G43, an application termination image G45, and an initial screen image G46. The current parking space image G41 is an image that displays the current parking space Pnow. The vehicle image G42 is an image that displays the vehicle 200 itself. The candidate exit direction image G43 is an image that displays directions that can be selected as the direction for the vehicle 200 to exit the parking lot. The application termination image G45 is the same image as the application termination image G25. The initial screen image G46 is the same image as the initial screen image G26.

[0163] The current parking space image G41 and the vehicle image G42 are displayed in an area approximately in the center of the display 110. A candidate exit direction image G43 corresponding to the traveling direction when the vehicle 200 moves forward is displayed above and near the current parking space image G41. On the other hand, a candidate exit direction image G43 corresponding to the traveling direction when the vehicle 200 moves backward is displayed below and near the current parking space image G41.

[0164] The application termination image G45 is displayed in the lower right corner region of the display 110. The initial screen image G46 is displayed in the lower left corner region of the display 110.

[0165] When a user performs a touch operation by touching any of the candidate leaving direction image portions P43 with a finger, the terminal ECU 190 transmits an leaving direction signal S22 to the outside. The candidate leaving direction image portion P43 is a portion of the display 110 that corresponds to the candidate leaving direction image G43. The leaving direction signal S22 is a signal that indicates the candidate leaving direction image portion P43 that the user touched with a finger.

[0166] When the vehicle ECU 290 receives the leaving direction signal S22 transmitted by the terminal ECU 190, the vehicle ECU 290 starts remote leaving control and calculates a leaving route. The leaving route is a route along which the vehicle 200 travels in the leaving direction indicated by the leaving direction signal S22 to a predetermined location Pout_tgt (i.e., a target leaving point Pout_tgt that is a target location to which the vehicle 200 is moved and stopped).

[0167] When the vehicle ECU 290 completes the calculation of the leaving route, it transmits a signal S23 indicating the distance between the target leaving point Pout_tgt and the current position of the vehicle 200 (hereinafter referred to as "remaining leaving distance signal S23") to the outside.

[0168] The vehicle ECU 290 obtains the distance between the target departure point Pout_tgt and the current position of the vehicle 200 by calculation using the image data transmitted from the camera 277 .

[0169] On the other hand, when the user performs a touch operation of touching any of the candidate leaving direction image portions P43 with a finger, the terminal ECU 190 displays an leaving touch operation image G50 on the display 110 as shown in Fig. 13. The leaving touch operation image G50 is an image for prompting the user to perform a touch operation to leave the vehicle 200 in the leaving direction selected by the user through a touch operation. Fig. 13 shows an example in which the leaving direction selected by the user is the direction in which the vehicle 200 is to move forward.

[0170] The exit touch operation image G50 includes a current parking space image G51, a vehicle image G52, an exit direction image G57, a remaining exit distance image G53, a touch operation area image G54, an application termination image G55, and an initial screen image G56.

[0171] The current parking space image G51 is the same as the current parking space image G41 described above. The vehicle image G52 is the same as the vehicle image G42 described above. The exit direction image G57 is an image that represents the exit direction selected by the user through a touch operation. The remaining exit distance image G53 is an image that represents the distance between the current position of the vehicle 200 and the target exit point Pout_tgt. The touch operation area image G54 is the same as the touch operation area image G34 described above. The application termination image G55 is the same as the application termination image G45 described above. The initial screen image G56 is the same as the initial screen image G46 described above.

[0172] The current parking space image G51 and the vehicle image G52 are displayed in a substantially central area of ​​the upper half area 110A of the display 110. The exit direction image G57 is displayed in an area of ​​the display 110 immediately above the current parking space image G51, which is the upper half area 110A of the display 110. The remaining exit distance image G53 is displayed in the uppermost area of ​​the upper half area 110A of the display 110. Therefore, in this example, the upper half area 110A of the display 110 is used as an area (i.e., an information display area) for displaying information for providing the user with information regarding the exit of the vehicle 200 to the target exit point Pout_tgt.

[0173] The terminal ECU 190 receives the remaining distance to leave signal S23 transmitted by the vehicle ECU 290, and acquires the distance indicated by the remaining distance to leave signal S23. The terminal ECU 190 displays the acquired distance on the remaining distance to leave image G53.

[0174] The touch operation area image G54 is displayed in the lower half area 110B of the display 110. The area (i.e., the predetermined area) of the touch operation area image G54 displayed when the vehicle 200 is taken out by the remote leaving control is the same as the area of ​​the touch operation area image G34 displayed when the vehicle 200 is taken in by the remote entering control.

[0175] However, the area of ​​the touch operation area image G54 displayed when the vehicle 200 is removed by remote removal control may be set to an area different from the area of ​​the touch operation area image G34 displayed when the vehicle 200 is brought into the warehouse by remote entrance control.

[0176] The application termination image G55 is displayed in the lower right corner of the display 110. The initial screen image G56 is displayed in the lower left corner of the display 110.

[0177] 14, while the user keeps touching the touch operation portion P54 with his / her finger and moves the finger on the touch operation portion P54 so as to satisfy a predetermined touch operation condition Cout (hereinafter referred to as "leaving the garage touch operation condition Cout"), the terminal ECU 190 transmits a control execution command signal S16 to the outside. The touch operation portion P54 is a portion of the display 110 corresponding to the touch operation area image G54. The control execution command signal S16 is a signal indicating that a touch operation that satisfies the leaving the garage touch operation condition Cout has been performed.

[0178] In this example, the exit touch operation condition Cout is a condition C1out that when the user's finger moves on the touch operation part P54 while touching the touch operation part P54, the movement trajectory of the finger on the touch operation part P54 for a predetermined time T1out includes a predetermined number of bends N1out or more.

[0179] In this example, when the movement vector of the finger on the touch operation part P54 changes by a predetermined angle Ath or more, it is determined that one bending has occurred. For example, when the user moves the finger on the touch operation part P54 in a certain direction and then moves the finger in the opposite direction (i.e., when the user reverses the direction of moving the finger), the angle by which the movement vector of the finger on the touch operation part P54 has changed is approximately 180 degrees. Therefore, if the predetermined angle Ath is set to, for example, 150 degrees, when the user reverses the direction of moving the finger as described above, the movement vector of the finger changes by the predetermined angle Ath or more, and it is determined that one bending has occurred.

[0180] Thus, in this example, the exit touch operation condition Cout does not include the user's finger moving in a specific, prescribed path on the touch operation part P54 while touching the touch operation part P54. The specific, prescribed path is, for example, the path L1 to L4 shown in (A) to (D) of FIG. 9. Therefore, in this example, the exit touch operation condition Cout is satisfied even if the path of the user's finger moving on the touch operation part P54 while touching the touch operation part P54 is "a random path including a straight line in the up-down direction, a straight line in the left-right direction, a straight line in the diagonal direction, a curved line, and a combination of at least two of these lines."

[0181] The predetermined time T1out and the predetermined number of times N1out may be set to a time and a number of times appropriate for determining that the user intends to cause the vehicle ECU 290 to perform the remote leaving control described below. For example, the predetermined number of times N1out may be set to 1 time, and the time that the user is expected to move his / her finger on the touch operation portion P54 along a path including at least one bend when the user intends to cause the vehicle ECU 290 to perform the remote leaving control may be set as the predetermined time T1out.

[0182] The predetermined time T1out and the predetermined number of times N1out may be the same as the predetermined time T1in and the predetermined number of times N1in in the above-mentioned condition C1in, respectively, or may be different from each other.

[0183] Furthermore, the leaving touch operation condition Cout may be a condition C2out that when the user's finger moves on the touch operation part P54 while touching the touch operation part P54, the moving distance Dtouch of the finger on the touch operation part P54 in the predetermined time T2out is equal to or greater than the predetermined distance D2out. The predetermined time T2out and the predetermined distance D2out may be set to a time and distance suitable for determining that the user intends to have the vehicle ECU 290 perform the remote leaving control described later. For example, first, a distance suitable for determining that the user intends to have the vehicle ECU 290 perform the remote leaving control may be set as the predetermined distance D2out, and a time required for the user to move the finger on the touch operation part P54 by the predetermined distance D2out when the user intends to have the vehicle ECU 290 perform the remote leaving control may be set as the predetermined time T2out.

[0184] The predetermined time T2out and the predetermined distance D2out may be the same as the predetermined time T2in and the predetermined distance D2in in the condition C2in described above, or may be different from each other.

[0185] The leaving touch operation condition Cout may be a condition C3out that the time T during which the finger moving speed Vtouch on the touch operation part P54 continues to be equal to or greater than a predetermined speed V3out when the user's finger moves on the touch operation part P54 while touching the touch operation part P54 is equal to or greater than a predetermined time T3out. The predetermined time T3out and the predetermined speed V3out may be set to a time and a speed suitable for determining that the user intends to have the vehicle ECU 290 perform the remote leaving control described later. For example, first, a time suitable for determining that the user intends to have the vehicle ECU 290 perform the remote leaving control may be set as the predetermined time T3out, and a speed predicted as the speed at which the user will move his / her finger on the touch operation part P54 when the user intends to have the vehicle ECU 290 perform the remote leaving control may be set as the predetermined speed V3out.

[0186] The predetermined time T3out and the predetermined speed V3out may be the same as the predetermined time T3in and the predetermined speed V3in in the above-mentioned condition C3in, respectively, or may be different from each other.

[0187] In addition, any two or all of the above-mentioned conditions C1out, C2out, and C3out may be set as the exit touch operation conditions Cout.

[0188] After completing the calculation of the departure route, while receiving the control execution command signal S16, the vehicle ECU 290 controls the operation of the fuel injection valve actuator 212, the friction brake actuator 232 and the motor driver 242 so that the vehicle 200 travels along the departure route and moves to the target departure point Pout_tgt.

[0189] Furthermore, the vehicle ECU 290 continues to transmit the vehicle position information signal S21 and the remaining distance to leave signal S23 to the outside during the execution of the remote leaving control.

[0190] The terminal ECU 190 receives the vehicle position information signal S21 transmitted by the vehicle ECU 290, and changes the position of the vehicle image G52 displayed on the display 110 so that the position of the vehicle image G52 corresponds to the position of the vehicle 200 relative to the current parking space Pnow represented by the vehicle position information signal S21, as shown in FIG. 13.

[0191] Furthermore, the terminal ECU 190 receives the remaining distance to leave signal S23 transmitted by the vehicle ECU 290, and displays the distance indicated by the remaining distance to leave signal S23 on the remaining distance to leave image G53.

[0192] Furthermore, when the vehicle ECU 290 detects a target object within a predetermined distance Dth based on a signal from any of the clearance sonars 278 while executing remote exit control, it transmits to the outside a sonar signal S18 representing the clearance sonar 278 that transmitted the signal representing that target object.

[0193] When the terminal ECU 190 receives the sonar signal S18, it displays the sonar image G58 in a portion of the display 110 near the vehicle image G52, as shown in Fig. 14. The sonar image G58 is an image that represents a portion of the vehicle image G52 that corresponds to the clearance sonar 278 represented by the sonar signal S18.

[0194] The display form of the sonar image G58 when the vehicle 200 is leaving the warehouse by the remote leaving control is the same as the display form of the sonar image G38 when the vehicle 200 is entering the warehouse by the remote entering control.

[0195] However, the display form of the sonar image G58 when the vehicle 200 is being exited by the remote exit control may be different from the display form of the sonar image G38 when the vehicle 200 is being entered by the remote entry control.

[0196] In addition, the operation terminal 100 may be configured to emit an alarm sound in conjunction with the display of the sonar image G58 when the vehicle 200 is being removed by remote removal control, in the same way as when the vehicle 200 is being brought into the warehouse by remote entrance control.

[0197] Of course, when the vehicle 200 is being removed by remote removal control, the operation terminal 100 may be configured to emit an alarm sound in conjunction with the display of the sonar image G58 in a manner different from that when the vehicle 200 is being brought into the warehouse by remote entrance control.

[0198] When the leaving touch operation condition Cout is no longer satisfied, for example because the user removes his / her finger from the touch operation portion P54, the terminal ECU 190 stops transmitting the control execution command signal S16 to the outside.

[0199] When the vehicle ECU 290 stops receiving the control execution command signal S16, the vehicle ECU 290 causes the friction brake mechanism 231 to stop the vehicle 200.

[0200] When the vehicle 200 has completed leaving the depot by arriving at the target leaving point Pout_tgt, the vehicle ECU 290 locks each wheel of the vehicle 200 using the parking brake 233, unlocks the doors of the vehicle 200 using the door lock mechanism 250, and stops the operation of vehicle driving equipment such as the camera 277, the internal combustion engine 210, the first MG 221, and the second MG 222 which are necessary for the vehicle 200 to travel, thereby ending the remote leaving control.

[0201] Furthermore, when leaving of vehicle 200 is completed, vehicle ECU 290 transmits to the outside a signal S24 indicating that leaving of vehicle 200 is completed (hereinafter referred to as "leaving completion signal S24").

[0202] When the terminal ECU 190 receives the leaving completion signal S24 transmitted by the vehicle ECU 290, it displays a leaving completion image G59 on the display 110 as shown in Fig. 15. The leaving completion image G59 is an image indicating that leaving of the vehicle 200 has been completed. The leaving completion image G59 is displayed in the topmost area of ​​the display 110.

[0203] In this traveling system 10, the exit touch operation condition Cout is a condition C1out that when a user's finger moves on the touch operation part P54 while touching the touch operation part P54, the movement trajectory of the finger on the touch operation part P54 for a predetermined time T1out includes a predetermined number of bends N1out or more.

[0204] Therefore, if the user touches the touch operation part P54 by mistake, the possibility that the trajectory of the finger touching the touch operation part P54 satisfies the above-mentioned leaving touch operation condition Cout is extremely small. Therefore, the possibility that the remote leaving control is performed when the user touches the touch operation part P54 by mistake is extremely small. Therefore, according to the present traveling system 10, the remote leaving control can be performed only when it is certain that the user has the intention to leave the vehicle 200.

[0205] Furthermore, if there is a problem with the touch sensing function of the display 110 and the terminal ECU 190 recognizes that the user's finger is touching the touch operation part P54 even though the user's finger is not touching the touch operation part P54, the possibility that the contact trajectory recognized by the terminal ECU 190 satisfies the above-mentioned leaving touch operation condition Cout is extremely small. Therefore, if the terminal ECU 190 recognizes that the user's finger is touching the touch operation part P54 even though the user's finger is not touching the touch operation part P54, the possibility that the remote leaving control is performed is extremely small. From this point of view, according to the present traveling system 10, it is possible to perform the remote leaving control only when it is certain that the user has the intention to leave the vehicle 200.

[0206] Furthermore, as described above, the leaving touch operation condition Cout does not include the user's finger moving in a specific, defined path (see FIG. 9) on the touch operation part P54 while touching the touch operation part P54. Therefore, the user can leave the vehicle 200 by freely moving the finger on the touch operation part P54 in a random path. In other words, there is no prescribed way for the user to move the finger when touching the touch operation part P54 to leave the vehicle 200. Therefore, the user can cause the vehicle ECU 290 to perform remote entry control by a simple touch operation. Furthermore, even if the user must touch the touch operation part P54 with the finger of the hand holding the operation terminal 100, the user can easily perform a touch operation that satisfies the leaving touch operation condition Cout.

[0207] In addition, there is no set way for the user to move his / her finger when touching the touch operation portion P54 to cause the vehicle 200 to leave the garage, so the user can perform a touch operation that satisfies the leaving touch operation condition Cout even if he / she takes his / her eyes off the display 110 of the operation terminal 100. Therefore, the user can easily directly check the traveling of the vehicle 200 and the situation around the vehicle 200 with his / her eyes.

[0208] <Exiting the app, etc.> When the user performs a touch operation by touching with his / her finger portions P15, P25, P35, P45, and P55 of the display 110 corresponding to the application termination images G15, G25, G35, G45, and G55 (hereinafter referred to as "application termination image portions P15, etc."), the terminal ECU 190 terminates the remote driving application.

[0209] On the other hand, when the user performs a touch operation by touching with his / her finger portions P26, P36, P46, and P56 of the display 110 corresponding to the initial screen images G26, G36, G46, and G56 (hereinafter referred to as "initial screen image portions P26, etc."), the terminal ECU 190 displays the entry / exit selection image G10 on the display 110.

[0210] Furthermore, when the user performs a touch operation of touching the application termination image portion P15 with a finger, the terminal ECU 190 transmits a signal S25 for terminating the remote driving control (hereinafter referred to as a "control termination command signal S25") to the outside. Furthermore, when the user performs a touch operation of touching the initial screen image portion P26 with a finger, the terminal ECU 190 also transmits the control termination command signal S25 to the outside.

[0211] When the vehicle ECU 290 receives the control end command signal S25 transmitted by the terminal ECU 190, it stops the vehicle 200 using the friction brake mechanism 231, then locks the wheels of the vehicle 200 using the parking brake 233, and stops the operation of vehicle driving equipment such as the camera 277, the internal combustion engine 210, the first MG 221 and the second MG 222 which are necessary for the driving of the vehicle 200, and terminates the remote entry control if it is executing the remote entry control, or terminates the remote exit control if it is executing the remote exit control.

[0212] In addition, the driving system 10 is configured so that the operation terminal 100 and the vehicle ECU 290 directly exchange various signals with each other via wireless communication.

[0213] However, the driving system 10 may be configured to include an operation terminal 100, a vehicle 200 (particularly, a vehicle ECU 290), and the Internet 300, as shown in FIG. 16, and to exchange various signals between the operation terminal 100 and the vehicle ECU 290 via the Internet 300.

[0214] Furthermore, as shown in FIG. 17, the driving system 10 may be configured to include an operation terminal 100, a vehicle 200 (particularly, a vehicle ECU 290), the Internet 300, and a server 301 arranged on the Internet 300, and to exchange various signals between the operation terminal 100 and the vehicle ECU 290 via the Internet 300 and the server 301.

[0215] Furthermore, in the case where the driving system 10 includes an operation terminal 100, a vehicle 200 (particularly, a vehicle ECU 290), the Internet 300, and a server 301 (see FIG. 17), the driving system 10 may be configured to have some of the functions performed by the remote driving app installed in the terminal ECU 190 performed by application software installed in the server 301.

[0216] Similarly, when the driving system 10 includes an operation terminal 100, a vehicle 200 (particularly, a vehicle ECU 290), the Internet 300, and a server 301 (see FIG. 17), the driving system 10 may be configured to have a program installed in the server 301 handle some of the processing performed by the remote driving control installed in the vehicle ECU 290.

[0217] Furthermore, in a case where the driving system 10 includes an operation terminal 100, a vehicle 200 (particularly, a vehicle ECU 290), the Internet 300, and a server 301 (see FIG. 17), the driving system 10 may be configured so that the remote driving app installed in the terminal ECU 190 is updated by the server 301.

[0218] Similarly, when the driving system 10 includes an operation terminal 100, a vehicle 200 (particularly, a vehicle ECU 290), the Internet 300, and a server 301 (see FIG. 17), the driving system 10 may be configured so that the remote driving control program installed in the vehicle ECU 290 is updated by the server 301.

[0219] <Specific operation of this driving system> Next, we will explain the specific operation of the present traveling system 10. The CPU 191 (hereinafter referred to as "terminal CPU 191") of the terminal ECU 190 of the operation terminal 100 of the present traveling system 10 executes a routine shown in the flowchart of FIG. 18 at predetermined time intervals.

[0220] 18, the terminal CPU 191 starts the process from step 1800 and proceeds to step 1805 to determine whether the value of the application startup flag X10 is "1." The value of the application startup flag X10 is set to "1" when the remote driving application is started, and is set to "0" when the remote driving application is ended.

[0221] If the value of the application startup flag X10 is "1", the terminal CPU 191 determines "Yes" in step 1805 and proceeds to step 1810 to determine whether the value of the first application termination operation flag X11 is "0". The value of the first application termination operation flag X11 is set to "1" when a touch operation is performed on the application termination image portion P15 or the like, and is set to "0" when the remote driving application is started.

[0222] If the value of the first application termination operation flag X11 is "0", the terminal CPU 191 judges "Yes" in step 1810 and proceeds to step 1815 to judge whether or not either of the values ​​of the first application launch operation flag X12 and the first initial screen operation flag X13 is "1". The value of the first application launch operation flag X12 is set to "1" when an operation for launching a remote driving application is performed, and is set to "0" when the entry / exit selection image G10 is displayed on the display 110. The value of the first initial screen operation flag X13 is set to "1" when a touch operation is performed on the initial screen image portion P26 or the like, and is set to "0" when the entry / exit selection image G10 is displayed on the display 110.

[0223] If either the value of the first application launch operation flag X12 or the value of the first initial screen operation flag X13 is “1”, the terminal CPU 191 determines “Yes” in step 1815 and performs the process of step 1820 described below.

[0224] Step 1820: The terminal CPU 191 displays the entry / exit selection image G10 on the display 110.

[0225] On the other hand, if both the values ​​of the first application launch operation flag X12 and the first initial screen operation flag X13 are “0”, the terminal CPU 191 determines “No” in step 1815 and proceeds directly to step 1825.

[0226] When the terminal CPU 191 proceeds to step 1825, it determines whether or not the value of the warehousing selection flag X14 is "1." The value of the warehousing selection flag X14 is set to "1" when a touch operation is performed on the warehousing selection image portion P11, is set to "0" when a touch operation is performed on the application termination image portion P15 or the like, and is set to "0" when a touch operation is performed on the initial screen image portion P26 or the like.

[0227] If the value of the storage selection flag X14 is “1”, the terminal CPU 191 determines “Yes” in step 1825 and performs the process of step 1830 described below.

[0228] Step 1830: The terminal CPU 191 displays the parking space selection image G20 on the display 110.

[0229] When the terminal CPU 191 proceeds to step 1835, it determines whether or not the value of the first parking space selection flag X15 is "1." The value of the first parking space selection flag X15 is set to "1" when a touch operation is performed on the parking space image portion P21, is set to "0" when a touch operation is performed on the application termination image portion P15 or the like, and is set to "0" when a touch operation is performed on the initial screen image portion P26 or the like.

[0230] If the value of the first parking space selection flag X15 is "1", the terminal CPU 191 determines "Yes" in step 1835 and performs the process of step 1840 described below. After that, the terminal CPU 191 proceeds to step 1895 and temporarily ends this routine.

[0231] Step 1840: The terminal CPU 191 displays the entrance touch operation image G30 on the display 110. Furthermore, the terminal CPU 191 changes the display position of the vehicle image G32 based on the vehicle position information signal S13 transmitted by the vehicle ECU 290, changes the display of the entrance remaining distance image G33 based on the entrance remaining distance signal S15 transmitted by the vehicle ECU 290, and changes the traveling direction image G37 based on the traveling direction signal S17 transmitted by the vehicle ECU 290. Furthermore, when the terminal CPU 191 receives a sonar signal S18 transmitted by the vehicle ECU 290, it displays a sonar image G38 corresponding to the clearance sonar 278 represented by the sonar signal S18 on the display 110.

[0232] On the other hand, if the value of the first parking space selection flag X15 is "0", the terminal CPU 191 determines "No" in step 1835, proceeds directly to step 1895, and temporarily ends this routine.

[0233] If the value of the entry selection flag X14 is "0" at the time when the terminal CPU 191 executes the process of step 1825, the terminal CPU 191 determines "No" in step 1825 and proceeds to step 1845 to determine whether the value of the exit selection flag X16 is "1." The value of the exit selection flag X16 is set to "1" when a touch operation is performed on the exit selection image portion P12, is set to "0" when a touch operation is performed on the application termination image portion P15 or the like, and is set to "0" when a touch operation is performed on the initial screen image portion P26 or the like.

[0234] If the value of the delivery selection flag X16 is “1”, the terminal CPU 191 determines “Yes” in step 1845 and performs the process of step 1850 described below.

[0235] Step 1850: The terminal CPU 191 displays the exit direction selection image G40 on the display 110.

[0236] When the terminal CPU 191 proceeds to step 1855, it determines whether or not the value of the first leaving direction selection flag X17 is "1." The value of the first leaving direction selection flag X17 is set to "1" when a touch operation is performed on the candidate leaving direction image portion P43, is set to "0" when a touch operation is performed on the application termination image portion P15 or the like, and is set to "0" when a touch operation is performed on the initial screen image portion P26 or the like.

[0237] If the value of the first outgoing direction selection flag X17 is "1", the terminal CPU 191 judges "Yes" in step 1855 and performs the process of step 1860 described below. After that, the terminal CPU 191 proceeds to step 1895 and temporarily ends this routine.

[0238] Step 1860: The terminal CPU 191 displays the exit touch operation image G50 on the display 110. Furthermore, the terminal CPU 191 changes the display position of the vehicle image G52 based on the vehicle position information signal S21 transmitted by the vehicle ECU 290, and changes the display of the exit remaining distance image G53 based on the exit remaining distance signal S23 transmitted by the vehicle ECU 290. Furthermore, when the terminal CPU 191 receives a sonar signal S18 transmitted by the vehicle ECU 290, it displays a sonar image G58 corresponding to the clearance sonar 278 represented by the sonar signal S18 on the display 110.

[0239] On the other hand, if the value of the first outgoing direction selection flag X17 is "0", the terminal CPU 191 determines "No" in step 1845 and proceeds directly to step 1895, where it temporarily ends this routine.

[0240] If the value of the delivery selection flag X16 is "0" when the terminal CPU 191 executes the process of step 1845, the terminal CPU 191 determines "No" in step 1845 and proceeds directly to step 1895, temporarily terminating this routine.

[0241] If the value of the first application termination operation flag X11 is "1" at the time when the terminal CPU 191 executes the process of step 1810, the terminal CPU 191 determines "No" in step 1810 and executes the process of step 1865 described below. After that, the terminal CPU 191 proceeds to step 1895 and temporarily ends this routine.

[0242] Step 1865: The terminal CPU 191 ends the remote driving application.

[0243] If the value of the application startup flag X10 is "0" when the terminal CPU 191 executes the process of step 1805, the terminal CPU 191 determines "No" in step 1805 and proceeds directly to step 1895, temporarily terminating this routine.

[0244] Furthermore, the terminal CPU 191 executes a routine shown in the flowchart of FIG. 19 at predetermined time intervals.

[0245] Therefore, at a predetermined timing, the terminal CPU 191 starts the process from step 1900 and proceeds to step 1910 to determine whether the value of the second application startup operation flag X18 is "1." The value of the second application startup operation flag X18 is set to "1" when an operation to start a remote driving application is performed, and is set to "0" when the application startup signal S10 is transmitted to the outside.

[0246] If the value of the second application launch operation flag X18 is “1”, the terminal CPU 191 determines “Yes” in step 1910 and performs the process of step 1920 described below.

[0247] Step 1920: The terminal CPU 191 transmits an application start signal S10 to the outside.

[0248] On the other hand, if the value of the second application launch operation flag X18 is “0”, the terminal CPU 191 determines “No” in step 1910 and proceeds directly to step 1930.

[0249] When the terminal CPU 191 proceeds to step 1930, it determines whether or not either of the values ​​of the second application termination operation flag X19 and the second initial screen operation flag X20 is "1." The value of the second application termination operation flag X19 is set to "1" when a touch operation is performed on the application termination image portion P15 or the like, and is set to "0" when a control termination command signal S25 is transmitted to the outside. The value of the second initial screen operation flag X20 is set to "1" when a touch operation is performed on the initial screen image portion P26 or the like, and is set to "0" when a control termination command signal S25 is transmitted to the outside.

[0250] If either the value of the second application termination operation flag X19 or the value of the second initial screen operation flag X20 is “1”, the terminal CPU 191 determines “Yes” in step 1930 and performs the process of step 1940 described below.

[0251] Step 1940: The terminal CPU 191 transmits a control end command signal S25 to the outside.

[0252] On the other hand, if both the values ​​of the second application termination operation flag X19 and the second initial screen operation flag X20 are “0”, the terminal CPU 191 determines “No” in step 1930 and proceeds directly to step 1950.

[0253] When the terminal CPU 191 proceeds to step 1950, it determines whether or not the value of the second parking space selection flag X21 is "1." The value of the second parking space selection flag X21 is set to "1" when a touch operation is performed on the parking space image portion P21, and is set to "0" when the parking space signal S14 is transmitted to the outside.

[0254] If the value of the second parking space selection flag X21 is “1”, the terminal CPU 191 determines “Yes” in step 1950 and performs the process of step 1960 described below.

[0255] Step 1960: The terminal CPU 191 transmits the parking space signal S14 to the outside.

[0256] On the other hand, if the value of the second parking space selection flag X21 is “0”, the terminal CPU 191 determines “No” in step 1950 and proceeds directly to step 1970.

[0257] When the terminal CPU 191 proceeds to step 1970, it determines whether the value of the second leaving direction selection flag X22 is "1." The value of the second leaving direction selection flag X22 is set to "1" when a touch operation is performed on the candidate leaving direction image portion P43, and is set to "0" when the leaving direction signal S22 is transmitted to the outside.

[0258] If the value of the second outgoing direction selection flag X22 is "1", the terminal CPU 191 judges "Yes" in step 1970 and performs the process of step 1980 described below. After that, the terminal CPU 191 proceeds to step 1995 and temporarily ends this routine.

[0259] Step 1980: The terminal CPU 191 transmits the outgoing direction signal S22 to the outside.

[0260] On the other hand, if the value of the second outgoing direction selection flag X22 is "0", the terminal CPU 191 determines "No" in step 1970 and proceeds directly to step 1995, where it temporarily ends this routine.

[0261] Furthermore, the terminal CPU 191 executes a routine shown in the flowchart of FIG. 20 at predetermined time intervals.

[0262] Therefore, at a predetermined timing, the terminal CPU 191 starts the process from step 2000 in FIG. 20, proceeds to step 2010, and determines whether the entry touch operation image G30 or the exit touch operation image G50 is being displayed.

[0263] When the entry touch operation image G30 or the exit touch operation image G50 is being displayed, the terminal CPU 191 determines "Yes" in step 2010 and proceeds to step 2020 to determine whether the value of the entry completion signal flag X23 is "0." The value of the entry completion signal flag X23 is set to "1" when the entry completion signal S19 transmitted to the outside by the vehicle ECU 290 is received, is set to "0" when a touch operation is performed on the application termination image portion P15 or the like, and is set to "0" when a touch operation is performed on the initial screen image portion P26 or the like.

[0264] If the value of the entry completion signal flag X23 is "0", the terminal CPU 191 determines "Yes" in step 2020 and proceeds to step 2030 to determine whether the value of the exit completion signal flag X24 is "0". The value of the exit completion signal flag X24 is set to "1" when the exit completion signal S24 transmitted by the vehicle ECU 290 to the outside is received, is set to "0" when a touch operation is performed on the application termination image portion P15 or the like, and is set to "0" when a touch operation is performed on the initial screen image portion P26 or the like.

[0265] If the value of the leaving completion signal flag X24 is “0”, the terminal CPU 191 judges “Yes” in step 2030 and proceeds to step 2040, where it determines whether a touch operation that satisfies the entering touch operation condition Cin or the leaving touch operation condition Cout (i.e., a specified touch operation) has been performed.

[0266] If a predetermined touch operation has been performed, the terminal CPU 191 determines "Yes" in step 2040, and performs the process of step 2050 described below. After that, the terminal CPU 191 proceeds to step 2095, and temporarily ends this routine.

[0267] Step 2050: The terminal CPU 191 transmits a control execution command signal S16 to the outside.

[0268] On the other hand, if the predetermined touch operation has not been performed, the terminal CPU 191 determines "No" in step 2040 and proceeds directly to step 2095 to temporarily end this routine. In this case, the control execution command signal S16 is not transmitted.

[0269] If the value of the delivery completion signal flag X24 is "1" when the terminal CPU 191 executes the process of step 2030, the terminal CPU 191 determines "No" in step 2030 and executes the process of step 2070 described below. After that, the terminal CPU 191 proceeds to step 2095 and temporarily ends this routine.

[0270] Step 2070: The terminal CPU 191 displays a leaving-store completion image G59 on the display 110.

[0271] If the value of the warehousing completion signal flag X23 is "1" when the terminal CPU 191 executes the process of step 2020, the terminal CPU 191 determines "No" in step 2020 and executes the process of step 2060 described below. After that, the terminal CPU 191 proceeds to step 2095 and temporarily ends this routine.

[0272] Step 2060: The terminal CPU 191 displays a storage completion image G39 on the display 110.

[0273] If the entry touch operation image G30 or the exit touch operation image G50 is not being displayed at the time the terminal CPU 191 executes the processing of step 2010, the terminal CPU 191 judges "No" in step 2010 and proceeds directly to step 2095, temporarily terminating this routine.

[0274] On the other hand, a CPU 291 of the vehicle ECU 290 (hereinafter referred to as the "vehicle CPU 291") executes a routine shown in the flowchart of FIG. 21 at predetermined time intervals.

[0275] 21, proceeds to step 2110, and determines whether the value of the control end command flag X25 is "0." The value of the control end command flag X25 is set to "1" when the vehicle CPU 291 receives a control end command signal S25 transmitted from the terminal CPU 191 to the outside, and is set to "0" when the vehicle CPU 291 receives an application start signal S10 transmitted from the terminal CPU 191 to the outside.

[0276] If the value of the control end command flag X25 is "0", the vehicle CPU 291 determines "Yes" in step 2110 and proceeds to step 2120 to determine whether the value of the entry completion flag X26 is "0". The value of the entry completion flag X26 is set to "1" when entry of the vehicle 200 is completed, and is set to "0" when the vehicle CPU 291 receives the control end command signal S25 transmitted by the terminal CPU 191 to the outside.

[0277] If the value of the entry completion flag X26 is "0", the vehicle CPU 291 determines "Yes" in step 2120 and proceeds to step 2130 to determine whether the value of the parking space signal flag X27 is "1". The value of the parking space signal flag X27 is set to "1" when the vehicle CPU 291 receives the parking space signal S14 transmitted by the terminal CPU 191 to the outside, and is set to "0" when the vehicle CPU 291 receives the control end command signal S25 transmitted by the terminal CPU 191 to the outside.

[0278] If the value of the parking space signal flag X27 is "1", the vehicle CPU 291 determines "Yes" in step 2130 and proceeds to step 2140 to determine whether the value of the entrance route calculation completion flag X28 is "0". The value of the entrance route calculation completion flag X28 is set to "1" when the calculation of the entrance route is completed, and is set to "0" when the vehicle CPU 291 receives the control end command signal S25 transmitted by the terminal CPU 191 to the outside.

[0279] If the value of the warehouse entry route calculation completion flag X28 is "0", the vehicle CPU 291 determines "Yes" in step 2140 and performs the process of step 2150 described below.

[0280] Step 2150: The vehicle CPU 291 calculates the entry route.

[0281] On the other hand, if the value of the entrance route calculation completion flag X28 is “1”, the vehicle CPU 291 determines “No” in step 2140 and proceeds directly to step 2160.

[0282] When the vehicle CPU 291 proceeds to step 2160, it determines whether the value of the control execution command signal flag X29 is "1." The value of the control execution command signal flag X29 is set to "1" when the vehicle CPU 291 receives the control execution command signal S16 transmitted from the terminal CPU 191 to the outside, and is set to "0" when the vehicle CPU 291 no longer receives the control execution command signal S16.

[0283] If the value of the control execution command signal flag X29 is "1", the vehicle CPU 291 determines "Yes" in step 2160 and performs the process of step 2170 described below. Thereafter, the vehicle CPU 291 proceeds to step 2195 and temporarily ends this routine.

[0284] Step 2170: The vehicle CPU 291 performs remote entry control. As described above, when the vehicle 200 arrives at the target parking space Pin_tgt, the vehicle CPU 291 stops the vehicle 200 by the friction brake mechanism 231, locks each wheel of the vehicle 200 by the parking brake 233, and locks the doors of the vehicle 200 by the door lock mechanism 250.

[0285] On the other hand, if the value of the control execution command signal flag X29 is "0", the vehicle CPU 291 determines "No" in step 2160 and performs the process of step 2180 described below. Thereafter, the vehicle CPU 291 proceeds to step 2195 and temporarily ends this routine.

[0286] Step 2180: The vehicle CPU 291 brakes the vehicle 200 using the friction brake mechanism 231 to stop it.

[0287] If the value of the parking space signal flag X27 is "0" at the time when the vehicle CPU 291 executes the process of step 2130, the vehicle CPU 291 determines "No" in step 2130 and proceeds directly to step 2195, and temporarily ends this routine.

[0288] If the value of the entry completion flag X26 is "1" when the vehicle CPU 291 executes the process of step 2120, the vehicle CPU 291 determines "No" in step 2120 and proceeds directly to step 2195, temporarily terminating this routine.

[0289] If the value of the control end command flag X25 is "1" when the vehicle CPU 291 executes the process of step 2110, the vehicle CPU 291 determines "No" in step 2110 and executes the process of step 2190 described below. Thereafter, the vehicle CPU 291 proceeds directly to step 2195 and temporarily ends this routine.

[0290] Step 2190: The vehicle CPU 291 brakes the vehicle 200 to a stop using the friction brake mechanism 231, and locks the wheels of the vehicle 200 using the parking brake 233.

[0291] Furthermore, the vehicle CPU 291 executes a routine shown in the flowchart of FIG. 22 at predetermined time intervals.

[0292] Therefore, at a predetermined timing, the vehicle CPU 291 starts the process from step 2200 in FIG. 22, proceeds to step 2210, and determines whether the value of the control end command flag X25 is "0" or not.

[0293] If the value of the control end command flag X25 is "0", the vehicle CPU 291 determines "Yes" in step 2210 and proceeds to step 2220 to determine whether the value of the leaving completion flag X30 is "0" or not.

[0294] The value of the leaving completion flag X30 is set to "1" when leaving of the vehicle 200 is completed, and is set to "0" when the vehicle CPU 291 receives the control end command signal S25 transmitted from the terminal CPU 191 to the outside.

[0295] If the value of the leaving completion flag X30 is "0", the vehicle CPU 291 determines "Yes" in step 2220 and proceeds to step 2230 to determine whether the value of the leaving direction signal flag X31 is "1". The value of the leaving direction signal flag X31 is set to "1" when the vehicle CPU 291 receives the leaving direction signal S22 transmitted by the terminal CPU 191 to the outside, and is set to "0" when the vehicle CPU 291 receives the control end command signal S25 transmitted by the terminal CPU 191 to the outside.

[0296] If the value of the leaving direction signal flag X31 is "1", the vehicle CPU 291 judges "Yes" in step 2230 and proceeds to step 2240 to judge whether the value of the leaving route calculation completion flag X32 is "0". The value of the leaving route calculation completion flag X32 is set to "1" when the calculation of the leaving route is completed, and is set to "0" when the vehicle CPU 291 receives the control end command signal S25 transmitted by the terminal CPU 191 to the outside.

[0297] If the value of the leaving route calculation completion flag X32 is "0", the vehicle CPU 291 determines "Yes" in step 2240 and performs the process of step 2250 described below.

[0298] Step 2250: The vehicle CPU 291 calculates a route for leaving the facility.

[0299] On the other hand, if the value of the leaving route calculation completion flag X32 is “1”, the vehicle CPU 291 determines “No” in step 2240 and proceeds directly to step 2260.

[0300] When the vehicle CPU 291 proceeds to step 2260, it determines whether the value of the control execution command signal flag X29 is "1" or not.

[0301] If the value of the control execution command signal flag X29 is "1", the vehicle CPU 291 determines "Yes" in step 2260, and performs the process of step 2270 described below. Thereafter, the vehicle CPU 291 proceeds to step 2295, and temporarily ends this routine.

[0302] Step 2270: The vehicle CPU 291 performs remote leaving control. As described above, when the vehicle 200 arrives at the target leaving point Pout_tgt, the vehicle CPU 291 stops the vehicle 200 by the friction brake mechanism 231, locks the wheels of the vehicle 200 by the parking brake 233, and unlocks the doors by the door lock mechanism 250.

[0303] On the other hand, if the value of the control execution command signal flag X29 is "0", the vehicle CPU 291 determines "No" in step 2260 and performs the process of step 2280 described below. Thereafter, the vehicle CPU 291 proceeds to step 2295 and temporarily ends this routine.

[0304] Step 2280: The vehicle CPU 291 brakes the vehicle 200 using the friction brake mechanism 231 to stop it.

[0305] If the value of the leaving direction signal flag X31 is "0" when the vehicle CPU 291 executes the process of step 2230, the vehicle CPU 291 determines "No" in step 2230 and proceeds directly to step 2295, and temporarily ends this routine.

[0306] If the value of the leaving completion flag X30 is "1" when the vehicle CPU 291 executes the process of step 2220, the vehicle CPU 291 determines "No" in step 2220 and proceeds directly to step 2295, temporarily terminating this routine.

[0307] If the value of the control end command flag X25 is "1" when the vehicle CPU 291 executes the process of step 2210, the vehicle CPU 291 determines "No" in step 2210 and executes the process of step 2290 described below. Thereafter, the vehicle CPU 291 proceeds directly to step 2295 and temporarily ends this routine.

[0308] Step 2290: The vehicle CPU 291 brakes the vehicle 200 to a stop using the friction brake mechanism 231, and locks the wheels of the vehicle 200 using the parking brake 233.

[0309] Furthermore, the vehicle CPU 291 executes a routine shown in the flowchart of FIG. 23 at predetermined time intervals.

[0310] Therefore, at a predetermined timing, the vehicle CPU 291 starts the process from step 2300 in FIG. 23, proceeds to step 2310, and determines whether or not remote driving control is being executed.

[0311] If remote driving control is being executed, the vehicle CPU 291 judges “Yes” in step 2310 and proceeds to step 2320 to determine whether or not a target has been detected by any of the clearance sonars 278 within a predetermined distance Dth from the vehicle 200.

[0312] If a target is detected by any of the clearance sonars 278 within a range of the predetermined distance Dth from the vehicle 200, the vehicle CPU 291 determines "Yes" in step 2320 and performs the process of step 2330 described below. Thereafter, the vehicle CPU 291 proceeds to step 2340.

[0313] Step 2330: The vehicle CPU 291 transmits to the outside a sonar signal S18 indicating that the clearance sonar 278 has detected a target within a range of the predetermined distance Dth from the vehicle 200.

[0314] On the other hand, if the clearance sonar 278 has not detected any target within the range of the predetermined distance Dth from the vehicle 200, the vehicle CPU 291 determines “No” in step 2320 and proceeds directly to step 2340.

[0315] When the vehicle CPU 291 proceeds to step 2340, it determines whether or not the entry of the vehicle 200 has been completed.

[0316] If the entry of the vehicle 200 has been completed, the vehicle CPU 291 determines “Yes” in step 2340, and performs the process of step 2350 described below.

[0317] Step 2350: The vehicle CPU 291 transmits an entrance completion signal S19 to the outside.

[0318] On the other hand, if entry of the vehicle 200 has not been completed, the vehicle CPU 291 determines “No” in step 2340 and proceeds directly to step 2360 .

[0319] When the vehicle CPU 291 proceeds to step 2360, it determines whether or not the vehicle 200 has completed leaving the garage.

[0320] If leaving of the vehicle 200 is complete, the vehicle CPU 291 determines "Yes" in step 2360, and performs the process of step 2370 described below. Thereafter, the vehicle CPU 291 proceeds to step 2395, and temporarily ends this routine.

[0321] Step 2370: The vehicle CPU 291 transmits a leaving-parking completion signal S24 to the outside.

[0322] On the other hand, if the vehicle 200 has not yet left the garage, the vehicle CPU 291 determines “No” in step 2360 and proceeds directly to step 2395 .

[0323] If remote driving control is not being executed when the vehicle CPU 291 executes the process of step 2310, the vehicle CPU 291 determines "No" in step 2310 and proceeds directly to step 2395, and temporarily ends this routine.

[0324] The above is a specific operation of the traveling system 10. The traveling system 10 executes the routines shown in Figures 18 to 23, so that the remote entry control is performed only when it is certain that the user intends to enter the vehicle 200 into the target parking space Pin_tgt, and the remote exit control is performed only when it is certain that the user intends to exit the vehicle 200.

[0325] Furthermore, the user can perform remote driving control by a simple touch operation. Even if the user must perform a touch operation on the touch operation part P34 or the touch operation part P54 with the fingers of the hand holding the operation terminal 100, the user can easily perform a touch operation that satisfies the entry touch operation condition Cin or the exit touch operation condition Cout. Furthermore, the user can easily directly check the driving of the vehicle 200 and the situation around the vehicle 200 with his or her eyes.

[0326] The present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the present invention. [Explanation of symbols]

[0327] 10... remote driving system, 100... operation terminal, 110... display, 190... terminal ECU, 200... vehicle, 290... vehicle ECU, G34... touch operation area image, G54... touch operation area image

Claims

1. A remote driving system including an operation terminal having a touch sensing portion that has a function of sensing touch by a human finger, and a vehicle, The vehicle is a control execution command for requesting execution of the remote driving control is received and the vehicle is moved by the remote driving control while the user's finger is freely moved on the touch sensing portion of the operation terminal along a random trajectory that satisfies a predetermined touch operation condition that reduces the possibility that the user will erroneously perform a remote driving control to move the vehicle to a target location; When the user's finger does not move freely along a random trajectory that satisfies the predetermined touch operation condition on the touch sensing portion of the operation terminal, the movement by the remote traveling control is not performed. It is configured as follows: Remote driving system.

2. The remote traveling system according to claim 1, the predetermined touch operation condition includes a condition that the random trajectory is a trajectory including at least one bend; Remote driving system.

3. 3. The remote traveling system according to claim 1, the predetermined touch operation condition includes a condition that the random trajectory is a trajectory including a trajectory of the finger moving a predetermined distance or more within a predetermined time. Remote driving system.

4. The remote traveling system according to any one of claims 1 to 3, the predetermined touch operation condition includes a condition that the random trajectory is a trajectory including a trajectory of the finger moving continuously at a speed equal to or greater than a predetermined speed for a predetermined period of time or more; Remote driving system.

5. The remote traveling system according to any one of claims 1 to 4, The operation terminal includes: It has a display that displays images, An image partitioning a predetermined area as the touch sensing portion is displayed in a lower region of the display, and an image showing information regarding the traveling of the vehicle by the remote traveling control is displayed as an information image in an upper region of the display. Remote driving system.

6. The remote traveling system according to claim 5, the operation terminal is configured to display, as a part of the information image, a sensor image which is an image representing a sensor arranged in the vehicle for detecting a target which is an obstacle to the traveling of the vehicle, in a predetermined display form; The predetermined display form is at least one of a form in which the sensor image is displayed in a color different from a color of an image other than the sensor image and a form in which the sensor image is displayed by blinking. Remote driving system.

7. 7. The remote traveling system according to claim 5, The operation terminal is configured to display an image representing a distance from the vehicle to the target location as a part of the information image. Remote driving system.

8. The remote traveling system according to any one of claims 5 to 7, the operation terminal is configured not to transmit the control execution command when a trajectory of the movement of the finger on the touch sensitive portion is not the random trajectory; The vehicle is configured to stop when the control execution command is not transmitted from the operation terminal. Remote driving system.

9. The remote traveling system according to any one of claims 1 to 8, The remote driving control is at least one of the following controls: a control for setting a parking space as the target location and moving the vehicle to the parking space; and a control for setting a location outside the parking space as the target location and moving the vehicle parked in the parking space to the target location. Remote driving system.

10. The remote traveling system according to any one of claims 1 to 9, The operation terminal and the vehicle are configured to communicate with each other via wireless communication. Remote driving system.

11. The remote traveling system according to any one of claims 1 to 10, The operation terminal is configured to be able to communicate with a server installed on the Internet via wireless communication. Remote driving system.

12. The remote traveling system according to any one of claims 1 to 11, The vehicle is configured to be able to communicate with a server located on the Internet via wireless communication. Remote driving system.

Images