Information terminal, control method, and computer-readable recording medium

Abstract

Provided are an information terminal, a control method, and a computer-readable recording medium that can improve the operability during movement control. A smartphone (60) that can be carried by a user (M) of a vehicle (10) includes: a smartphone screen (61) capable of performing a position indication operation on a display image; and a processor (81) that performs movement control of the vehicle (10) based on a specific operation of the user (M) on the smartphone screen (61). When waiting for acceptance of a specific operation, the processor (81) causes the smartphone screen (61) to display a guidance image prompting the specific operation, stores position information of the specific operation performed by the user (M) based on the guidance image, and when waiting for acceptance of the specific operation and when the position information is stored, displays a guidance image with an operation range image (181) prompting the specific operation superimposed thereon based on the position information in the smartphone screen (61).

Description

Technical Field

[0001] The present invention relates to an information terminal, a control method, and a computer-readable recording medium. Background Art

[0002] In recent years, efforts to provide the use of sustainable transportation systems that consider vulnerable road users have become active. To achieve the above object, research and development have been dedicated to further improving traffic safety and convenience through research and development related to autonomous driving technology.

[0003] Conventionally, a remote parking system is known in which a smartphone is used to remotely operate a vehicle to park the vehicle in a designated regulated parking space or to drive the vehicle out of the parking space. Patent Document 1 describes a portable device that displays guidance information such as messages and patterns for remote operation on a screen when a user remotely operates a vehicle to park in a parking space or the like using the portable device. Patent Document 2 describes an operation system of a portable device in which when a user operates on the screen of the portable device, guidance information such as an arrow and a circle is displayed on the screen, which prompts an operation on a region opposite to the region where the communication antenna of the portable device is disposed.

[0004] Prior Art Documents

[0005] Patent Documents

[0006] Patent Document 1: International Publication No. 2019 / 163165 Pamphlet

[0007] Patent Document 2: Japanese Unexamined Patent Application Publication No. 2019-514088 Summary of the Invention

[0008] Problems to be Solved by the Invention

[0009] According to the portable device of Patent Document 1, by operating the portable device according to the guidance information, the vehicle can be parked in a parking space or the like. According to the operation system of Patent Document 2, by operating the region where the guidance information is displayed, the communication quality between the antenna of the portable device and the vehicle can be ensured. However, the operation methods of each user when operating the screen of the portable device are not necessarily the same, and each user has a certain degree of operation characteristics. Therefore, users sometimes feel burdened by the operation methods of the screens shown in Patent Document 1 and Patent Document 2. Therefore, in the field of autonomous driving technology, there is room for improvement in the ease of operation of the screen of the portable device.

[0010] An object of the present invention is to provide an information terminal, a control method, and a computer-readable recording medium that can improve the operability when performing movement control. Moreover, it further contributes to the development of sustainable transportation systems.

[0011] Means for Solving the Problem

[0012] The present invention is an information terminal that can be carried by a user of a moving body, and the information terminal includes:

[0013] A display unit capable of performing a position indication operation on a display image; and

[0014] A control unit that performs movement control of the moving body based on a specific operation of the user on the display unit,

[0015] The control unit performs the following processing:

[0016] When waiting for acceptance of the specific operation, cause the display unit to display a guidance image prompting the specific operation,

[0017] Store the position information of the specific operation performed by the user based on the guidance image,

[0018] When waiting for acceptance of the specific operation, if the position information is stored, display a guidance image prompting the specific operation based on the position information in the display unit.

[0019] The present invention is a control method, which is a control method of an information terminal. The information terminal can be carried by a user of a moving body and includes: a display unit capable of performing a position indication operation on a display image; and a control unit that performs movement control of the moving body based on a specific operation of the user on the display unit. Among them,

[0020] The control unit performs the following processing:

[0021] When waiting for acceptance of the specific operation, cause the display unit to display a guidance image prompting the specific operation,

[0022] Store the position information of the specific operation performed by the user based on the guidance image,

[0023] When waiting for acceptance of the specific operation, if the position information is stored, display a guidance image prompting the specific operation based on the position information in the display unit.

[0024] The present invention is a computer-readable recording medium that records a control program for causing a control unit of an information terminal to execute the following processing. The information terminal can be carried by a user of a moving body and includes: a display unit capable of performing a position indication operation on a display image; and the control unit that performs movement control of the moving body based on a specific operation of the user on the display unit. Among them, the processing includes:

[0025] While waiting for the acceptance of the specific operation, cause the display unit to display a guidance image prompting the specific operation.

[0026] Store the position information of the specific operation performed by the user based on the guidance image.

[0027] While waiting for the acceptance of the specific operation, when the position information is stored, display a guidance image prompting the specific operation based on the position information in the display unit.

[0028] Advantages of the Invention

[0029] According to the present invention, an information terminal, a control method, and a computer-readable recording medium capable of improving the operability during movement control can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a side view showing an example of a vehicle whose movement is controlled by the information terminal of the present embodiment.

[0031] Figure 2 is Figure 1 a top view of the vehicle shown.

[0032] Figure 3 is showing Figure 1 a block diagram of the internal structure of the vehicle shown.

[0033] Figure 4 is a diagram showing a situation where a vehicle is controlled for out-of-warehouse indication using a smartphone outside the vehicle.

[0034] Figure 5 is a diagram showing an example of the hardware structure of a smartphone.

[0035] Figure 6 is a flowchart showing the out-of-warehouse indication control of the vehicle performed by the processor.

[0036] Figure 7 is a flowchart showing the out-of-warehouse indication control of the vehicle performed by the processor.

[0037] Figure 8 is a diagram showing an example of an image displayed on a smartphone for controlling the out-of-warehouse indication of the vehicle.

[0038] Figure 9 is a diagram showing an example of an image displayed on a smartphone for controlling the out-of-warehouse indication of the vehicle.

[0039] Figure 10 is a diagram showing an example of an image displayed on a smartphone for controlling the out-of-warehouse indication of the vehicle.

[0040] Figure 11 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0041] Figure 12 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0042] Figure 13 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0043] Figure 14 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0044] Figure 15 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0045] Figure 16 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0046] Figure 17 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0047] Figure 18 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0048] Figure 19 This is a diagram showing an example of an image displayed on a smartphone for controlling the vehicle's outbound instruction.

[0049] Figure 20 This is a diagram showing an example of an image displayed on a smartphone in the first modified example of the movement instruction control.

[0050] Figure 21 This is a diagram showing an example of an image displayed on a smartphone in the second modified example of the movement instruction control.

[0051] Figure 22 This is a diagram showing an example of an image displayed on a smartphone in the third modified example of the movement instruction control.

[0052] Figure 23 This is a diagram showing an example of an image displayed on a smartphone in the fourth modified example of the movement instruction control.

[0053] Figure 24This is a diagram showing a situation where a vehicle is controlled for parking guidance using a smartphone from outside the vehicle.

[0054] Figure 25 This is a diagram showing an example of an image displayed on a smartphone that controls parking guidance for a vehicle.

[0055] Explanation of reference numerals

[0056] 10 Vehicle (mobile body)

[0057] 60 Smartphone (information terminal)

[0058] 61 Smartphone screen (display unit)

[0059] 81 Processor (control unit) Detailed implementation mode

[0060] Hereinafter, an embodiment of the information terminal, control method, and computer-readable recording medium of the present invention will be described based on the drawings. In addition, the drawings should be observed in the direction of the symbols. Furthermore, in this specification and the like, in order to make the description simple and clear, the front, rear, left, and right, and up and down directions are described according to the Figure 1 and Figure 2 direction observed by the driver of the vehicle 10 shown. In the drawings, the front of the vehicle 10 is represented as Fr, the rear as Rr, the left as L, the right as R, the upper as U, and the lower as D.

[0061] <Vehicle 10 controlled for movement by the information terminal of the present invention>

[0062] Figure 1 This is a side view of the vehicle 10 controlled for movement by the information terminal of the present invention. Figure 2 This is Figure 1 a top view of the vehicle 10 shown. The vehicle 10 is an example of the mobile body of the present invention.

[0063] The vehicle 10 is an automobile having a drive source (not shown) and wheels, and the wheels include drive wheels driven by the power of the drive source and steerable wheels. In the present embodiment, the vehicle 10 is a four-wheel automobile having a pair of left and right front wheels and rear wheels. The drive source of the vehicle 10 is, for example, an electric motor. In addition, the drive source of the vehicle 10 may be an internal combustion engine such as a gasoline engine or a diesel engine, or a combination of an electric motor and an internal combustion engine. Furthermore, the drive source of the vehicle 10 may drive a pair of left and right front wheels, a pair of left and right rear wheels, or all four of the pair of left and right front wheels and rear wheels. The front wheels and rear wheels may both be steerable wheels, or either one may be a steerable wheel.

[0064] The vehicle 10 is also equipped with side rearview mirrors 11L and 11R. The side rearview mirrors 11L and 11R are mirrors (rearview mirrors) provided on the outer sides of the front seat doors of the vehicle 10 for the driver to confirm the rear and the rear sides. The side rearview mirrors 11L and 11R are respectively fixed to the main body of the vehicle 10 by a rotation axis extending in the vertical direction and can be opened and closed by rotating around this rotation axis.

[0065] The vehicle 10 is also equipped with a front camera 12Fr, a rear camera 12Rr, a left-side camera 12L, and a right-side camera 12R. The front camera 12Fr is a digital camera provided in front of the vehicle 10 and used to photograph the front of the vehicle 10. The rear camera 12Rr is a digital camera provided behind the vehicle 10 and used to photograph the rear of the vehicle 10. The left-side camera 12L is a digital camera provided in the left-side rearview mirror 11L of the vehicle 10 and used to photograph the left side of the vehicle 10. The right-side camera 12R is a digital camera provided in the right-side rearview mirror 11R of the vehicle 10 and used to photograph the right side of the vehicle 10.

[0066] <Internal Structure of Vehicle 10>

[0067] Figure 3 It represents Figure 1 a block diagram showing an example of the internal structure of the vehicle 10 shown. As Figure 3 shown, the vehicle 10 has a sensor group 16, a navigation device 18, a control ECU (Electronic Control Unit) 20, an EPS (Electric Power Steering) system 22, and a communication unit 24. The vehicle 10 also has a driving force control system 26 and a braking force control system 28.

[0068] The sensor group 16 obtains various detection values used in the control performed by the control ECU 20. The sensor group 16 includes the front camera 12Fr, the rear camera 12Rr, the left-side camera 12L, and the right-side camera 12R. In addition, the sensor group 16 includes a front sonar group 32a, a rear sonar group 32b, a left-side sonar group 32c, and a right-side sonar group 32d. In addition, the sensor group 16 includes wheel sensors 34a, 34b, a vehicle speed sensor 36, and an operation detection unit 38.

[0069] The front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R acquire identification data (e.g., surrounding images) for identifying the outside world of the vehicle 10 by photographing the surroundings of the vehicle 10. The surrounding images captured by the front camera 12Fr, rear camera 12Rr, left-side camera 12L, and right-side camera 12R are respectively referred to as the front image, rear image, left-side image, and right-side image. The image composed of the left-side image and the right-side image can also be referred to as the side image.

[0070] The front sonar group 32a, rear sonar group 32b, left-side sonar group 32c, and right-side sonar group 32d emit sound waves to the surroundings of the vehicle 10 and receive the reflected sound from other objects. The front sonar group 32a includes, for example, four sonars. The sonars constituting the front sonar group 32a are respectively arranged at the left front obliquely, front left, front right, and right front obliquely of the vehicle 10. The rear sonar group 32b includes, for example, four sonars. The sonars constituting the rear sonar group 32b are respectively arranged at the left rear obliquely, rear left, rear right, and right rear obliquely of the vehicle 10. The left-side sonar group 32c includes, for example, two sonars. The sonars constituting the left-side sonar group 32c are respectively arranged at the front of the left side portion and the rear of the left side portion of the vehicle 10. The right-side sonar group 32d includes, for example, two sonars. The sonars constituting the right-side sonar group 32d are respectively arranged at the front of the right side portion and the rear of the right side portion of the vehicle 10.

[0071] The wheel sensors 34a, 34b detect the rotation angles of the wheels of the vehicle 10. The wheel sensors 34a, 34b can be composed of angle sensors or displacement sensors. The wheel sensors 34a, 34b output detection pulses every time the wheels rotate a predetermined angle. The detection pulses output from the wheel sensors 34a, 34b are used for calculating the rotation angle of the wheels and the rotation speed of the wheels. The moving distance of the vehicle 10 is calculated based on the rotation angle of the wheels. The wheel sensor 34a, for example, detects the rotation angle θrl of the left rear wheel. The wheel sensor 34b, for example, detects the rotation angle θrr of the right rear wheel.

[0072] The vehicle speed sensor 36 detects the speed of the vehicle body of the vehicle 10, that is, the vehicle speed V, and outputs the detected vehicle speed V to the control ECU 20. The vehicle speed sensor 36 detects the vehicle speed V based on the rotation of the countershaft of the transmission, for example.

[0073] The operation detection unit 38 detects the operation content of the user using the operation input unit 14 and outputs the detected operation content to the control ECU 20. The operation input unit 14 includes, for example, various user interfaces such as side mirror switches for switching the opening and closing states of the side mirrors 11L, 11R, and a shift lever (gearshift lever, selector).

[0074] The navigation device 18 detects the current position of the vehicle 10 using, for example, GPS (Global Positioning System), and guides the route to the destination to the user. The navigation device 18 has a storage device (not shown) having a map information database.

[0075] The navigation device 18 includes a touch panel 42 and a speaker 44. The touch panel 42 functions as an input device and a display device for controlling the ECU 20. The speaker 44 outputs various guidance information to the user of the vehicle 10 by sound.

[0076] The touch panel 42 is configured to be able to input various instructions to the control ECU 20. For example, the user can input instructions related to the movement support of the vehicle 10 via the touch panel 42. The movement support includes parking support and departure support of the vehicle 10. In addition, the touch panel 42 is configured to display various screens related to the control content of the control ECU 20. For example, a screen related to the movement support of the vehicle 10 is displayed on the touch panel 42. Specifically, a parking support button for requesting parking support of the vehicle 10 and a departure support button for requesting departure support are displayed on the touch panel 42. The parking support button includes an automatic parking button for requesting parking by automatic steering of the control ECU 20 and an assisted parking button for assisting when parking by the driver's operation. The departure support button includes an automatic departure button for requesting departure by automatic steering of the control ECU 20 and an assisted departure button for assisting when departing by the driver's operation. In addition, components other than the touch panel 42, such as a smart phone or a tablet terminal, can be used as an input device or a display device.

[0077] The control ECU 20 has an input / output unit 50, an arithmetic unit 52, and a storage unit 54. The arithmetic unit 52 is constituted by, for example, a CPU (Central Processing Unit). The arithmetic unit 52 controls each unit based on a program stored in the storage unit 54, thereby performing various controls. In addition, the arithmetic unit 52 inputs and outputs signals between each unit connected to the control ECU 20 via the input / output unit 50.

[0078] The operation unit 52 includes an automatic parking control unit 55 that controls the movement execution of the vehicle 10. The automatic parking control unit 55 supports automatic parking and automatic out-of-parking of the vehicle 10 through automatic steering, which means automatically operating the steering wheel 110 under the control of the automatic parking control unit 55. During automatic parking support and automatic out-of-parking support, the accelerator pedal (not shown), the brake pedal (not shown), and the operation input unit 14 are automatically operated. In addition, when the driver operates the accelerator pedal, the brake pedal, and the operation input unit 14 to manually park and out-park the vehicle 10, the automatic parking control unit 55 provides auxiliary parking support and auxiliary out-of-parking support.

[0079] For example, based on the recognition data of the outside of the vehicle 10 obtained by the front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R, and the specified parking space designated by the user, the automatic parking control unit 55 performs automatic parking control to park the vehicle 10 in the specified parking space and automatic out-of-parking control to out-park from the specified parking space.

[0080] In addition, the automatic parking control unit 55 registers the specified parking space designated by the user as the designated parking space in the storage unit 54. Based on the recognition data of the outside of the vehicle 10 obtained by the front camera 12Fr, the rear camera 12Rr, the left side camera 12L, and the right side camera 12R, the automatic parking control unit 55 registers the feature points related to the designated parking space in the storage unit 54. Based on the recognition data of the outside of the vehicle 10 and the feature points of the designated parking space designated by the user, the automatic parking control unit 55 controls automatic parking to park the vehicle 10 in the designated parking space and automatic out-of-parking from the designated parking space.

[0081] The EPS system 22 includes a steering angle sensor 100, a torque sensor 102, an EPS motor 104, a resolver 106, and an EPS ECU 108. The steering angle sensor 100 detects the steering angle θsteering of the steering wheel 110. The torque sensor 102 detects the torque TQ applied to the steering wheel 110.

[0082] The EPS motor 104 enables the occupant to operate the steering wheel 110 by applying a driving force or a reaction force to the steering column 112 connected to the steering wheel 110, and can perform automatic steering during parking support. The resolver 106 detects the rotation angle θrot of the EPS motor 104. The EPS ECU 108 is responsible for the overall control of the EPS system 22. The EPS ECU 108 includes an input / output unit (not shown), an operation unit (not shown), and a storage unit (not shown).

[0083] The communication unit 24 can communicate wirelessly with other communication devices 120. Other communication devices 120 are base stations, communication devices of other vehicles, smartphones or tablet terminals carried by the user of the vehicle 10, etc. Smartphones and tablet terminals are examples of information terminals of the present invention.

[0084] The driving force control system 26 includes a driving ECU 130. The driving force control system 26 performs driving force control of the vehicle 10. The driving ECU 130 controls the driving force of the vehicle 10 by controlling an engine (not shown) and the like based on a user's operation of an accelerator pedal (not shown).

[0085] The braking force control system 28 includes a brake ECU 132. The braking force control system 28 performs braking force control of the vehicle 10. The brake ECU 132 controls the braking force of the vehicle 10 by controlling a brake mechanism (not shown) and the like based on an operation of a brake pedal (not shown) by a user.

[0086] <Movement Instruction Control by Information Terminal>

[0087] Figure 4 1 is a diagram showing an example of a situation in which a user M of a vehicle 10 uses a smartphone 60 carried by the user M from outside the vehicle 10 to perform movement instruction control for automatically leaving the vehicle 10 from a parking space P. Movement instruction control is an example of movement control of the present invention. Movement instruction control includes, for example, parking instruction control for automatically parking the vehicle 10 in the parking space P and exit instruction control for automatically leaving the vehicle 10 from the parking space P. Figure 4 The illustrated example shows a case where the departure instruction control is performed on the vehicle 10 .

[0088] An application is installed in the smartphone 60, which can control the movement of the vehicle 10 by transmitting and receiving information related to the movement instruction control of the vehicle 10 between the smartphone 60 and the vehicle 10. The smartphone 60 is operated by a user M on a smartphone screen 61 (see FIG. 1 ) configured as a touch panel. Figure 8 When a touch operation is performed on the smart phone 60, an instruction signal for automatically leaving the parking garage of the vehicle 10 is sent to the vehicle 10 through wireless communication. As wireless communication, for example, BLE (Bluetooth Low Energy: Bluetooth Low Energy (registered trademark)) is used. The vehicle 10 receives the instruction signal sent from the smart phone 60 through the communication unit 24. The automatic parking control unit 55 of the vehicle 10 performs the control of automatically leaving the parking garage of the vehicle 10 according to the instruction signal received through the communication unit 24.

[0089] <Hardware Structure of Smartphone 60>

[0090] Figure 5 60 is a diagram showing an example of a hardware configuration of a smartphone 60. The smartphone 60 can, for example,Figure 5 It is implemented by the information processing device 80 shown. The information processing device 80 includes a processor 81, a memory 82, a communication interface 83, and a user interface 84. The processor 81, the memory 82, the communication interface 83, and the user interface 84 are connected by a bus 85, for example.

[0091] The processor 81 is a circuit for signal processing, for example, a CPU (Central Processing Unit) responsible for the overall control of the information processing device 80. The processor 81 is an example of the control unit of the present invention. In addition, the processor 81 can also be implemented by other digital circuits such as an FPGA (Field Programmable Gate Array) or a DSP (Digital Signal Processor). Furthermore, the processor 81 can also be implemented by combining multiple digital circuits.

[0092] The memory 82 includes, for example, a main memory and an auxiliary memory. The main memory is, for example, a RAM (Random Access Memory). The main memory is used as the working area of the processor 81.

[0093] The auxiliary memory is, for example, a non-volatile memory such as a magnetic disk, an optical disk, or a flash memory. Various programs for operating the information processing device 80 are stored in the auxiliary memory. The programs stored in the auxiliary memory are loaded into the main memory and executed by the processor 81.

[0094] In addition, the auxiliary memory can also include a removable memory that can be removed from the information processing device 80. Examples of the removable memory include a USB (Universal Serial Bus) flash drive, an SD (Secure Digital) memory card, and an external hard disk drive.

[0095] The communication interface 83 is a communication interface for wireless communication with the outside of the information processing device 80 (for example, the communication unit 24 of the vehicle 10). The communication interface 83 is controlled by the processor 81.

[0096] The user interface 84 includes, for example, an input device for receiving operation inputs from the user M, an output device for outputting information to the user M, and the like. The input device can be implemented by a touch panel, for example. The output device can be implemented by a display, a speaker, for example. The user interface 84 is controlled by the processor 81.

[0097] For example, the processor 81 performs movement indication control for indicating the movement of the vehicle 10. Specifically, the processor 81 performs movement indication control for the vehicle 10 based on specific operations of the user M on the smartphone screen 61 of the smartphone 60. The specific operations include continuous position indication operations (e.g., swipe operations), rotation indication operations in a specified rotation direction (e.g., rotation swipe operations), etc. The smartphone screen 61 is an example of the display unit of the present invention.

[0098] In addition, the processor 81 performs the following control: generating a guidance image that prompts the user M to perform a specific operation on the smartphone screen 61 of the smartphone 60, and causing the generated guidance image to be displayed on the smartphone screen 61. The guidance image includes, for example, a position guidance image that prompts a position indication operation, a rotation guidance image that prompts a rotation indication operation, etc. Specifically, the processor 81 causes the smartphone screen 61 to display the guidance image when waiting for the acceptance of a specific operation. The waiting for acceptance includes: situations where a specific operation needs to be performed by the user M, and situations where a specific operation is expected to be performed by the user M, etc.

[0099] In addition, when the user M performs a specific operation based on the guidance image displayed on the smartphone screen 61, the processor 81 stores the position information related to the performed specific operation in the memory 82. The processor 81 stores the position information related to the specific operation for a clockwise rotation indication operation in the clockwise direction and a counterclockwise rotation indication operation in the counterclockwise direction, respectively. The position information related to the specific operation refers to information such as at which position (e.g., a position on the right side of the screen, a position on the left side, etc.) of the smartphone screen 61 the user M performed the specific operation. Among them, when the specific operation of the user M is a specific operation not based on the guidance image displayed on the smartphone screen 61, for example, when it is an operation completely different from the guidance image displayed on the smartphone screen 61, the processor 81 does not store this operation information in the memory 82. In addition, the processor 81 stores the position information related to the specific operation in the memory 82 when the processing of the movement indication control of the vehicle 10 is completed, and does not store it when the processing is not completed.

[0100] In addition, when the processor 81 is waiting for the acceptance of a specific operation and there is position information stored in the memory 82, based on this position information in the smartphone screen 61, it displays a guidance image prompting the specific operation. Specifically, when the processor 81 is waiting for the acceptance of a specific operation and there is position information of a specific operation performed by the user M stored, it causes the smartphone screen 61 to display a guidance image that prompts the specific operation and indicates the position based on this position information in the smartphone screen 61. In addition, when the processor 81 is waiting for the acceptance of a specific operation and there is position information representing the range of a specific operation performed by the user M stored, it causes the smartphone screen 61 to display a guidance image that prompts the specific operation and has an image representing the range of the specific operation performed by the user M on the smartphone screen 61 superimposed thereon. The position based on the position information represents a narrow area in the smartphone screen 61, and the range where the specific operation has been performed represents a wider area in the smartphone screen 61.

[0101] When the processor 81 is waiting for the acceptance of a specific operation and there is position information stored, based on the history of accepting the specific operation from the user M, it changes the image representing the range where the specific operation has been performed. Specifically, when the number of times of accepting the same specific operation increases, it sets the fade-in display of the image representing the range where this specific operation has been performed to a lighter display or hides it. In addition, it displays the image representing the range of the previous specific operation more prominently, and displays the image representing the range of the specific operation two operations before last lighter than the previous image. Furthermore, it is also possible to store the ranges where multiple specific operations have been performed and display the image within the range obtained by averaging them.

[0102] <Example of movement instruction control performed by the processor 81>

[0103] Next, refer to Figures 6 to 14 An example of the movement instruction control of the vehicle 10 performed by the processor 81 of the smartphone 60 will be described.

[0104] Figure 6 and Figure 7 is a flowchart showing the out-of-warehouse instruction control of the vehicle 10 performed by the processor 81 of the smartphone 60. Figures 8 to 14 is a diagram showing an example of the image displayed on the smartphone screen 61 of the smartphone 60 that performs the out-of-warehouse instruction control for the vehicle 10.

[0105] Figure 6 The processing shown, for example, starts when a touch operation is performed on an automatic out-of-warehouse button (not shown) displayed on the smartphone screen 61 of the smartphone 60. In addition, the control of this out-of-warehouse instruction, for example, is executed in a situation as shown when the user M wants to use the smartphone 60 to automatically move the vehicle 10 out of the parking space P. Figure 4 as shown.

[0106] The processor 81 determines whether an automatic storage and retrieval instruction indicating the start of automatic storage and retrieval has been received (step S11).

[0107] In step S11, when an instruction for automatic storage and retrieval has not been received (step S11: No), the processor 81 repeatedly performs the process of step S11 until an instruction for automatic storage and retrieval is received.

[0108] In step S11, when an instruction for automatic storage and retrieval has been received (step S11: Yes), the processor 81 causes the smartphone screen 61 to display a first guidance image 62 that guides the instruction operation for automatic storage and retrieval (step S12).

[0109] As Figure 8 and Figure 9 shown, the first guidance image 62 is displayed, for example, as a long horizontal bar extending in the left - right direction on the smartphone screen 61. In addition, the up - down, left - right directions of the smartphone screen 61 are, for example, directions based on the orientation of the text displayed on the smartphone screen 61. Below the first guidance image 62 on the smartphone screen 61, guidance messages 63a of the operation guidance "Place your finger on the strip (long horizontal bar)" (refer to Figure 8 ), and guidance messages 63b of the operation guidance "If the finger continues to move up and down on the screen, the vehicle moves" (refer to Figure 9 ) are alternately displayed. The first guidance image 62 is displayed at approximately the central part in the up - down direction of the smartphone screen 61. Further, above the first guidance image 62 on the smartphone screen 61, an execution status image 64 indicating the execution status of the storage and retrieval instruction control for the vehicle 10 is displayed. In addition, on the right side of the execution status image 64, a caution message 65 of "Please directly confirm the surrounding situation" to remind the user M performing the instruction operation of caution, and a stop button 66 for aborting the automatic storage and retrieval control are displayed.

[0110] Next, the processor 81 determines whether the first guidance image 62 displayed in step S12 has been touched (step S13). Specifically, the processor 81 determines whether a finger is placed on the first guidance image 62 (long horizontal bar).

[0111] In step S13, when the first guidance image 62 has not been touched (step S13: No), the processor 81 repeatedly performs the process of step S13 until the first guidance image 62 is touched.

[0112] In step S13, when the first guidance image 62 has been touched (step S13: Yes), the processor 81 displays a second guidance image 67a or a second guidance image 67b at the touched position (step S14).

[0113] As Figure 10 and Figure 11 shown, the second guiding images 67a, 67b are, for example, horizontal bars that are shorter in the left - right direction than the first guiding image 62 and are displayed at the first positions 62a, 62b where the user M touches (places a finger) in the first guiding image 62. Figure 10 The second guiding image 67a shown is the guiding image displayed when the user M touches the right - hand side portion of the first guiding image 62, that is, the first position 62a. Figure 11 The second guiding image 67b shown is the guiding image displayed when the user M touches the left - hand side portion of the first guiding image 62, that is, the first position 62b.

[0114] Next, the processor 81 determines whether the memory 82 stores position information corresponding to the first positions 62a, 62b where the user M touches in the first guiding image 62, that is, the positions where the second guiding images 67a, 67b are displayed (step S15).

[0115] In step S15, when the memory 82 does not store the position information (step S15: No), the processor 81 directly proceeds to the process of step S17.

[0116] In step S15, when the memory 82 stores the position information (step S15: Yes), the processor 81 sets the reference position of the guiding image prompting a specific operation based on this stored position information (step S16), and then proceeds to step S17.

[0117] Here, the automatic out - of - garage of the vehicle 10 using the smart phone 60 is the first automatic out - of - garage, and the position information related to the guiding image prompting a specific operation is not stored in the memory 82. Therefore, the processor 81 directly proceeds to the process of step S17 in step S15.

[0118] Next, the processor 81 displays third guiding images 68a, 68b in the first direction, and this first direction corresponds to the positions where the second guiding images 67a, 67b are displayed in step S14, that is, the positions where the user M touches (step S17).

[0119] The first direction corresponding to the position where the second guiding image 67a is displayed is the direction corresponding to the right - hand side first position 62a where the user M touches in the first guiding image 62. For example, in Figure 10 the shown case, it is the direction including the left - hand side direction symmetric to the right - hand side first position 62a. Specifically, as Figure 10As in the example shown, the first direction is the direction toward the second position 69a in the region diagonally below and to the left of the first position 62a. Similarly, the first direction corresponding to the position where the second guidance image 67b is displayed is the direction corresponding to the leftmost first position 62b touched by the user M in the first guidance image 62. For example, in the Figure 11 case shown, it is the direction including the right-side direction that is symmetrical to the leftmost first position 62b in the left-right direction. Specifically, as in the Figure 11 example shown, the first direction is the direction toward the second position 69b in the region diagonally below and to the right of the first position 62b.

[0120] In addition, as in the Figure 10 example shown, the third guidance image 68a is displayed as, for example, an arrow image that goes from the rightmost first position 62a diagonally below and to the left to the indicated position, i.e., the second position 69a. The third guidance image 68a is displayed as an image that prompts the user M to perform a continuous position indication operation in the first direction, i.e., from the first position 62a toward the second position 69a. The continuous position indication operation is, for example, a sliding operation from the first position 62a toward the second position 69a. Similarly, as in the Figure 11 example shown, the third guidance image 68b is displayed as, for example, an arrow image that goes from the leftmost first position 62b diagonally below and to the right to the indicated position, i.e., the second position 69b. The third guidance image 68b is displayed as an image that prompts the user M to perform a continuous position indication operation in the first direction, i.e., from the first position 62b toward the second position 69b. In addition, a guidance message 63c is displayed below the third guidance images 68a and 68b, guiding a continuous position indication operation such as "Please move downward while touching the screen."

[0121] In this way, when the rightmost first position 62a is touched, the second guidance image 67a is displayed, and then an arrow image guiding a sliding operation diagonally below and to the left is displayed. In addition, when the leftmost first position 62b is touched, the second guidance image 67b is displayed, and then an arrow image guiding a sliding operation diagonally below and to the right is displayed. The second guidance images 67a and 67b displayed at the first positions 62a and 62b are guidance images that are the starting points of continuous position indication operations (sliding operations).

[0122] Next, the processor 81 determines whether a first sliding operation has been performed, which is in the first direction from the position where the second guidance images 67a and 67b are displayed, i.e., the direction indicated by the arrow images of the third guidance images 68a and 68b (step S18).

[0123] In step S18, when the first sliding operation is not performed (step S18: No), the processor 81 repeatedly performs the process of step S18 until the first sliding operation is performed.

[0124] In step S18, when the first sliding operation is performed (step S18: Yes), the processor 81 displays the fourth guidance image 70, which faces the second direction corresponding to the position where the second guidance images 67a and 67b are displayed, that is, the position touched by the user M (step S19).

[0125] For example, when the first sliding operation is performed from the first position 62a where the second guidance image 67a is displayed toward the first direction, as Figure 10 and Figure 12 shown, the second guidance image 67a displayed at the first position 62a moves and is displayed at the second position 69a in the lower left diagonal. Moreover, the fourth guidance image 70 is displayed from the position of the second guidance image 67a that has moved and is displayed at this second position 69a in the second direction.

[0126] The second direction corresponding to the position where the second guidance image 67a is displayed is the direction corresponding to the right-side first position 62a touched by the user M in the first guidance image 62. In addition, the second direction is Figure 10 shown, a direction different from the first direction from the first position 62a toward the second position 69a in the lower left diagonal, that is, a direction different from the direction shown by the third guidance image 68a.

[0127] Therefore, when the first sliding operation is performed from the first position 62a where the second guidance image 67a is displayed toward the first direction (the direction toward the second position 69a), as Figure 12 shown, the fourth guidance image 70 is displayed from the position of the second guidance image 67a displayed at the second position 69a as an arrow image in a direction different from the first direction (the direction shown by the third guidance image 68a), for example, toward the right direction. The fourth guidance image 70 is displayed as an image prompting the user M to perform a continuous position indication operation in the second direction different from the first direction. The continuous position indication operation, for example, refers to a sliding operation from the second position 69a toward the right direction. In addition, a guidance message 63d is displayed above the fourth guidance image 70, which guides a continuous position indication operation such as "Please move horizontally while touching the screen".

[0128] Although not shown in the figure, when the first sliding operation is performed from the first position 62b where the second guidance image 67b is displayed toward the first direction (the direction toward the second position 69b), the second direction becomes Figure 11The third guide image 68b is shown in a different direction from the oblique right lower direction (first direction). The fourth guide image 70 is displayed as an arrow image pointing, for example, to the left from the position of the second guide image 67b displayed at the second position 69b.

[0129] Thus, when the first position 62a on the right side is touched, the second guide image 67a moves and is displayed at the second position 69a on the lower left side, and then an arrow image for guiding a sliding operation in the right direction is displayed. In addition, when the first position 62b on the left side is touched, the second guide image 67b moves and is displayed at the second position 69b on the lower right side, and then an arrow image for guiding a sliding operation in the left direction is displayed.

[0130] In addition, the fourth guide image 70 is, for example, Figure 10 The fourth guide image 70 is displayed in a second direction different from the first direction before the leading end of the third guide image 68a indicated by the arrow image, which indicates the position of the slide operation in the first direction, reaches the end of the smartphone screen 61 (before it extends from the smartphone screen 61). In addition, the fourth guide image 70 may be displayed in a second direction different from the first direction when the distance from the position of the slide operation in the first direction, which indicates the length of the third guide image 68a indicated by the arrow image, exceeds a predetermined length.

[0131] Next, the processor 81 determines whether a second sliding operation is performed from the second position 69a, 69b where the second guide images 67a, 67b are displayed toward the second direction, ie, the direction indicated by the arrow image of the fourth guide image 70 (step S20).

[0132] In step S20 , when the second slide operation is not performed (step S20 : ​​No), processor 81 repeats the process of step S20 until the second slide operation is performed.

[0133] In step S20 , when the second slide operation is performed (step S20 : ​​YES), the processor 81 displays the fifth guide image 71 in the rotation direction corresponding to the position where the second guide images 67 a , 67 b were first displayed, that is, the position where the touch operation was started (step S21 ).

[0134] For example, the rotation direction corresponding to the position where the second guide image 67a is displayed becomes the rotation direction corresponding to the first position 62a on the right side touched by the user M in the first guide image 62. Furthermore, when the first position 62a on the right side is touched, the corresponding rotation direction becomes counterclockwise. At this time, the fifth guide image 71 rotating counterclockwise is displayed on the smartphone screen 61. Specifically, Figure 13As shown, when the first position 62a on the right side is touched, the fifth guiding image 71 that rotates counterclockwise as shown by the arrow 72 is displayed. For example, the fifth guiding image 71 is displayed in a manner where 6 spheres rotate counterclockwise. The fifth guiding image 71 is displayed in a manner that rotates from the position of the second guiding image 67a displayed at the second position 69a. The fifth guiding image 71 is displayed as a rotation guiding image that prompts the user M to perform a rotation indicating operation that generates a counterclockwise rotation. The rotation indicating operation that generates a counterclockwise rotation refers to, for example, a rotation sliding operation of sliding counterclockwise. In addition, a guiding message 63e is displayed above the fifth guiding image 71, which guides a rotation indicating operation such as "Please rotate while touching the screen."

[0135] Although not shown in the figure, when the first position 62b on the left side is touched, the corresponding fifth guiding image 71 that rotates clockwise is displayed. Specifically, the fifth guiding image 71 is displayed such that 6 spheres rotate clockwise from the position of the second guiding image 67b displayed at the second position 69b. Figure 11 from the position of the second guiding image 67b displayed at the second position 69b.

[0136] In this way, when the first position 62a on the right side is touched, a plurality of counterclockwise rotating sphere images are displayed, which guide a rotation sliding operation that rotates counterclockwise from the position of the second guiding image 67a that moves to the second position 69a. In addition, when the first position 62b on the left side is touched, a plurality of clockwise rotating sphere images are displayed, which guide a rotation sliding operation that rotates clockwise from the position of the second guiding image 67b that moves to the second position 69b. The second guiding images 67a and 67b displayed at the first positions 62a and 62b are rotation guiding images that serve as the starting points of the rotation indicating operation (rotation sliding operation).

[0137] Next, the processor 81 enters Figure 7 the process shown, and determines whether the rotation sliding operation has started (step S22).

[0138] In step S22, when the rotation sliding operation has not started (step S22: No), the processor 81 repeatedly performs the process of step S22 until the rotation sliding operation starts.

[0139] In step S22, when the rotation sliding operation has started (step S22: Yes), the processor 81 hides the fifth guiding image 71 displayed on the smartphone screen 61, and displays an indicating icon 73 that moves following the indicating position of the rotation sliding operation of the user M (step S23).

[0140] For example, as Figure 14 shown, the fifth guiding image 71 (refer to Figure 13)Set to hidden, and display the indication icon 73 indicating the automatic out-of-warehouse instruction of the vehicle 10. The indication icon 73 moves following the touch position of the user M on the smartphone screen 61. The indication icon 73 is Figure 13 the indication icon whose displayed second guidance image 67a moves following the movement of the finger of the user M performing the rotation and sliding operation. In addition, a guidance message 63f such as "Stop when the finger leaves" is displayed, which is used to stop the automatic out-of-warehouse of the vehicle 10. Thus, when performing a rotation and sliding operation on the smartphone screen 61, the indication icon 73 rotates and moves following its touch position, and as it rotates and moves, the vehicle 10 starts to move. In addition, if the finger touching the smartphone screen 61 leaves the smartphone screen 61, the operation of the vehicle 10 stops.

[0141] Next, the processor 81 starts the automatic out-of-warehouse of the vehicle 10 by sending an out-of-warehouse start signal to the vehicle 10 (step S24). Thus, the automatic parking control unit 55 of the vehicle 10 starts the automatic out-of-warehouse of the vehicle 10 according to the out-of-warehouse start signal sent from the smartphone 60. When starting the automatic out-of-warehouse of the vehicle 10, as Figure 14 shown, an image corresponding to the movement of the vehicle 10, that is, the execution state image 64 of the vehicle 10 exiting from the parking space P, is displayed.

[0142] Next, the processor 81 determines whether the rotation and sliding operation on the smartphone screen 61 has stopped (step S25).

[0143] In step S25, when the rotation and sliding operation has stopped (step S25: Yes), the processor 81 sends an out-of-warehouse stop signal to the vehicle 10, thereby stopping the automatic out-of-warehouse of the vehicle 10 (step S26).

[0144] Next, the processor 81 determines whether the rotation and sliding operation on the smartphone screen 61 starts again (step S27).

[0145] In step S27, when the rotation and sliding operation has not started again (step S27: No), the processor 81 stands by until the rotation and sliding operation starts again.

[0146] In step S27, when the rotation and sliding operation starts again (step S27: Yes), the processor 81 starts the indication control of the automatic out-of-warehouse of the vehicle 10 again (step S28), returns to step S23, and repeats each process.

[0147] On the other hand, in step S25, when the rotational sliding operation has not stopped (step S25: No), the processor 81 determines whether the automatic exit of the vehicle 10 is completed (step S29). The completion of the automatic exit of the vehicle 10 can be identified based on, for example, an exit completion signal sent from the automatic parking control unit 55 of the vehicle 10. The automatic exit is completed, for example, when the vehicle 10 moves outside the parking space P.

[0148] In step S29, when the automatic exit of the vehicle 10 is not completed (step S29: No), the processor 81 returns to step S25 and repeats each process.

[0149] In step S29, when the automatic exit of the vehicle 10 is completed (step S29: Yes), the processor 81 stores the position information of the sliding operation performed by the user M in the memory 82 (step S30), and ends the process of this exit instruction control. The position information of the sliding operation performed by the user M includes: the information of the touch position of the first guidance image 62 touched by the user M in step S13, the position information of the first sliding operation performed by the user M in step S18, the position information of the second sliding operation performed by the user M in step S20, and the position information of the rotational sliding operation performed by the user M in step S22. The processor 81 stores in the memory 82 the position information of where on the smartphone screen 61 the user M performed the sliding operation. In addition, in the middle of the above series of processes, for example, when the user M touches and operates the abort button 66, the processor 81 ends the process of this exit instruction control.

[0150] <Subsequent Movement Instruction Control>

[0151] Next, with reference to Figures 15 to 19 and Figure 6 and Figure 7 The flowchart of, the exit instruction control in the subsequent automatic exits using the smartphone 60 will be described. Figures 15 to 19 is a diagram showing an example of the image displayed on the smartphone screen 61 of the smartphone 60, corresponding to Figures 10 to 14 in the above initial exit instruction control.

[0152] In the case of the subsequent exit instruction control, Figure 6 The processes from step S11 to step S16 are also executed in the same manner as the processes from step S11 to step S16 in the above initial exit instruction control. Hereinafter, it will be described that this exit instruction control is the second exit instruction control.

[0153] For example, in the second automatic out-of-warehouse operation, the first positions 62a and 62b of the first guiding image 62 touched by user M are on the same side as the first positions 62a and 62b of the first guiding image 62 touched by user M in the first automatic out-of-warehouse operation. Specifically, at the time of the first automatic out-of-warehouse operation and the second automatic out-of-warehouse operation, user M touched the first position 62a on the right side of the first guiding image 62, or at the time of the first automatic out-of-warehouse operation and the second automatic out-of-warehouse operation, user M touched the first position 62b on the left side of the first guiding image 62.

[0154] Since the position information of the first sliding operation performed by user M is stored in the memory 82, the processor 81 enters step S16 in the branch process of step S15, and sets the reference position of the guiding image for prompting a specific operation based on the stored position information of the first sliding operation. The reference position of the guiding image refers to the position that serves as the display reference for the guiding images (the third guiding images 68a and 68b, the fourth guiding image 70, and the fifth guiding image 71) displayed on the smartphone screen 61. The reference position of the guiding image is, for example, the second position 69a corresponding to Figure 10 , Figure 12 , Figure 13 as shown, and the position of the second position 69b shown in Figure 11 .

[0155] For example, in the first out-of-warehouse instruction control, when the position of the first sliding operation performed by user M in step S18 is a position to the right of the third guiding image 68a (refer to Figure 10 ) displayed on the smartphone screen 61, in the second out-of-warehouse instruction control, the reference position of the guiding image set in step S16, that is, the second position 169a, is set to a position to the right of the second position 69a shown in Figure 10 . Similarly, in the first out-of-warehouse instruction control, when the position of the first sliding operation performed by user M in step S18 is a position to the left of the third guiding image 68b (refer to Figure 11 ) displayed on the smartphone screen 61, in the second out-of-warehouse instruction control, the reference position of the guiding image set in step S16, that is, the second position 169b, is set to a position to the left of the second position 69b shown in Figure 11 .

[0156] Next, the processor 81 displays the third guiding images 168a and 168b, which in step S14 face the first direction corresponding to the position where the second guiding images 67a and 67b are displayed, that is, the position touched by user M, and face the second position 169a or 169b that is to the right or left set in step S16 (step S17).

[0157] Towards the first direction corresponding to the position where the second guiding image 67a is displayed, that is, the third guiding image 168a towards the second position 169a on the right is, for example, Figure 15 the third guiding image 168a shown as follows. As Figure 15 shown, the third guiding image 168a is displayed towards the second position 169a, and this second position is displayed at a position to the right of the Figure 10 second position 69a shown as follows. In addition, towards the first direction corresponding to the position where the second guiding image 67b is displayed, that is, the third guiding image 168b towards the second position 169b on the left is, for example, Figure 16 the third guiding image 168b shown as follows. As Figure 16 shown, the third guiding image 168b is displayed towards the second position 169b, and this second position is displayed at a position to the left of the Figure 11 second position 69b shown as follows.

[0158] Next, the processor 81 determines whether a first sliding operation in the first direction shown by the third guiding images 168a and 168b has been performed (step S18). If the first sliding operation has been performed (step S18: Yes), starting from the position of the second guiding image 167a that has moved and is displayed at the second position 169a on the right or left, the fourth guiding image 170 towards the second direction is displayed (step S19).

[0159] Specifically, when the first sliding operation is performed from the first position 62a where the second guiding image 67a is displayed towards the first direction (the direction towards the second position 169a), as Figure 17 shown, the fourth guiding image 170 starts from the position of the second guiding image 167a displayed at the second position 169a on the right and is displayed as an arrow image, for example, in a direction different from the first direction (the direction shown by the third guiding image 168a), such as towards the right direction. The fourth guiding image 170 is displayed at a position to the right of the Figure 12 fourth guiding image 70 shown as follows.

[0160] Although not shown in the figure, when the first sliding operation is performed from the first position 62b where the second guiding image 67b is displayed towards the first direction (the direction towards the second position 169b), the fourth guiding image 170 starts from the position of the second guiding image displayed at the second position 169b on the left and is, for example, displayed as an arrow image towards the left direction.

[0161] In addition, at this time, for example, in the initial outbound instruction control, the position of the second sliding operation performed by the user M in step S20 is at a position higher than the Figure 12When the position of the fourth guiding image 70 shown is to the right or left, the position indicated by the fourth guiding image 170 shown in step S19 can also be displayed as a position to the right or left of the position indicated by the fourth guiding image 70 shown. Figure 17 The position indicated by the fourth guiding image 170 is displayed as being to the right or left of the position indicated by the fourth guiding image 70 shown. Figure 12 When the position of the fourth guiding image 70 shown is to the right or left, the position indicated by the fourth guiding image 170 shown in step S19 can also be displayed as a position to the right or left of the position indicated by the fourth guiding image 70 shown.

[0162] Next, the processor 81 determines whether a second sliding operation in the second direction shown by the fourth guiding image 170 has been performed (step S20). When the second sliding operation has been performed (step S20: Yes), starting from the position of the second guiding image 167a shown at the second position 169a or 169b to the right or left, a fifth guiding image 171 in the rotational direction corresponding to the position where the touch operation started is displayed (step S21).

[0163] Specifically, when a second sliding operation in the second direction is performed from the second position 169a to the right where the second guiding image 167a is displayed, as Figure 18 shown, starting from the position of the second guiding image 167a shown at the second position 169a to the right, a guiding image for guiding a counterclockwise rotational sliding operation as shown by the arrow 172 is displayed. The fifth guiding image 171 is displayed at a position to the right of the Figure 13 fifth guiding image 71 shown.

[0164] Although not shown in the figure, when a second sliding operation in the second direction is performed from the second position 169b to the left where the second guiding image is displayed, the fifth guiding image 171 starts from the position of the second guiding image shown at the second position 169b to the left and is displayed as a guiding image for clockwise rotation.

[0165] In addition, at this time, for example, in the initial outbound instruction control, when the position of the rotational sliding operation performed by the user M in step S22 is to the right or left of the position of the rotational sliding operation shown Figure 13 the rotational position of the fifth guiding image 171 shown in step S21 can also be displayed as a position to the right or left of the rotational position of the Figure 18 fifth guiding image 71 shown. Figure 13 When the position of the rotational sliding operation performed by the user M in step S22 is to the right or left of the position of the rotational sliding operation shown, the rotational position of the fifth guiding image 171 shown in step S21 can also be displayed as a position to the right or left of the rotational position of the fifth guiding image 71 shown.

[0166] Next, the processor 81 proceeds to Figure 7For the process shown, it is determined whether a rotation and slide operation has started (step S22). If the rotation and slide operation has started (step S22: Yes), the fifth guiding image 171 is set to be hidden, and then, starting from the position of the second guiding image 167a displayed at the second position 169a on the right or the second position 169b on the left, an indicating icon 173 that moves following the indicating position of the rotation and slide operation of the user M is displayed (step S23).

[0167] Specifically, as Figure 19 shown, a slide box 174 for performing a rotation and slide operation is displayed starting from the position of the second guiding image 167a displayed at the second position 169a on the right, and the indicating icon 173 is displayed within this slide box 174. The slide box 174 and the indicating icon 173 are displayed at a position to the right of the Figure 14 position of the slide box 74 and the indicating icon 73 shown.

[0168] Next, the processor 81 enters the process of step S22 to start the automatic out-of-warehouse of the vehicle 10. The processes of steps S22 to S29 are executed in the same manner as the processes of steps S22 to S29 in the above-mentioned first out-of-warehouse instruction control. Moreover, in step S30, the processor 81 stores the position information of the slide operation performed by the user M in the second out-of-warehouse instruction control in the memory 82, and ends the process of this out-of-warehouse instruction control.

[0169] In addition, in the above out-of-warehouse instruction control example, the second out-of-warehouse instruction control is described, and the same control is also performed in the out-of-warehouse instruction control after the third time. Furthermore, in the second out-of-warehouse instruction control, a guiding image prompting a specific operation is displayed based on the position information of the slide operation stored in the memory 82 in the first out-of-warehouse instruction control, but this is not limited in the case of the out-of-warehouse instruction control after the third time. For example, in the out-of-warehouse instruction control after the third time, a guiding image can be displayed based on the last position information, or a guiding image can be displayed based on the average position information among the position information of several past times.

[0170] As described above, when waiting to receive a specific operation, the processor 81 of the smartphone 60 causes the smartphone screen 61 to display a guiding image prompting a specific operation (slide operation, rotation and slide operation), and stores the position information of the specific operation performed by the user M based on the guiding image. Moreover, when waiting to receive a specific operation, if the position information is stored, a guiding image prompting a specific operation is displayed based on the position information in the smartphone screen 61. Thus, a guiding image can be displayed based on the position of the operation that was accepted as a specific operation based on the slide operation performed by the user M in the past. Therefore, it is possible to guide the user M to perform an operation with less burden and improve usability.

[0171] In addition, when the processor 81 of the smartphone 60 is waiting for the acceptance of a specific operation and the location information is stored, a guidance image that prompts the specific operation and indicates the location based on the location information in the smartphone screen 61 is displayed. Thus, the specific operation can be performed according to the guidance image indicating the operation location, so that an operation with less burden can be guided to the user M, improving usability.

[0172] In addition, the processor 81 of the smartphone 60 stores the location information for the rotation indication operation in the clockwise direction and the rotation indication operation in the counterclockwise direction respectively. Thus, even when the user M performs a rotation indication operation in either the clockwise direction or the counterclockwise direction (right hand / left hand), the rotation guidance image based on the stored location information of the rotation indication operation can be displayed on the smartphone screen 61. Therefore, a sliding operation with less burden can be guided to the user M.

[0173] <Variant Example of Movement Indication Control>

[0174] Next, refer to Figures 20 to 23 A variant example of the movement indication control of the vehicle 10 performed by the processor 81 of the smartphone 60 will be described.

[0175] <First Variant Example>

[0176] Figure 20 FIG. is a diagram showing a third guidance image displayed on the smartphone screen 61 of the smartphone 60 in the first variant example of the movement indication control. Figure 20 The image is equivalent to the Figure 10 and Figure 15 images in the above-described embodiment. As Figure 20 shown, when the user M touches the first position 62a on the right side of the first guidance image 62, the processor 81 displays a third guidance image 68a from the second guidance image 67a displayed at the first position 62a toward the second position 69a, and superimposes and displays an operation range image 181 indicating the range of the specific operation performed by the user M on the guidance image. In the specific operation, as described above, it includes a sliding operation and a rotation sliding operation.

[0177] In the illustrated example, the operation range image 181 is displayed as an image of a slanted region surrounding the second guidance image 67a and the third guidance image 68a. The operation range image 181 is an image representing the range in which a specific operation was performed during the last automatic storage and retrieval. Thus, for example, if during the last automatic storage and retrieval, the user M performed a specific operation within a range to the right of the position of the guidance image displayed on the smartphone screen 61, the operation range image 181 is displayed at a position to the right on the smartphone screen 61. In contrast, if during the last automatic storage and retrieval, the user M performed a specific operation within a range to the left of the position of the guidance image displayed on the smartphone screen 61, the operation range image 181 is displayed at a position to the left on the smartphone screen 61. In the illustrated example, a case where the operation range image 181 is displayed at a position to the right is shown.

[0178] Thus, according to the processor 81 of the first modification example, when waiting for acceptance of a specific operation, if position information is stored, the smartphone screen 61 is caused to display a guidance image that prompts a specific operation and superimposes an image representing the range in which the specific operation was performed. Thereby, it is possible to perform a specific operation in accordance with the guidance image indicating the operation range, and thus it is possible to guide the user M to perform an operation with less burden, improving usability.

[0179] <Second Modification Example>

[0180] Figure 21 FIG. shows the third guidance image 68a and the operation range image 182 displayed on the smartphone screen 61 of the smartphone 60 in the second modification example of the movement instruction control. As Figure 21 shown, the processor 81 causes the operation range image 182, which is superimposed and displayed on the third guidance image 68a, to be displayed with a changed display state according to the number of times of acceptance of a specific operation. Specifically, if the user M performs a specific operation within a range to the right of the position of the guidance image displayed on the smartphone screen 61 and the number of times of the specific operation to the right exceeds a certain value, the processor 81 displays the operation range image 182 in a lighter shade. The display of the operation range image 182 may also be set to be hidden when the number of acceptances exceeds a specified number. Thereby, even if there is no need for the operation range image that represented the range in which the user performed a specific operation in the past, it is possible to prevent this operation range image from being conspicuously displayed.

[0181] <Third Modification Example>

[0182] Figure 22 FIG. shows the third guidance image 68a and the operation range images 183 and 184 displayed on the smartphone screen 61 of the smartphone 60 in the third modification example of the movement instruction control. As Figure 22As shown, the display mode of the operation range images 183 and 184 superimposed on the third guidance image 68a for display by the processor 81 changes according to the number of times of acceptance of a specific operation (before and after acceptance). Specifically, the processor 81 displays the range where a specific operation was performed during the last automatic outbound shipment with a darker operation range image 183, displays the range where a specific operation was performed during the automatic outbound shipment two times before with a lighter operation range image 184, and sets the operation range image of the range where a specific operation was performed during the automatic outbound shipment three or more times before to be hidden. Thus, it is possible to reflect the tendency of the specific operation of the user M and display the operation range image, so the usability is improved. In addition, the display mode can be, for example, displayed according to the level of display transparency of the image in addition to the display of light and dark.

[0183] <Fourth Modification Example>

[0184] Figure 23 It is a diagram showing the third guidance image 168a and the operation range image 181 displayed on the smartphone screen 61 of the smartphone 60 in the fourth modification example of the movement instruction control. Figure 23 The image is the one in the above-described embodiment Figure 15 of the third guidance image 168a and the one in the first modification example Figure 20 of the operation range image 181 combined. As Figure 23 shown, the processor 81 displays the third guidance image 168a toward the second position 169a on the right, and superimposes and displays the operation range image 181 on the right indicating the range where the user M last performed a specific operation. In this way, by combining and displaying the guidance image and the operation range image, it is possible to guide the user M to perform an operation with less burden, and the usability is improved.

[0185] In the above first to fourth modification examples, the case of superimposing the operation range image on the third guidance image has been described, but it is not limited thereto. For example, it may also be possible to superimpose the operation range image on Figure 12 the fourth guidance image, Figure 13 the fifth guidance image, and Figure 14 the indication icon.

[0186] In addition, in the movement instruction control performed by the above-described processor 81, the automatic outbound shipment of the vehicle 10 has been described, but for example, it can also be applied to the automatic parking of the vehicle 10. Figure 24 It is a diagram showing the situation where the user M of the vehicle 10 uses the smartphone 60 carried by the user M from the outside of the vehicle 10 to give a parking instruction for automatically parking the vehicle 10 in the parking space P.

[0187] In addition, Figure 25 it shows that when Figure 6 and Figure 7The processing of the outbound instruction control shown is applied to the parking instruction control. An example of an image including a first guidance image displayed on the smartphone screen 61 when an instruction to start automatic parking is received in steps S11 and S12. As Figure 25 shown, a first guidance image 62 on which the user M performs a touch operation is displayed on the smartphone screen 61, and it is shown that the vehicle 10 stops in front of the parking space P and waits for automatic parking control.

[0188] The embodiments of the present invention have been described above, but the present invention is not limited to the above embodiments, and can be appropriately modified, improved, etc.

[0189] For example, in the above embodiment, an example in which the moving body is a vehicle (a four-wheeled car) has been described, but the present invention is not limited thereto. For example, it may also be a two-wheeled vehicle, a self-balancing vehicle, or the like. Furthermore, the idea of the present invention is not limited to vehicles, and can also be applied to robots, ships, airplanes, etc. that have a drive source and can move by the power of the drive source.

[0190] In addition, the control method described in the above embodiment can be implemented by a computer executing a pre-prepared control program. This control program is recorded on a computer-readable storage medium and executed by reading it out from the storage medium. In addition, this control program can be provided in the form of being stored in a non-temporary storage medium such as a flash memory, or can be provided via a network such as the Internet. The computer that executes this control program may be included in the control device, may be included in an electronic device such as a smartphone, a tablet terminal, or a personal computer that can communicate with the control device, or may be included in a server device that can communicate with these control devices and electronic devices.

[0191] In addition, at least the following matters are described in this specification. In addition, the corresponding components, etc. in the above embodiment are shown in parentheses, but are not limited thereto.

[0192] (1) An information terminal (smartphone 60) that can be carried by a user (user M) of a moving body (vehicle 10), the information terminal including:

[0193] A display unit (smartphone screen 61) that can perform a position indication operation on a display image; and

[0194] A control unit (processor 81) that performs movement control of the moving body based on a specific operation of the user on the display unit,

[0195] The control unit performs the following processing:

[0196] When waiting to receive the specific operation, cause the display unit to display a guidance image prompting the specific operation,

[0197] Store the location information of the specific operation performed by the user based on the guidance image.

[0198] When waiting for acceptance of the specific operation, if the location information is stored, display a guidance image that prompts the specific operation based on the location information in the display unit.

[0199] According to (1), by displaying a guidance image based on the location of an operation accepted as a specific operation performed by the user in the past, it is possible to guide an operation with less burden on the user, improving usability.

[0200] (2) The information terminal according to (1), wherein

[0201] When waiting for acceptance of the specific operation, if the location information is stored, the control unit displays a guidance image that prompts the specific operation and indicates the location based on the location information in the display unit.

[0202] According to (2), by displaying a guidance image indicating the location where a specific operation was performed, it is possible to guide an operation with less burden on the user, improving usability.

[0203] (3) The information terminal according to (1) or (2), wherein

[0204] The location information represents the range in the display unit where the user performed the specific operation.

[0205] When waiting for acceptance of the specific operation, if the location information is stored, the control unit causes the display unit to display a guidance image that prompts the specific operation and superimposes an image representing the range.

[0206] According to (3), by displaying a guidance image indicating the range where a specific operation was performed, it is possible to guide an operation with less burden on the user, improving usability.

[0207] (4) The information terminal according to (3), wherein

[0208] When waiting for acceptance of the specific operation, if the location information is stored, the control unit changes the image representing the range based on the history of accepting the specific operation from the user.

[0209] According to (4), it is possible to display a guidance image that is more effective for the user.

[0210] (5) The information terminal according to any one of (1) to (4), wherein

[0211] The specific operation includes a rotation indication operation.

[0212] The guiding image includes a rotation guiding image for prompting the rotation indication operation.

[0213] The control unit stores the position information for the rotation indication operation in the clockwise direction and the rotation indication operation in the counterclockwise direction respectively.

[0214] According to (5), even when the user M performs a rotation indication operation in either the clockwise direction or the counterclockwise direction (right hand / left hand), it is possible to display a rotation guiding image based on the stored position information of the rotation indication operation, which can guide an operation with less burden on the user, thus improving usability.

[0215] (6) A control method for an information terminal, which can be carried by a user of a moving body and includes: a display unit capable of performing a position indication operation on a display image; and a control unit that performs movement control of the moving body based on a specific operation of the user on the display unit. Among them,

[0216] The control unit performs the following processing:

[0217] When waiting for acceptance of the specific operation, cause the display unit to display a guiding image for prompting the specific operation.

[0218] Store the position information of the specific operation performed by the user based on the guiding image.

[0219] When waiting for acceptance of the specific operation, if the position information is stored, display a guiding image for prompting the specific operation based on the position information in the display unit.

[0220] According to (6), by displaying a guiding image based on the position of an operation accepted as a specific operation performed by the user in the past, it is possible to guide an operation with less burden on the user, thus improving usability.

[0221] (7) A computer-readable recording medium that records a control program for causing a control unit of an information terminal to execute the following processing. The information terminal can be carried by a user of a moving body and includes: a display unit capable of performing a position indication operation on a display image; and the control unit that performs movement control of the moving body based on a specific operation of the user on the display unit. Among them, the processing includes:

[0222] When waiting for acceptance of the specific operation, cause the display unit to display a guiding image for prompting the specific operation.

[0223] Store the position information of the specific operation performed by the user based on the guidance image.

[0224] When waiting for acceptance of the specific operation, if the position information is stored, display a guidance image prompting the specific operation based on the position information in the display unit.

[0225] According to (7), by displaying a guidance image based on the position of the operation accepted as a specific operation performed by the user in the past, it is possible to guide an operation with less burden on the user, improving usability.

Claims

1. An information terminal that can be carried by a user of a moving body, the information terminal comprising: A display unit that can perform a position indication operation on a display image; and A control unit that performs movement control of the moving body based on a specific operation of the user on the display unit, The control unit performs the following processing: When waiting for acceptance of the specific operation, cause the display unit to display a guidance image prompting the specific operation, Store position information representing the range in which the user on the display unit has performed the specific operation based on the guidance image, When waiting for acceptance of the specific operation, if the position information is stored, cause the display unit to display a guidance image prompting the specific operation and superimposed with an image representing the range.

2. The information terminal according to claim 1, wherein, When waiting for acceptance of the specific operation, if the position information is stored, the control unit displays a guidance image prompting the specific operation and indicating the position based on the position information in the display unit.

3. The information terminal according to claim 1 or 2, wherein, When waiting for acceptance of the specific operation, if the position information is stored, the control unit changes the image representing the range based on the history of acceptance of the specific operation from the user.

4. The information terminal according to claim 1 or 2, wherein, The specific operation includes a rotation indication operation, The guidance image includes a rotation guidance image prompting the rotation indication operation, The control unit stores the position information for the rotation indication operation in the clockwise direction and the rotation indication operation in the counterclockwise direction respectively.

5. A control method, which is a control method of an information terminal that can be carried by a user of a moving body and comprises: a display unit that can perform a position indication operation on a display image; and a control unit that performs movement control of the moving body based on a specific operation of the user on the display unit, wherein, The control unit performs the following processing: When waiting for acceptance of the specific operation, cause the display unit to display a guidance image prompting the specific operation, Store position information representing the range in which the user on the display unit has performed the specific operation based on the guidance image, When waiting for acceptance of the specific operation, if the position information is stored, cause the display unit to display a guidance image prompting the specific operation and superimposed with an image representing the range.

6. A computer-readable recording medium that records a control program for causing a control unit of an information terminal to execute the following processing, the information terminal can be carried by a user of a moving body and comprises: a display unit that can perform a position indication operation on a display image; and the control unit that performs movement control of the moving body based on a specific operation of the user on the display unit, wherein, The processing includes: When waiting for acceptance of the specific operation, cause the display unit to display a guidance image prompting the specific operation, Store position information representing the range in which the user in the display unit has performed the specific operation based on the guidance image. When waiting for acceptance of the specific operation, if the position information is stored, cause the display unit to display a guidance image that prompts the specific operation and superimposes an image representing the range.

Images