Control method for input device capable of presenting tactile sense

Abstract

To provide a control method for an input device capable of presenting tactile sense, which can convey a warning to a user in a more easily recognizable and understandable manner in order to prevent an erroneous operation by the user.SOLUTION: A user terminal 51, which can use an input device 11 capable of changing tactile sense in operation, executes: operation reception means that receives an operation from a user via the input device 11; alert detection means that detects establishment of an alert condition for the operation of the user on the input device 11; and alert transmission means that transmits, when the alert condition is met, an alert tactile signal capable of providing tactile sense for warning to the input device 11.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to user alerts in a user interface. [Background technology]

[0002] When a user uses a device or system, it is common to contribute to safety by providing a warning to the user in situations where caution is required or an error has occurred. On general PCs and smartphones, an error message is often displayed on the screen or an error sound is emitted. In addition, when an interface that can provide a tactile sensation to the user is equipped, various methods of presenting warnings are being considered.

[0003] Patent Document 1 proposes a portable multi-function device with a touch screen that provides different tactile outputs for different alert conditions.

[0004] Patent Document 2 describes a vehicle volume warning system, which proposes a system that applies pressure to the head of a passenger using ultrasound to warn the passenger. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2023-40093 [Patent Document 2] Patent Publication No. 2021-5200 Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to provide a warning that is more easily recognized and understood by the user in order to prevent erroneous operations by the user. [Means for solving the problem]

[0007] This invention is On the computer, an alert detection means for detecting whether an alert condition is satisfied in response to a user's operation on an input device; an alert sending means for sending an alert tactile signal to the input device when the alert condition is met; The above problem was solved by the first solution, which is a program for executing the above.

[0008] In addition, the present invention provides a first solution, the alert detection means detects an alert condition that is established from the plurality of alert conditions; The alert sending means may employ a second solution, which sends different alert haptic signals depending on the different alert conditions.

[0009] Furthermore, in the first or second solving means of the present invention, A third solution may be adopted, in which the establishment of the alert condition is a preliminary step of the operation.

[0010] This invention is An input device capable of changing the tactile sensation during operation is available, an operation receiving means for receiving an operation from a user via the input device; an alert detection means for detecting whether an alert condition is satisfied in response to a user's operation on the input device; an alert sending means for sending an alert tactile signal to the input device when the alert condition is met; A fourth solution can be adopted in which the user terminal has a control unit that executes the above.

[0011] This invention is receiving an operation from a user using an input device capable of changing the tactile sensation when operated; detecting whether an alert condition is met in response to a user's operation on the input device; transmitting an alert haptic signal to the input device when the alert condition is met, the alert haptic signal being capable of providing a warning haptic sensation; A fifth solution can be adopted, which is a method for controlling a computer that executes the above. [Effects of the Invention]

[0012] This invention makes it possible to provide a user of a personal computer, smartphone, or the like with a different feeling than usual when performing or attempting to perform a specific operation. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a functional block diagram of an embodiment of a system having a user terminal and an input device according to the present invention. [Figure 2] Conceptual diagram of an MRF device, an example of an input device used in this invention. [Figure 3] Conceptual diagram of the input device (tap unit) used in this invention [Figure 4] (a) A table showing examples of tactile information used in the present invention; (b) A table showing examples of tactile information with increased resistance compared to (a); (c) A table showing examples of tactile information that gives the feeling of bumpy resistance. [Figure 5] (a) An example of a situation where an alert is not detected on the billing amount selection screen, (b) an example of a situation where an alert may be detected on the billing amount selection screen, and (c) an example of a situation where an alert may be detected on the final billing confirmation screen. [Figure 6] Figure 5 (a) and (b) show an example of the flow of steps to detect an alert. [Figure 7] An example of when an alert is detected on a selection screen that poses a security risk DETAILED DESCRIPTION OF THE INVENTION

[0014] DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a user terminal that is operated by a user via an input device, a control method thereof, and a program therefor.

[0015] The user terminal 51 used in this invention is a computer having an image output device 54 that displays images on a screen such as a display 72, and an input device 11 through which the user inputs data. A functional block diagram of an embodiment of this user terminal is shown in Figure 1. Specifically, general terminals such as personal computers and smartphones can be used.

[0016] The image output device 54 used in this invention is not particularly limited as long as it can display a screen user interface for operation, such as a liquid crystal monitor, an organic EL monitor, a projector, a head-mounted display, etc. In Fig. 1, a display 72 integrated into a housing is shown as an example of an output destination from the image output unit 70 that processes graphics of the user terminal 51, but the present invention is not particularly limited to this form and may be a separate housing connected to the image output unit 70 by wire or wirelessly.

[0017] The input device 11 used in this invention is preferably one that transmits an operation signal corresponding to the amount of change to the control unit when a certain amount of operation such as pressing, rotating, or sliding is performed.Specific examples include devices that move physical parts, such as multi-step or stepless buttons or levers, wheels such as mice that can rotate infinitely, trackballs, and tap devices that can detect the amount of pressure applied.

[0018] Furthermore, the input devices must be able to change the tactile sensation felt by the user when operating them: for example, the strength of resistance when pressing or sliding a button or lever, the strength of resistance when rotating a wheel or trackball, or the strength of resistance when pressing a tap device.

[0019] 2 and 3, a specific example of a tap device that can be suitably used as the input device 11 will be described. The tap device shown in this example is an input device and tactile presentation device that has a magnetorheological fluid (MRF) device (hereinafter abbreviated as "MRF device").

[0020] This MRF device is composed of a rotating shaft 41, a disk 32, yokes 34 and 35, a coil 37, a magnetorheological fluid 38, and casings 31 and 36. As shown in Figure 2, a space is provided around the disk 32 attached to the rotating shaft 41, sandwiched between the yokes 34 and 35, and a magnetorheological fluid 38, whose viscosity changes depending on the strength of the magnetic field applied, is introduced into this space. Also, a coil 37, which is supported by the yoke 35 and generates a magnetic field (indicated by the arrow in the figure), is housed in the space. The rotating shaft 41 is supported by a bearing 39 and is integrated with the disk 32, which is surrounded by the magnetorheological fluid 38. The resistance of the magnetorheological fluid 38 when rotating the rotating shaft 41 can be adjusted by the amount of current supplied to the coil 37, which generates the magnetic field. When a user applies force to rotate the rotating shaft 41, the viscosity of the magnetorheological fluid 38 is increased or decreased, thereby increasing or decreasing the resistance to the disk 32 integrated with the rotating shaft 41, and the user experiences an active haptic effect as "difficulty in rotating" the rotating shaft 41. In this example, the disk 32 corresponds to the rotating body that contacts the magnetorheological fluid 38 in the haptic output device 14. The sensor 16 may be attached to detect the angle of the disk 32 itself or the angle of the rotating shaft 41 that rotates in conjunction with the disk 32.

[0021] Figures 3(a) and 3(b) show a tap unit, an input device 11 using such an MRF device. It has a base 40 that is held between the fingers or palm and a finger rest 42 that rotates in conjunction with a rotation axis 41 of the MRF device attached to the base 40. The finger rest 42 corresponds to one or all of the index finger, middle finger, ring finger, and little finger. When the user bends the fingers to grip, it rotates around a fulcrum 43. In other words, this finger rest 42 is the part that the user directly touches and operates. When the finger rest 42 is pressed, a first link member 44 and a second link member 45 linked via a pin 46 rotate as shown in Figure 3(c). This movement rotates the rotation axis 41 connected to the second link member 45 relative to the base 40. The resistance to this rotation is increased or decreased by the magnetorheological fluid 38 of the MRF device.

[0022] The sensors 16 attached to this MRF device may include an angle sensor that measures displacement as an angle relative to the center of rotation of the finger rest part 42, and a torque sensor that measures the load on the finger rest part 42. Based on the data from these sensors 16, an appropriate amount of current is applied in that situation, generating an appropriate resistance and outputting a tactile sensation appropriate to the situation.

[0023] When the user grips the tap unit and pushes in the finger rest part 42, the second link member 45 rotates in conjunction with the rotation, changing the rotational position of the rotating shaft 41 and the disk 32. This is detected by the angle sensor 16. When the rotational position reaches a predetermined angle, the tactile control unit 21 checks the current value to be passed through the MRF device by referring to a tactile information table stored in the data storage unit 22 (described later). The tactile control unit 21 applies a voltage corresponding to this current value to the MRF device, allowing the user to experience the expected tactile sensation. When the sensor 16 detects a further change in the rotational position, it refers to the current value corresponding to the new rotational position, calculates a signal, and causes the user to experience the tactile sensation.

[0024] In other words, as an input device 11, by pressing the finger from the standard 0° position to a specified angle, it can be used as an input device for selection operations similar to a mouse button, and by adjusting the strength of the magnetic field applied to the magnetorheological fluid 38 with the amount of current, it is possible to adjust the resistance applied to the user's finger when pressing.

[0025] The tactile sensations provided by the input device 11 include normal tactile sensations, and when a user's operation is detected to meet an alert condition, a different tactile sensation is set as a warning sensation. Here, it is preferable that the warning sensation has a portion with stronger resistance than normal. This is because a more nimble operation than normal would not serve as a warning. Specifically, the resistance may be strong throughout the entire operation, or it may be set to alternate between relatively weak and relatively strong resistance. In the former case, the user simply feels heavy, making it easier for them to understand that the operation is important. In the latter case, the user directly experiences the scratchy fluctuations in resistance, which is unpleasant, and the setting emphasizes the warning element. However, regardless of the strength of the resistance, it is desirable not to prohibit the user from attempting to operate the device despite understanding the warning. This is because operation should not be stopped even if the user understands the warning.

[0026] The alert conditions include, for example, when an operation important from a security or economic perspective is performed or attempted. This alert can be used to alert the user that these operations require particular attention. Examples of operations that could pose financial problems include operations that involve a significant financial burden, such as proceeding to a charge exceeding a set amount or final confirmation during a bank transfer. Examples of operations that could pose security problems include deleting an account for an online service, uninstalling an app from a personal device, clearing user data on a personal device, and deleting system data or OS programs. In recent years, the number of situations requiring confirmation through button operations via dialog boxes has increased, which can lead to users repeatedly pressing buttons without carefully checking the contents, resulting in irreversible actions. According to this invention, when a user dialog box is displayed and a button is selected or operated, a different tactile sensation is provided, warning the user that the operation is not a normal operation that can be easily reversed, thereby preventing the user from unintentionally performing such an operation. The above is merely an example, and the alert conditions applicable to this invention are not limited to those described above. For example, a condition that prompts the user to take a rest can be set, such as when the user's continuous operation time exceeds a certain period of time.

[0027] In the case of the above tap device, the normal tactile sensation as resistance felt by the user depending on the finger position is as shown in Figure 4(a). In contrast, the tactile sensation data for a warning can be replaced with a table with a higher overall resistance as shown in Figure 4(b), or with a table with strong resistance felt in various places depending on the finger position as shown in Figure 4(c). Alternatively, instead of replacing the table itself, it is also possible to correct the original table by multiplying it by a coefficient or function.

[0028] Next, we will explain the other parts of the embodiment of the user terminal 51. However, these parts may be modified as appropriate within the scope of realizing the tactile sensations provided by the input device 11 required in the present invention.

[0029] The input device 11 has a communication unit 26 that can communicate with the communication unit 63 of the user terminal 51. It is necessary that the input device 11 is capable of at least receiving communication from the user terminal 51, and it is preferable that the communication unit 26 is capable of mutual communication, as this allows for more diverse control by the user terminal 51. Data and commands received by the communication unit 26 are sent to the output device control unit 20, and the input device 11 operates using these. In addition, data, history, logs, etc. from sensors, etc. may be sent from the output device control unit 20 to the user terminal 51 via the communication unit 26. In addition, the input device 11 may send part of the data to the user terminal 51 and receive the results as a reply.

[0030] Communication between communication unit 26 and communication unit 63 may be wired or wireless. In the case of wired communication, power may be supplied via a wired cable. The standard is not particularly limited; at the time of filing the present application, USB cable, Lightning (registered trademark) cable, Thunderbolt (registered trademark) cable, etc. are selectable, but any standard that enables similar or upwardly compatible communication may be used. In the case of wireless communication, any short-range wireless communication standard may be used, including various wireless LAN standards, Bluetooth (registered trademark; omitted below), Bluetooth LE, and wireless USB. However, because the amount of data required to implement at least the adjustment itself in this invention is small, a relatively slow standard with low power consumption, such as Bluetooth or Bluetooth LE, is preferably used. Of course, a high-speed communication standard that requires a large amount of data for operations other than those required in this invention may also be used.

[0031] The input device 11 preferably has a sensor 16 that measures the displacement of the device itself for the tactile sensation provided by the tactile output device 14, as well as the position, displacement, load, etc. when the user operates the tactile output device 14.

[0032] The user terminal 51 may be connectable to a server 91 located at the end of a network 92 connected via a router or terminal. It is preferable that the present invention can be used to operate services such as web pages and web applications provided by the server 91. To this end, the user terminal 51 preferably has a network interface (NWIF) 69 that has a wired LAN function, a wireless LAN function, or a function for connecting to a mobile communication network. The network interface 69 may be shared with the communication unit 63, or may be independent. Since the communication volume varies greatly, it is often preferable that the network interface be independent. The figure shows an example in which the network interface is independent.

[0033] Although not shown, other input devices may be provided in addition to the input device 11. In addition to the input device 11 that allows for the experience of tactile sensations, it is preferable to have a device that allows for general character input operations, such as a keyboard. However, it is difficult to provide tactile sensations with pointing devices such as on / off switches and mice without wheels. For this reason, it is preferable that these are not connected.

[0034] The user terminal 51 has a control unit 61 that performs calculations, etc. Specifically, it is a calculation device such as a CPU or GPU, and controls the behavior of the device, such as reading content, calculations, receiving input and output to the image output device 54, and communication with the input device 11.

[0035] The user terminal 51 has a memory unit 62 that stores data and programs. The memory unit 62 preferably has both a non-transient, tangible recording medium such as a non-volatile memory or a magnetic disk used for storage, and a volatile memory used for calculations. In the figure, it is not distinguished whether data or programs are loaded.

[0036] The storage unit 62 stores an operating system and application software (hereinafter, collectively referred to as "application 65") that display a user interface and can be operated by the user. The provision of tactile sensations by executing the various means described below may be performed by the application software or the operating system. The application 65 includes alert conditions that define the conditions under which a warning is issued to the user.

[0037] The app 65 also includes a haptic database 67 in the storage unit 62, which contains haptic information for reproducing the tactile sensations of objects in the virtual space, desktop, etc., via the input device 11 in parallel with output to the image output device 54. This haptic information is a set of values, functions, and other settings that determine under what circumstances and with what values ​​the input device 11 operates the haptic output device 14. For example, for the tap device described above, a table containing tactile sensations (current values) corresponding to finger positions, such as that shown in FIG. 4(a), can be used. Haptic information is selected for each object, such as a button, that the user actively attempts to operate in the user interface displayed on the display 72. Furthermore, haptic information is set to provide a tactile sensation for a warning when an alert condition for issuing a warning to the user is met. The tactile information for providing a tactile sensation for a warning may be a separate table prepared separately from the table for normal tactile sensations, or it may be a coefficient or function adjusted from the values ​​in the table for normal tactile sensations.

[0038] The above alert condition does not need to be limited to one, and multiple alert conditions may be set simultaneously. In this case, the alert detection means detects which of the multiple alert conditions is met. It is also preferable that different tactile information (alert tactile signal) is set for each of the multiple alert conditions. The alert transmission means transmits tactile information corresponding to the alert condition that is met, and the user is provided with a tactile sensation corresponding to the met condition.

[0039] A specific example of a method and program for controlling the user terminal 51 according to the present invention will be described below. The input device 11 uses a tap device, as shown in FIG. 3, to which an acceleration sensor or the like has been added, making it possible to detect the position on the screen to be touched (designated position information). That is, by pressing or grasping the finger position on an object at a designated position, the execution of a process associated with the object can be instructed, and the user can simultaneously feel a sensation corresponding to the object. The control unit 61 receives the designated position information from the input device 11 and accepts an operation to designate a position on the screen. The control unit 61 also executes an operation accepting means for accepting an operation from the user, so that if the finger position detected by the sensor 16 is pressed, it is processed as an operation being performed at that position.

[0040] When a user's operation satisfies one of the alert conditions stored in the app 65, a tactile warning is presented to the user. Here, a warning when making a charge will be described as a warning for confirming financial safety. As a first step, the app 65 displays a selection window 81 for the amount to be charged on the display 72, as shown in FIG. 5( a). Within the selection window 81, multiple charge amount buttons 82 for selecting the amount to be charged are displayed, one for each amount. The selection within this selection window 81 is indicated by a cursor 83 operated by the user via the input device 11. Note that the operation of the cursor 83 is not limited to this. It may also be based on position information of the input device 11 detected by a camera (not shown) separately connected to the user terminal 51, or on the user's operation of a trackball (not shown) separately connected to the user terminal 51.

[0041] The processing of the control unit 61 from this point on will be described with reference to the example flow chart of FIG. 6. When the screen of FIG. 5(a) is displayed, the alert conditions for issuing a warning have not been met (S101). As a preliminary step, the user operates the input device 11 to move the cursor 83 (S102). If the cursor 83 is not positioned on the billing amount button 82 (S103 → No), the normal tactile sensation set for the normal button 82a (here, the cancel button) is retrieved from the tactile database 67 and set as the tactile information to be sent to the input device 11 (S104). Here, the normal tactile sensation is, for example, the sensation of pressing a button. This tactile information is sent to the input device 11 (S105), and the system waits until the user performs a pressing operation with a tactile-sensitive finger on the input device 11 (S106). If the cursor 83 is moved without a pressing operation (S106 → No → S102), the destination of the cursor 83 is confirmed again (S103). When the user presses the input device 11 with his / her finger, the input device 11 provides a normal tactile sensation at the time of the operation (S107). In FIG. 5(a), when the cancel button is pressed, no particular warning is required, so the normal tactile sensation is provided.

[0042] 5(b), if the cursor 83 is positioned on the billing amount button 82 (S103 → Yes), the control unit 61 executes an alert detection process to calculate the sum of the additional planned billing amount and the monthly billing amount for the user of the user terminal 51, and determines whether the total exceeds the monthly billing limit set by the user (S111). If the upper limit is not exceeded (S111 → No), the control unit 61 sets the normal tactile sensation set for a normal button as tactile information (S104), transmits this tactile information to the input device 11 (S105), and provides the normal tactile sensation when the user presses the input device 11 to charge (S106 → Yes → S107). This is because the charge is within the set upper limit, and there is no need to warn the user by a tactile sensation. If the user moves the cursor without pressing the button, the process returns to the original flow (S106 → No → S102).

[0043] On the other hand, if the sum of the planned additional charge amount due to the charge amount button 82 where the cursor 83 is located and the monthly charge amount for the user of the user terminal 51 exceeds the monthly charge limit set by the user (S111 → Yes), it is determined that the alert condition for issuing a warning is met, and a tactile sensation for the warning is set. That is, the control unit 61 executes the alert transmission means by retrieving a table from the tactile database 67 as charge information, which is different from the table set for normal buttons and contains areas with higher resistance values ​​that feel resistance when pressed (S114), and transmitting the retrieved table to the input device 11 (S115). Note that in S114, a tactile signal in the form of a coefficient may be multiplied by each tactile value included in the normal tactile table to set a value. If the charge amount button 82 is not pressed (S116 → No → S102), the process returns to the cursor movement and the accompanying judgment. When the user presses the input device 11 with their finger as if to press the charge amount button 82 (S116 → Yes), the input device 11 provides a tactile sensation, including a portion with high resistance, as a warning (S117). As a result, even if the user does not fully check the screen display, the user can sense from the weight felt by their finger that the charge exceeds the upper limit they set, and can realize without overlooking that it is a financially problematic act (S118). If the user presses the input device to a sufficient angle despite the resistance, it means that the user understood the warning and proceeded with the charge, and the warning effect is exerted. At this time, it is preferable to set the input so that pressing to a degree that resistance is not enough to input consent.

[0044] The alert condition targeted by the alert detection means does not have to be the stage at which the charge amount is selected, but can be executed as long as it results in a warning at some stage of operation before the final decision. For example, we will explain this using the screen for the final decision on whether to charge, as shown in Figure 5(c), as an example. When the cursor 83 is on "Yes," the alert detection means is executed, and a tactile sensation for the warning is sent to the input device 11. When the cursor is on "No" or when the cursor is not on "Yes," a normal tactile sensation is sent to the input device 11, so that the warning can be experienced as a tactile sensation before the final decision.

[0045] A case in which it is used in a security situation will be described using an example of a dialog window 85 related to account settings, as shown in FIG. 7. The top three buttons 86 are common operations, so a normal tactile sensation is set. The bottom two buttons 86a are for deleting an account and uninstalling an app, and are actions that should not be performed unless necessary. Therefore, when the cursor 83 is positioned on these buttons 86a, an alert detection means is executed, and tactile information that provides a tactile sensation for a warning is transmitted to the input device 11. When the cursor 83 is moved away from the button 86a, the alert detection condition detects that no warning is necessary, and the control unit 61 transmits tactile information that provides a normal tactile sensation. [Explanation of symbols]

[0046] 11 Input Devices 14 Haptic Output Devices 16 sensors 20 Output device control section 21 Tactile control unit 22 Data storage unit 25 Power supply 26 Communications Department 31 Casing 32 disk 34 York 35 York 37 Coil 38 Magnetorheological fluid 39 Bearings 40 base 41 Rotation axis 42 parts 43 Fulcrum 44 First link material 45 Second link material 46 pins 51 User terminal 54 Image output device 61 Control Unit 62 Memory section 63 Communications Department 64 Tactile signal transmitter 65 apps 67 Tactile Database 69 Network Interfaces 70 Image output unit 72 Display 81 Selection Window 82 Charge amount button 82a, 86, 86a buttons 83 Cursor 85 Dialogue Windows 91 servers 92 Network

Claims

1. On the computer, an alert detection means for detecting whether an alert condition is satisfied in response to a user's operation on an input device; an alert sending means for sending an alert tactile signal to the input device when the alert condition is met; A program to execute.

2. the alert detection means detects an alert condition that is established from the plurality of alert conditions; The program according to claim 1 , wherein the alert transmitting means transmits different alert haptic signals depending on the detected alert condition.

3. 3. The program according to claim 1, wherein the establishment of the alert condition is a preliminary stage of the operation.

4. An input device capable of changing the tactile sensation during operation is available, an operation receiving means for receiving an operation from a user via the input device; an alert detection means for detecting whether an alert condition is satisfied in response to a user's operation on the input device; an alert sending means for sending an alert tactile signal to the input device when the alert condition is met; A user terminal having a control unit that executes the above.

5. receiving an operation from a user using an input device capable of changing the tactile sensation when operated; detecting whether an alert condition is met in response to a user's operation on the input device; transmitting an alert haptic signal to the input device when the alert condition is met, the alert haptic signal being capable of providing a warning haptic sensation; A method for controlling a computer that runs

Images