Information input apparatus and information input method

Abstract

To enhance correlationship between functions of an information input apparatus detachably having an electrode array at a side surface portion of a portable apparatus, and functions of the portable device.SOLUTION: An information input apparatus according to the present invention has an electrode array part capable of being fitted to a side surface portion held by a user of a portable device and including a plurality of electrodes arranged in a line, a switch part which can successively select one electrode among the plurality of electrodes, and a control unit which outputs current to the electrode selected by the switch part in a state in which the electrode array part is fitted to the side surface portion of the portable device, thereby transmitting, to the portable device, information relating to a detected holding state of the user as an operation input signal.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information input device and an information input method. [Background technology]

[0002] Conventionally, a user interface has been proposed in which an electrode array is placed on the side of a smartphone housing to present tactile sensations through electrical stimulation (see, for example, Non-Patent Documents 1 to 3). When a user holds a smartphone equipped with this user interface, the holding fingers come into contact with the electrodes, and tactile information can be presented to the user's holding hand through electrical stimulation. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Takami, Saito, Kameoka, and Kajimoto: "A study on a user interface for electrical skin stimulation on the side of a smartphone," Proceedings of the 27th Virtual Reality Society of Japan Conference, September 2022 [Non-Patent Document 2] Takami, Saito, Kameoka, and Kajimoto: "Study on tactile interaction in live streaming using electrical stimulation on the side of a smartphone", Information Processing Society of Japan Research Report, 2023 [Non-Patent Document 3] D.Groeger, M.Feick, A.Withana, J.Steimle, 2019, “Tactlets: Adding Tactile Feedback to 3D Objects Using Custom Printed Controls”, In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology Summary of the Invention [Problem to be solved by the invention]

[0004] As described above, a user interface has been proposed that includes an electrode array for tactile presentation that can be attached to the side of a mobile device such as a smartphone. In such a user interface, it is required to improve the interlocking between the functions of the user interface and the functions of the mobile device to which the user interface is attached.

[0005] The present invention has been made to meet the above demands. An object of the present invention is to provide a technique that can improve the interlocking between the functions of a user interface and the functions of a portable device in a user interface that includes a detachable electrode array attached to the side of the portable device. [Means for solving the problem]

[0006] In order to solve the above problems, the information input device of the present invention is capable of being attached to a side portion of a portable device held by a user, and comprises an electrode array unit including a plurality of electrodes arranged in a row, a switching unit capable of sequentially selecting one electrode from the plurality of electrodes, and a control unit that, when the electrode array unit is attached to the side portion of the portable device, outputs a current to the electrode selected by the switching unit to detect the user's holding state of the portable device, and transmits information regarding the user's holding state to the portable device as an operation input signal.

[0007] In addition, in order to solve the above-mentioned problems, the information input method of the present invention is an information input method using the information input device of the present invention, and includes the control unit outputting a current to an electrode selected by a switching unit with the electrode array unit attached to the side portion of the portable device, thereby detecting the user's gripping state of the portable device, and the control unit transmitting information regarding the user's gripping state to the portable device as an operation input signal. Effect of the Invention

[0008] According to the present invention having the above-mentioned configuration, in an information input device (user interface) having an electrode array section that is detachable from the side surface of a portable device, it is possible to improve the linkage between the functions of the information input device and the functions of the portable device. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram for explaining an outline of an operation during use of an information input device according to an embodiment of the present invention; [Diagram 2] 1 is a block diagram of an information input device according to an embodiment of the present invention; [Diagram 3] 2 is a diagram for explaining the configuration and operation of a switching IC of an information input device according to one embodiment of the present invention; [Figure 4] 2 is a diagram showing a configuration of an electrode array section of an information input device according to an embodiment of the present invention; FIG. [Diagram 5] 1 is a diagram showing an operation input to an information input device according to an embodiment of the present invention; [Figure 6] 1 is a diagram showing a detection principle of an operation input to an information input device according to an embodiment of the present invention; [Figure 7] 5A and 5B are diagrams showing an example of a detection signal obtained when an operation input function of the information input device according to the embodiment of the present invention is activated. [Figure 8] 11A and 11B are diagrams showing stimulation patterns provided by a tactile presentation function of an information input device according to an embodiment of the present invention. [Figure 9] 1 is a schematic diagram for explaining a tactile sensation provided to a user by a tactile sensation providing function of an information input device according to an embodiment of the present invention. [Figure 10] 10 is a flowchart showing a procedure for detecting a grip input operation performed by an information input device according to an embodiment of the present invention and a smartphone to which the information input device is attached. [Figure 11] 10 is a flowchart showing a procedure for a response process to a grip input operation performed by the information input device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information input device and an information input method according to an embodiment of the present invention will be specifically described below with reference to the drawings.

[0011] [Appearance and Function of Information Input Device] Fig. 1A is a diagram showing the appearance of a smartphone 2 equipped with an information input device 1 according to the present embodiment when held by a user, and Fig. 1B is a diagram showing a state of a tactile sensation providing operation by the information input device.

[0012] The information input device 1 of this embodiment is an interface that presents a tactile sensation to a user by electrical stimulation, and has an electrode unit 19 arranged on a long side of a smartphone 2 held by a user, as shown in FIG. 1A. The electrode unit 19 has two electrode array units 190 attached to one long side and the other long side of the smartphone 2, as shown in FIG. 1B. The information input device 1 passes a current (hereinafter referred to as a "stimulating current") through an electrode 19a of the electrode unit 19 to a finger 3 of a user holding the smartphone 2, and presents a tactile sensation by electrical stimulation. At this time, the stimulating current flows only through the electrode 19a (see the white circle in FIG. 1B) that the user's finger 3 is in contact with, so that a distributed tactile sensation can be presented on the electrode array unit 190.

[0013] 1A, in normal operation of the smartphone 2, the user operates the smartphone 2 by gripping the long side portion of the smartphone 2 to which the electrode unit 19 is attached. In the information input device 1 of this embodiment, the gripping operation is detected via the electrode unit 19, and the detection signal is used as an operation input signal for the smartphone 2.

[0014] That is, the information input device 1 is a user interface having both a user interface function for tactile presentation (hereinafter referred to as the "tactile presentation function") and a user interface function for operation input (hereinafter referred to as the "operation input function"). In this embodiment, when an operation input signal (information on skin impedance described later) corresponding to a user's gripping operation on the smartphone 2 is input to the smartphone 2 via the information input device 1, the smartphone 2 determines whether or not a predetermined gripping operation has been performed based on the operation input signal, and when the predetermined gripping operation has been performed, tactile information (electrical stimulation) corresponding to the predetermined gripping operation is presented to the user's fingers 3 in response.

[0015] [Internal configuration of information input device] Fig. 2 is a block diagram of the information input device 1. For convenience of explanation, Fig. 2 also shows the connection relationship between the information input device 1 and the smartphone 2 to which it is attached.

[0016] 2, the information input device 1 is connected to the smartphone 2 via a Universal Serial Bus (USB) cable 4. The information input device 1 operates with power supplied from the smartphone 2 via a power supply line 4a in the USB cable 4. Signals are transmitted and received between the information input device 1 and the smartphone 2 via a serial communication line 4b in the USB cable 4. Note that, in this embodiment, an example will be described in which the communication between the information input device 1 and the smartphone 2 is wired communication, but the present invention is not limited to this, and wireless communication may also be used.

[0017] In this embodiment, information on skin impedance (information on the user's gripping state) detected by the information input device 1 is transmitted as an operation input signal from the information input device 1 to the smartphone 2 via the serial communication line 4b. The smartphone 2 that has received the skin impedance information determines whether or not a predetermined gripping operation (grip input operation) is being performed based on the skin impedance information, and when it is determined that the predetermined gripping operation is being performed, a process (operation) corresponding to the predetermined gripping operation is performed. In addition, at this time, an output command signal for electrical stimulation corresponding to the predetermined gripping operation is transmitted from the smartphone 2 to the information input device 1 via the serial communication line 4b as a response signal (response command) to the predetermined gripping operation. Note that the output command signal for electrical stimulation transmitted from the smartphone 2 to the information input device 1 includes information such as, for example, stimulation intensity and stimulation pattern.

[0018] 2, the smartphone 2 includes a processor 2a for controlling functions of the smartphone 2. The process of determining whether or not a predetermined gripping operation (a grip input operation described later) has been performed based on the above-mentioned skin impedance information and the process of transmitting a response signal (a response command) based on the determination result are controlled by the processor 2a on software.

[0019] 2, the information input device 1 includes a control unit 10, a switching IC (Integrated Circuit) 18 (switching unit), and an electrode unit 19. The control unit 10 includes a microcontroller 11, a digital / analog converter 12 (hereinafter, referred to as “D / A converter 12”), and a voltage / current conversion circuit 13. The control unit 10 also includes a first resistive element 14, a second resistive element 15, an analog / digital converter 16 (hereinafter, referred to as “A / D converter 16”), and a boost circuit 17.

[0020] The connection relationship between the components of the information input device 1 is as follows. The microcontroller 11 is connected to the D / A converter 12, the A / D converter 16, and the switching IC 18. The microcontroller 11 is connected to the smartphone 2 via the power supply line 4a and the serial communication line 4b in the USB cable 4. The D / A converter 12 is connected to the voltage / current conversion circuit 13. The voltage / current conversion circuit 13 is connected to the boost circuit 17, one terminal of the first resistor element 14, and the switching IC 18. The other terminal of the first resistor element 14 is connected to one terminal of the second resistor element 15 and the A / D converter 16, and the other terminal of the second resistor element 15 is grounded. The boost circuit 17 is connected to the power supply line 4a. The switching IC 18 is connected to the electrode unit 19.

[0021] Each unit included in the control unit 10 is mounted on, for example, a single board, and the board (control unit 10) is detachably attached to the side opposite the display screen (not shown) of the smartphone 2, i.e., the palm side of the hand holding the smartphone 2. The switching IC 18 is also detachably attached to the side opposite the display screen of the smartphone 2. Note that, although the control unit 10 and the switching IC 18 are separate components in this embodiment, the present invention is not limited to this, and the control unit 10 and the switching IC 18 may be packaged.

[0022] As described above, the electrode unit 19 is configured by two electrode array units 190 (see FIG. 1B) that are detachably attached to the two long side surfaces of the smartphone 2, respectively.

[0023] In the information input device 1, when the operation input function or the tactile sense providing function is activated, the electrodes 19a provided in the electrode unit 19 are sequentially scanned and selected one by one at a predetermined time interval (switch state switching period), and a current is supplied to the selected electrode 19a (anode). At this time, all the electrodes 19a other than the one electrode 19a that is the anode are controlled to be cathodes. The scanning and selection operation of the electrodes 19a is controlled by a switch state switching control signal (see FIG. 2) input from the microcontroller 11.

[0024] The value of the stimulation current supplied to the electrode 19a (anode) selected when the tactile sensation providing function is activated is about several mA, but can be adjusted appropriately according to the user. On the other hand, the value of the current supplied to the electrode 19a (anode) selected when the operation input function is activated (hereinafter referred to as the "operation detection current") is set to a current value that is smaller than the stimulation current and does not cause the user to feel electrical stimulation. However, the present invention is not limited to this, and the value of the operation detection current may be set to the same value as the stimulation current.

[0025] [Principle of determining whether or not the electrode is in contact with the skin] In this embodiment, contact / non-contact (presence or absence of contact) between the electrode 19a and the finger 3 is determined based on the skin impedance Z (resistance value) of the finger 3 detected by the information input device 1 during a user's gripping operation. In order to measure the skin impedance Z, it is necessary to measure the voltage Vo (see FIG. 2) applied to the skin of the finger 3 and the current flowing through the skin. In this embodiment, the current value (operation detection current) output to the electrode unit 19 when the operation input function is activated is a predetermined current value (constant) that is set in advance. Therefore, according to Ohm's law, the voltage Vo at the output end of the voltage / current conversion circuit 13 is a parameter equivalent to the skin impedance Z.

[0026] Therefore, in this embodiment, a series circuit (voltage divider circuit) of the first resistor element 14 and the second resistor element 15 is provided at the output end of the voltage / current conversion circuit 13, a voltage Vd (see FIG. 2) at a connection point between the first resistor element 14 and the second resistor element 15 is detected, and the skin impedance Z is calculated based on the voltage Vd. Since the resistance values ​​of the first resistor element 14 and the second resistor element 15 are constant, the ratio of the voltage Vd between the two resistor elements and the voltage Vo at the output end of the voltage / current conversion circuit 13 is constant even if the skin impedance Z (voltage Vo) changes. In addition, the ratio of the voltage Vd to the voltage Vo is determined by the resistance value of the first resistor element 14 and that of the second resistor element 15, and is a constant value. Therefore, the voltage Vd at the connection point between the first resistor element 14 and the second resistor element 15 is also a parameter substantially equivalent to the skin impedance Z, similar to the voltage Vo. Therefore, if the voltage Vd can be detected, the skin impedance Z can be calculated using known parameters, that is, the operation detection current, the resistance value of the first resistor element 14, and the resistance value of the second resistor element 15. In this embodiment, if the skin impedance Z (resistance value) is greater than a predetermined threshold value for determining whether the skin is in contact with or not in contact with the skin, it is determined to be in contact with the skin, and if it is equal to or less than the predetermined threshold value, it is determined to be in contact with the skin.

[0027] In this embodiment, the skin impedance Z is measured every time a current is supplied to the electrodes 19a (anodes) selected one by one in a switching cycle (predetermined cycle) by the switching IC 18. Therefore, it is possible to determine the contact / non-contact of the fingers 3 for each electrode 19a on the electrode array unit 190, and to detect the spatial distribution of the contact / non-contact of the fingers 3 on the electrode array unit 190, that is, the contact state of the user's skin. In this embodiment, the presence or absence of an operation input by the user's gripping operation on the electrode unit 19 is determined using information on the spatial distribution of the contact / non-contact of the fingers 3 on the electrode array unit 190 (for example, the number of contact points, which will be described later).

[0028] In this embodiment, even when the tactile sensation providing function is activated, the skin impedance Z (resistance value) is measured based on the voltage Vd at the connection point between the first resistor element 14 and the second resistor element 15. In this case, the measured skin impedance Z is used for feedback control of the stimulation current, etc.

[0029] [Configuration and operation of each part of the control unit] The microcontroller 11 is composed of a microprocessor including a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and functions as an arithmetic processing device and a control device, controlling the operation of each part of the information input device 1.

[0030] When the operation input function is activated, the microcontroller 11 outputs a digital voltage signal for detecting the operation input to the D / A converter 12. At this time, the microcontroller 11 also outputs a switch state changeover control signal to the switching IC 18.

[0031] Furthermore, when the operation input function is activated, the microcontroller 11 calculates the skin impedance Z (resistance value) for each electrode 19a based on the voltage value Vd input from the A / D converter 16. Furthermore, when the operation input function is activated, the microcontroller 11 transmits information on the skin impedance Z (resistance value) obtained for each electrode 19a to the smartphone 2. In addition, the smartphone 2 (processor 2a) that has received the information on the skin impedance Z determines whether the fingers 3 are in contact with each electrode 19a based on the skin impedance Z, that is, determines the spatial distribution (skin contact state) of the contact / non-contact of the fingers 3 on the electrode array unit 190, and thereby determines whether or not a user operation input (a predetermined gripping operation) has been made (see FIG. 10 described later).

[0032] On the other hand, when the tactile sensation provision function is activated, the microcontroller 11 provides a tactile sensation by the electrode unit 19 based on an output command signal for electrical stimulation transmitted from the smartphone 2 as a response signal to an operation input. Specifically, when the tactile sensation provision function is activated, the microcontroller 11 outputs a digital voltage signal for electrical stimulation having a stimulation intensity and stimulation pattern specified by the received output command signal for electrical stimulation to the D / A converter 12. At this time, the microcontroller 11 also outputs a switching control signal for the switch state to the switching IC 18.

[0033] The stimulation patterns include a pattern in which electrical stimulation is applied to the entire electrode area on the electrode array unit 190 in contact with the finger 3 (hereinafter referred to as "whole stimulation"), and a pattern in which electrical stimulation is applied only to the electrode 19a at the edge of the electrode area in contact with the finger 3 (hereinafter referred to as "edge stimulation") (see Figs. 8A and 8B described later). Therefore, the switching control signal of the switch state output to the switching IC 18 when the tactile sense provision function is activated changes according to the type of stimulation pattern. Also, the switching cycle of the switch state when the tactile sense provision function is activated may be the same as or different from that when the operation input function is activated.

[0034] When the operation input function is activated, the D / A converter 12 converts the digital voltage signal for detecting operation input input from the microcontroller 11 into an analog voltage signal for detecting operation input, and outputs the converted analog voltage signal for detecting operation input to the voltage / current conversion circuit 13. When the tactile sensation provision function is activated, the D / A converter 12 converts the digital voltage signal for electrical stimulation input from the microcontroller 11 into an analog voltage signal for electrical stimulation, and outputs the converted analog voltage signal for electrical stimulation to the voltage / current conversion circuit 13.

[0035] When the operation input function is activated, the voltage / current conversion circuit 13 converts the analog voltage signal for detecting operation input input from the D / A converter 12 into an operation detection current, and outputs the converted operation detection current to the switching IC 18. When the tactile sense provision function is activated, the voltage / current conversion circuit 13 converts the analog voltage signal for electrical stimulation input from the D / A converter 12 into a stimulation current, and outputs the converted stimulation current to the switching IC 18. The stimulation current output from the voltage / current conversion circuit 13 is supplied to the electrode 19a selected as the anode by the switching IC 18.

[0036] The series-connected circuit consisting of the first resistive element 14 and the second resistive element 15 is a voltage divider circuit for measuring the skin impedance Z corresponding to the voltage Vo at the output end (the input end of the switching IC 18) of the voltage / current conversion circuit 13. The first resistive element 14 is composed of, for example, a 10 megaohm resistive element, and the second resistive element 15 is composed of, for example, a 100 kiloohm resistive element.

[0037] When the operation input function is activated, the A / D converter 16 converts the voltage value Vd (analog voltage signal) at the connection point between the first resistive element 14 and the second resistive element 15 into a digital voltage signal for detecting the operation input, and outputs the converted digital voltage signal for detecting the operation to the microcontroller 11. When the tactile sense provision function is activated, the A / D converter 16 converts the voltage value Vd (analog voltage signal) at the connection point between the first resistive element 14 and the second resistive element 15 into a digital voltage signal for electrical stimulation, and outputs the converted digital voltage signal for electrical stimulation to the microcontroller 11. The digital voltage signal for electrical stimulation input to the microcontroller 11 is used for feedback control of the stimulation current, etc.

[0038] The boost circuit 17 converts (boosts) the power (voltage) supplied from the smartphone 2 into power required for the operation of the voltage / current conversion circuit 13, and supplies the converted power to the voltage / current conversion circuit 13.

[0039] [Switching IC configuration and operation] Next, an outline of the internal configuration and operation of the switching IC 18 will be described with reference to Fig. 3A and Fig. 3B. Fig. 3A is a diagram showing the on / off operation states (switch states) of the first switch 21 and the second switch 22 in each changeover switch 20 when scanning (selecting) the electrode 19a. Fig. 3B is a diagram showing the state of the current flowing through the user's finger 3 in the switch state shown in Fig. 3A.

[0040] As shown in Fig. 3A, the switching IC 18 is composed of a plurality of changeover switches 20. Specifically, the switching IC 18 is composed of the same number of changeover switches 20 as the number of electrodes 19a included in the electrode unit 19. The plurality of changeover switches 20 are connected in parallel with each other, and one terminal of each changeover switch 20 is connected to the voltage / current conversion circuit 13 (see Fig. 2), and the other terminal is grounded.

[0041] Each changeover switch 20 is configured by connecting a first switch 21 and a second switch 22 in series. A connection point between the first switch 21 and the second switch 22 in each changeover switch 20 is connected to a corresponding one of the electrodes 19a in the electrode section 19. Although not shown in the figure, the switching IC 18 is connected to the electrode section 19 via a connector.

[0042] In the example shown in FIG. 3A, the switching IC 18 is shown when the electrode 19a arranged second from the right in the drawing among the multiple electrodes 19a arranged in the electrode unit 19 is selected as the anode. Therefore, in the example shown in FIG. 3A, the microcontroller 11 controls the first switch 21 in the changeover switch 20 connected to the electrode 19a (anode) arranged second from the right in the drawing to be in the ON state and the second switch 22 to be in the OFF state. In addition, at this time, the first switch 21 in the changeover switch 20 connected to the electrode 19a (cathode) is controlled to be in the OFF state and the second switch 22 to be in the ON state for the other electrodes 19a (cathode). In this case, as shown in FIG. 3B, a current (operation detection current or stimulation current) flows from the electrode 19a (anode) arranged second from the right in the drawing to the other electrodes 19a (cathode) through the skin and subcutaneous tissue of the user's finger 3 (black solid line arrow in the drawing).

[0043] The switching of the anode in the switching IC 18 is controlled by a switching control signal of the switch state input from the microcontroller 11. The switching cycle (predetermined cycle) of the switch state by the switching IC 18 is appropriately set according to the required detection accuracy of the gripping operation, the processing power of the microcontroller, etc., and can be, for example, about several hundred μsec.

[0044] In the present embodiment, an example has been described in which one selected electrode 19a is set as an anode and the other electrodes 19a are set as cathodes in the switching state control by the switching IC 18, but the present invention is not limited to this. For example, in the switching state control by the switching IC 18, one selected electrode 19a may be set as a cathode and the other electrodes 19a may be set as an anode.

[0045] [Electrode array configuration] Fig. 4 is a configuration diagram of the electrode array unit 190 arranged on one long side surface of the smartphone 2. Note that the configuration of the electrode array unit 190 arranged on the other long side surface of the smartphone 2 is substantially the same as the configuration shown in Fig. 4, and therefore a description thereof will be omitted here.

[0046] As shown in FIG. 4, the electrode array portion 190 has a plurality of electrodes 19a arranged in one direction, a plurality of electrode wirings 19b each connected to each electrode 19a, and a flexible substrate 19c on which the plurality of electrodes 19a and the plurality of electrode wirings 19b are mounted.

[0047] The flexible substrate 19c is formed of a substantially T-shaped substrate, and may have, for example, a length of 120 mm in the arrangement direction of the electrodes 19a and a width of 7 mm in the direction perpendicular to the arrangement direction of the mounting area of ​​the electrodes 19a. Each electrode 19a may have, for example, a circular electrode with a diameter of 2 mm, and the interval (electrode pitch) between adjacent electrodes 19a may be, for example, 3.75 mm (see the enlarged view of the area S in FIG. 4). One end of each electrode wiring 19b is connected to the corresponding electrode 19a, and the other end is connected to a connection point between the first switch 21 and the second switch 22 in the corresponding changeover switch 20 in the switching IC 18 via a connector (not shown) (see FIG. 3A).

[0048] When the electrode array unit 190 is attached to the smartphone 2, the electrode array unit 190 is attached to one long side surface of the smartphone 2 so that the arrangement direction of the multiple electrodes 19a is approximately parallel to the longitudinal direction of one long side surface of the smartphone 2 (see FIG. 1A). In addition, the number of electrodes 19a in the electrode array unit 190 attached to the long side surface of the smartphone 2 on which a physical button such as a power button is provided is made smaller than the number of electrodes 19a in the electrode array unit 190 attached to the long side surface on which no physical button is provided (see FIG. 5A described later).

[0049] [Operation input function] Next, the operation contents of the operation input function of the information input device 1 will be described with reference to Fig. 5A, Fig. 5B, and Fig. 6. Fig. 5A is a diagram showing a state of gripping the information input device 1 attached to the smartphone 2 before a user performs a predetermined gripping operation, and Fig. 5B is a diagram showing a state of gripping when the predetermined gripping operation is performed. Fig. 6 is a diagram showing a change in the contact state between the user's finger 3 and the electrode 19a in the electrode array unit 190 before and after the user performs the predetermined gripping operation.

[0050] Here, an example will be described in which the user holds the smartphone 2 in his right hand and detects an operation of firmly gripping the smartphone 2 with his fingers 3 as shown by the white arrow in FIG. 5B (hereinafter referred to as a "grip input operation") as an operation input. In this example, a physical button such as a power button is provided on the long side of the smartphone 2 that the user's middle finger touches. Therefore, the number of electrodes 19a in the electrode array unit 190 attached to the long side of the smartphone 2 that the user's thumb touches is greater than that of the long side of the smartphone 2 that the user's middle finger touches. For example, the number of electrodes 19a in the electrode array unit 190 attached to the long side of the smartphone 2 that the user's thumb touches can be 32, and the number of electrodes 19a in the electrode array unit 190 attached to the opposite long side can be 21.

[0051] 5A and 5B, electrodes 19a (contact electrodes) in contact with fingers 3 are indicated by white squares, and electrodes 19a (non-contact electrodes) in non-contact with fingers 3 are indicated by black squares. Region 31 indicated by dashed line in Fig. 6 indicates the contact region between fingers 3 and electrodes 19a when no grip input operation is being performed, and region 32 indicated by dashed line in Fig. 6 indicates the contact region between fingers 3 and electrodes 19a when a grip input operation is being performed.

[0052] In the non-grip input state shown in FIG. 5A, when a grip input operation is performed by the user and a grip input state shown in FIG. 5B occurs, the skin of the finger 3 is deformed, and the contact area between the finger 3 and the electrode 19a in the electrode array unit 190 becomes large. As a result, the contact area between the finger 3 and the electrode 19a changes from the area 31 shown in FIG. 6 to the area 32, and the contact area becomes large. In this case, the number of electrodes 19a (white square marks in FIG. 5B) that contact the finger 3 in the grip input state becomes larger than that in the non-grip input state. Therefore, the presence or absence of a grip input operation can be detected by counting the number of electrodes 19a that are determined to be in contact with the skin based on the skin impedance Z calculated every time the electrode 19a is scanned. In this embodiment, the process of determining the presence or absence of contact between the electrode 19a and the skin based on the skin impedance Z and the process of counting the number of contact electrodes are executed on software by the smartphone 2 (see FIG. 10 described later).

[0053] Fig. 7 is a diagram showing the change over time in the number of electrodes 19a in contact with the skin of the finger 3 (hereinafter referred to as the "number of contact points") when the user repeatedly performs the non-grip input operation shown in Fig. 5A and the grip input operation shown in Fig. 5B. When the non-grip input operation shown in Fig. 5A and the grip input operation shown in Fig. 5B are repeatedly performed, the number of contact points also repeatedly increases and decreases over time, as shown in Fig. 7. Then, in the operation input function of this embodiment, if the number of contact points exceeds a specific threshold value (dashed line in the figure) set in advance, it is determined that a grip input operation has been performed, and if the number of contact points is equal to or less than the specific threshold, it is determined that a grip input operation has not been performed.

[0054] The criteria for determining whether or not a grip input operation has been performed are not limited to the example shown in FIG. 7, and the following criteria may be used.

[0055] For example, a plurality of specific thresholds for the number of contact points may be provided as a criterion for determining whether or not a grip input operation has been performed. In this case, a plurality of types of grip input operations can be distinguished and determined according to the contact area between the finger 3 and the electrode 19a in the electrode array unit 190, that is, the user's gripping force (amount of skin penetration).

[0056] In addition, in determining whether or not a grip input operation has been performed, for example, it may be determined that a specific grip input operation has been performed if the time during which the number of contact points continuously exceeds a specific threshold is equal to or longer than a predetermined time, rather than just the number of contact points. In this case, multiple types of grip input operations can be distinguished and determined according to a combination of the number of contact points and operation time.

[0057] In addition, in determining whether or not a grip input operation has been performed, the presence or absence of the grip input operation may be determined, for example, according to the spatial pattern of the electrodes 19a in contact with the skin of the fingers 3 (for example, only the index finger and thumb are pressed, etc.). In other words, the presence or absence of the grip input operation may be determined according to the spatial pattern of the gripping state. In this case, a plurality of types of grip input operations can be distinguished and determined according to the spatial pattern of the electrodes 19a in contact with the skin of the fingers 3.

[0058] Furthermore, a specific operation input may be performed by performing a grip input operation multiple times in succession, such as double-clicking a mouse. In this case, multiple types of grip input operations can be distinguished and determined according to the number of times the grip input operation is performed.

[0059] In addition, the above-mentioned various criteria for determining the presence or absence of a grip input operation may be appropriately combined. By using the above-mentioned various criteria for determining whether or not a grip input operation has occurred, it is possible to realize, by the grip input operation of the information input device 1, input operations similar to various touch operations on the display screen of the smartphone 2 and various operations that can be performed with a user interface such as a mouse.

[0060] In addition, in the present embodiment, as described above, an example has been described in which the parameter for determining whether or not a grip input operation has been performed is the number of contact points, but the present invention is not limited to this. Any parameter can be used as the parameter for determining whether or not a grip input operation has been performed, as long as it is a parameter that can distinguish between a grip input state and a non-grip input state. For example, in the grip input state, as described above, the contact area between the finger 3 and the electrode 19a in the electrode array unit 190 is larger than that in the non-grip input state, so that the resistance value between the anode and cathode in the electrode unit 19 in the grip input state is smaller than that in the non-grip input state. Therefore, the resistance value between the anode and cathode in the electrode unit 19, i.e., the skin impedance Z itself, may be used as the parameter for determining whether or not a grip input operation has been performed.

[0061] [How the haptic display function works] Next, with reference to Figures 8A, 8B, 9A and 9B, we will explain the operation of the tactile presentation function of the information input device 1 that is performed in response to a user performing a grip input operation (white arrow in the figure) on the information input device 1.

[0062] FIG. 8A is a diagram showing a contact state between the user's finger 3 and the electrode 19a in the electrode array unit 190 when whole-body stimulation is applied to the user's finger 3 holding the smartphone 2 in response to a grip input operation. FIG. 8B is a diagram showing a contact state between the user's finger 3 and the electrode 19a in the electrode array unit 190 when edge stimulation is applied to the user's finger 3 holding the smartphone 2 in response to a grip input operation. FIG. 9A is a schematic diagram for explaining a tactile sensation given to the user when whole-body stimulation is applied to the user's finger 3. FIG. 9B is a schematic diagram for explaining a tactile sensation given to the user when edge stimulation is applied to the user's finger 3.

[0063] In these drawings, electrodes 19a (stimulating electrodes) that are in contact with the finger 3 and through which a stimulating current flows are indicated by white squares, and electrodes 19a (non-stimulating electrodes) that are in contact with the finger 3 and through which a stimulating current does not flow are indicated by hatched squares. Electrodes 19a (non-contact electrodes) that are not in contact with the finger 3 are indicated by black squares.

[0064] When the tactile sensation providing function provides full stimulation to the finger 3, all of the electrodes 19a on the electrode array unit 190 that are in contact with the finger 3 become stimulation electrodes, as shown in Fig. 8A. On the other hand, when the tactile sensation providing function provides edge stimulation to the finger 3, only the electrodes 19a at the ends of the region of the electrodes 19a that are in contact with the finger 3 become stimulation electrodes, and the other electrodes 19a within that region become non-stimulation electrodes, as shown in Fig. 8B.

[0065] Although detailed description will be omitted here, it has been confirmed in a verification experiment by the present inventors that when electrical stimulation is performed on the user's fingers 3 holding the smartphone 2, the user perceives softness. Furthermore, it has been confirmed in the verification experiment by the present inventors that when whole-body stimulation is performed on the user's fingers 3 holding the smartphone 2, the user feels that the contact part is very soft, and when edge stimulation is performed, the user feels that the contact part is slightly soft.

[0066] Therefore, when whole-body stimulation is performed on the user's fingers 3 holding the smartphone 2, the user feels as if the long side of the smartphone 2 is largely bent (it does not actually bend), and gets a soft sensation, as shown in Fig. 9A. Also, when edge stimulation is performed on the user's fingers 3 holding the smartphone 2, the user feels as if the long side of the smartphone 2 is bent, and gets a soft sensation, as shown in Fig. 9B, although it feels harder than when whole-body stimulation is performed.

[0067] As described above, when a whole-body or edge stimulation is performed in response to a grip input operation, the user feels as if the long side of the smartphone 2 is bending, and thus feels the same sensation as when, for example, pressing a button, and can be confident that the operation input has been performed. In addition, in this embodiment, a plurality of types of grip input operations are provided, and it is possible to change the content of tactile information provided to the user according to the type of grip input operation, so that the user can also confirm the type of grip input operation by the presented tactile information.

[0068] In addition, as described above, the tactile sensation providing function of the present embodiment can provide softness as tactile information, so that, for example, when an operation is performed to select an item displayed on the display screen of the smartphone 2 by a grip input operation, tactile information corresponding to the softness of the item can be output as a response signal. In this case, the feel of the item can be provided to the user in response to the grip input operation.

[0069] In the above description, the tactile presentation function of the present embodiment is operated in response to the user's grip input operation (operation input function) on the information input device 1 (electrode unit 19) as an example, but the tactile presentation function may also be operated independently of the user's grip input operation. For example, the tactile presentation function of the present embodiment is also operated when tactile information corresponding to the content of an application being played on the smartphone 2 is unilaterally presented to the user.

[0070] [Processing flow for detecting grip input operations] Next, the contents of the detection process of the grip input operation performed by the information input device 1 and the smartphone 2 to which the information input device 1 is attached will be specifically described with reference to Fig. 10. Fig. 10 is a flowchart showing the procedure of the detection process of the grip input operation.

[0071] 10 is performed when the operation input function of the information input device 1 is set to be operable in the smartphone 2 (for example, when application software that can operate the operation input function of the information input device 1 is running). The processing steps represented by rectangular solid line blocks in FIG. 10 are processes executed by the information input device 1, and the processing steps represented by rectangular dashed line blocks are processes executed on the software by the smartphone 2.

[0072] First, the information input device 1 outputs an operation detection current and a switch state control signal to the electrode unit 19 (step S1). In this process, the voltage / current conversion circuit 13 outputs the operation detection current to the switching IC 18, and the microcontroller 11 outputs a switch state switching control signal to the switching IC 18. Through this process, one electrode 19a is selected as an anode from among the multiple electrodes 19a in the electrode unit 19, and an operation detection current is output to the selected electrode 19a.

[0073] Next, the information input device 1 (microcontroller 11) calculates the skin impedance Z (step S2). In this process, the microcontroller 11 calculates the skin impedance Z at the currently selected electrode 19a (anode) based on the voltage value input from the A / D converter 16.

[0074] Next, the information input device 1 (microcontroller 11) judges whether or not one cycle of the electrodes 19a has been scanned (step S3). In this embodiment, as described above, when the operation input function or the tactile sense providing function is activated, the electrodes 19a are sequentially scanned and selected one by one in a switching cycle of the switch state for the multiple electrodes 19a provided in the electrode unit 19, and a current is output to the selected electrode 19a (anode). Then, when the output of the current to the last selected electrode 19a is completed, the scanning of the electrodes 19a for one cycle is completed, and then, the selection and current supply for each electrode 19a are repeated, returning to the first electrode 19a. Therefore, in the process of step S3, if the currently selected electrode 19a (anode) is the last electrode to be selected, the information input device 1 judges that one cycle of the electrodes 19a has been scanned, and the result of the judgment process of step S3 is a YES judgment.

[0075] In step S3, if the information input device 1 determines that one period of the electrodes 19a has not been scanned (if step S3 is determined to be NO), the information input device 1 returns the process to step S1 and repeats the processes from step S1 onwards. At this time, the information input device 1 executes the processes from step S1 onwards for the unselected electrodes 19a. The processes from step S1 to S3 are repeated at a switching cycle (for example, about several hundred μsec).

[0076] On the other hand, in step S3, if the information input device 1 determines that one cycle of the electrodes 19a has been scanned (if step S3 is determined as YES), the information input device 1 (microcontroller 11) transmits information on the skin impedance Z calculated for each electrode 19a (information on the user's gripping state) to the smartphone 2 as an operation input signal (step S4). In this embodiment, an example in which information on the skin impedance Z of the electrodes 19a for one cycle is transmitted collectively will be described, but the present invention is not limited to this. For example, each time the skin impedance Z of each electrode 19a is calculated, the information on the skin impedance Z of the electrode 19a may be transmitted to the smartphone 2. In this case, after the process of step S2, a process of transmitting the information on the skin impedance Z is performed.

[0077] Next, the smartphone 2 (processor 2a) that has received the information on the skin impedance Z calculated for each electrode 19a determines whether or not the skin of the user's finger 3 is in contact with each electrode 19a (step S5). In this process, the smartphone 2 determines that there is no contact if the skin impedance Z (resistance value) is greater than a preset threshold value, and determines that there is contact if the skin impedance Z is equal to or less than the threshold value.

[0078] Next, the smartphone 2 (processor 2a) performs a process of counting the number of contact points (step S6). In this process, the smartphone 2 counts the number of electrodes 19a (the number of contact points) that have been determined to be in contact with the skin of the user's fingers 3 in the process of step S5.

[0079] Next, the smartphone 2 (processor 2a) determines whether or not a grip input operation has been performed (step S7). In this process, if the number of contact points counted in the process of step S6 exceeds a specific threshold set in advance, the smartphone 2 determines that a grip input operation has been performed, and if the number is equal to or less than the specific threshold, determines that a grip input operation has not been performed.

[0080] In step S7, when the smartphone 2 determines that there is no grip input operation (when the determination in step S7 is NO), the smartphone 2 (processor 2a) returns the process to the process in step S1.

[0081] On the other hand, if the smartphone 2 determines in step S7 that a grip input operation has been performed (if step S7 is determined as YES), the smartphone 2 (processor 2a) performs processing corresponding to the grip input operation, generates a response signal to the grip input operation, and transmits the generated response signal to the information input device 1 (step S8). In this process, the smartphone 2 transmits an output command signal for electrical stimulation as a response signal to the grip input operation. After the process of step S8, the process returns to the process of step S1, and the processes from step S1 onwards are repeated.

[0082] [Processing flow for response processing of grip input operations] Next, the contents of the grip input operation response process performed by the information input device 1 will be specifically described with reference to Fig. 11. Fig. 11 is a flowchart showing the procedure of the grip input operation response process. Note that the process flow shown in Fig. 11 is performed when the operation input function of the information input device 1 is set to be operable in the smartphone 2.

[0083] First, the information input device 1 (microcontroller 11) determines whether or not there is a response from the smartphone 2 to the grip input operation (step S11). In this process, the microcontroller 11 determines whether or not an output command signal for electrical stimulation has been received from the smartphone 2 as a response signal (response command) to the grip input operation, and if an output command signal for electrical stimulation has been received, the result of the determination process in step S11 is a YES determination.

[0084] In step S11, if the microcontroller 11 determines that there is no response from the smartphone 2 (if the determination in step S11 is NO), the information input device 1 (microcontroller 11) repeats the process of step S11.

[0085] On the other hand, in step S11, if the microcontroller 11 determines that there is a response from the smartphone 2 (if step S11 is determined as YES), the information input device 1 outputs a stimulation current and a switch state control signal to the electrode unit 19 (step S12). In this process, the voltage / current conversion circuit 13 outputs a stimulation current to the switching IC 18, and the microcontroller 11 outputs a switch state switching control signal to the switching IC 18. At this time, the output control of the stimulation current and the switching control of the switch state are performed according to the stimulation intensity and stimulation pattern specified by the output command signal for the electrical stimulation. As a result, tactile information (electrical stimulation) specified by the output command signal for the electrical stimulation is presented to the user's finger 3. Then, after the process of step S12, the information input device 1 returns the process to the process of step S11 and repeats the processes from step S11 onwards.

[0086] [Various effects and application examples] As described above, in this embodiment, first, the information input device 1 transmits information on the skin impedance of each electrode 19a to the smartphone 2 (mobile device) as information on the user's gripping state with respect to the electrode array unit 190. Next, the smartphone 2 that has received the skin impedance information determines whether or not a grip input operation (predetermined grip operation) is being performed on the electrode array unit 190 based on the skin impedance information. If it is determined that a grip input operation is being performed, the smartphone 2 performs a process (operation) corresponding to the grip input operation and transmits an output command signal for electrical stimulation to the information input device 1 as a response signal to the grip input operation. As a result, the information input device 1 that has received the output command signal performs electrical stimulation (tactile presentation) on the user's fingers 3 according to the stimulation intensity and stimulation pattern specified by the output command signal. Therefore, in the information input device 1 of this embodiment, the linkage between the function of the information input device 1 (the tactile presentation function using the electrode array unit 190) and the function of the smartphone 2 can be improved.

[0087] Furthermore, the information input device 1 of this embodiment can be applied in various ways as described below, and various effects described below are expected in each application example.

[0088] As described above, the information input device 1 of this embodiment can present the user with a simulated tactile sensation of softness in response to the user's grip input operation. Therefore, for example, when the user selects an item (e.g., clothing, etc.) on an EC (Electronic Commerce) site displayed on the smartphone 2 by a grip input operation, the simulated softness (feeling of touch) of the item is presented to the user as response information. Currently, on EC sites, the size of an item is easily understood by using AR (Augmented Reality) display technology, but further, by applying the above technology of the information input device 1 of this embodiment, information such as the feel of the item can also be obtained, which can further stimulate the use of EC sites.

[0089] The information input device 1 of the present embodiment can be applied to applications such as control and effect expression of a game played on the smartphone 2. In this case, for example, by making some of the game operation commands function with a grip input operation, a feeling of softness can be presented in a simulated manner as a game effect at that time.

[0090] In the information input device 1 of the present embodiment, the operation input function can detect the state in which the smartphone 2 is being held by the user, and the information input device 1 can be applied to, for example, a function for preventing the smartphone 2 from falling from the user's hand. In this case, for example, if the number of contact points becomes equal to or less than a specified threshold value by the operation input function, it can be determined that the smartphone 2 is about to fall, and an alarm can be generated from the smartphone 2. Therefore, in such an application example, the smartphone 2 can be prevented from falling immediately before it falls, and damage to the smartphone 2 can be prevented.

[0091] As described above, the information input device 1 of this embodiment can distinguish and determine a plurality of types of grip input operations according to the strength of the force (skin penetration) with which the smartphone 2 is gripped. Therefore, for example, in a website displayed on the display screen of the smartphone 2, when the force (grip force) of the grip input operation is relatively weak, a simplified menu can be selected and displayed, and when the force of the grip input operation is strong, the entire menu can be selected and displayed. In this case, faster access is possible than when the menu is displayed by a conventional touch operation or slide operation on the smartphone 2.

[0092] Furthermore, as described above, the information input device 1 of this embodiment can determine the presence or absence of a grip input operation, for example, according to the spatial pattern of the grip state (for example, pressing only the index finger and thumb, etc.). Therefore, in this case, the presence or absence of a grip input operation can be determined with a relatively small gripping force, so that the configuration can be made with consideration given to children and women who have a relatively weak gripping force.

[0093] Furthermore, in the information input device 1 of this embodiment, as described above, the presence or absence of a specific grip input operation can be determined according to the grip input operation pattern (for example, the spatial pattern of the gripping state, the number of consecutive grip input operations, etc.). In this case, it is also possible to apply the grip input operation pattern to the smartphone unlocking function. In this case, for example, the smartphone 2 on a desk or in a pocket, or the smartphone 2 hanging from a neck, can be unlocked simply by gripping it in a predetermined pattern, and the smartphone can be started up more quickly.

[0094] [Various variations] In the above embodiment, the smartphone 2 (processor 2a) performs the processes of determining whether or not the electrode 19a is in contact with the finger 3, counting the number of contact electrodes, and determining whether or not a grip input operation has been performed, based on the information on the skin impedance Z detected by the information input device 1, that is, steps S5 to S7 in FIG. 10, but the present invention is not limited to this. These processes may be performed by the information input device 1 (microcontroller 11). In this case, the information on the user's gripping state (operation input signal) transmitted from the information input device 1 to the smartphone 2 is not information on the skin impedance Z, but is a determination result on whether or not a grip input operation has been performed.

[0095] In the above embodiment, the information input device 1 is attached to the smartphone 2, but the present invention is not limited to this. The information input device 1 of the above embodiment can be applied to any portable device as long as the portable device can be held by a user, and the same effects can be obtained.

[0096] In the above embodiment, the information input device 1 and the smartphone 2 are provided separately, but the present invention is not limited to this. For example, the information input device 1 and the smartphone 2 may be integrated. That is, the smartphone 2 may be configured to include the above-mentioned components and functions of the information input device 1.

[0097] In the above embodiment, an example has been described in which a tactile sensation is presented by electrical stimulation in response to a grip input operation on the smartphone 2, but the present invention is not limited thereto. For example, a tactile sensation presented by electrical stimulation and a tactile sensation presented by vibration may be combined in response to a grip input operation. In this case, the types of tactile information that can be given to a user in response to a grip input operation are increased, so that the types of grip input operations that can be handled can be increased, and it is possible to present a tactile sensation that is detailed and rich in variety.

[0098] In the above embodiment, an example of a configuration in which the electrode array unit 190 is attached to each of the two long side surfaces of the smartphone 2 has been described, but the present invention is not limited to this, and the electrode array unit 190 may be attached to only one of the long side surfaces of the smartphone 2. Furthermore, in the above embodiment, the information input device 1 is configured to have an operation input function and a tactile presentation function, but the present invention is not limited to this, and the information input device 1 may be configured to have only an operation input function.

[0099] In addition, the above-mentioned embodiment is a detailed and specific description of the configuration of the device in order to easily explain the present invention, and is not necessarily limited to the configuration having all of the described components. In addition, the present invention may have various other applications and modifications without departing from the gist of the present invention described in the claims. [Explanation of symbols]

[0100] 1...information input device, 2...smartphone, 2a...processor, 3...finger, 4...USB cable, 10...control unit, 11...microcontroller, 12...D / A converter, 13...voltage / current conversion circuit, 14...first resistive element, 15...second resistive element, 16...A / D converter, 17...booster circuit, 18...switching IC, 19...electrode unit, 19a...electrode, 190...electrode array unit

Claims

1. an electrode array unit that can be attached to a side portion of the portable device that is held by a user and includes a plurality of electrodes arranged in a line; A switching unit capable of sequentially selecting one electrode from the plurality of electrodes; a control unit that, with the electrode array unit attached to the side surface of the portable device, outputs a current to the electrode selected by the switching unit to detect a gripping state of the portable device by the user, and transmits information regarding the detected gripping state of the user to the portable device as an operation input signal. Information input device.

2. the two electrode array units are attachable to the side surfaces of the portable device that are held by a user, When the control unit receives a response signal to the operation input signal from the portable device, the control unit outputs a stimulation current to the electrode via the switching unit to present tactile information to the user by electrical stimulation.

2. The information input device according to claim 1.

3. A determination is made as to whether or not the user is performing a predetermined gripping operation on the portable device based on information regarding each impedance of the output destination of the current detected when the control unit outputs the current to each of the plurality of electrodes.

3. The information input device according to claim 2.

4. In the determination of the predetermined gripping operation, it is determined whether or not the portable device is gripped with a predetermined gripping force.

4. The information input device according to claim 3.

5. The determination of the predetermined gripping motion includes determining whether the spatial pattern of the electrodes in contact with the user's skin is a predetermined pattern.

4. The information input device according to claim 3.

6. The tactile information provided to the user is tactile information indicating softness.

3. The information input device according to claim 2.

7. an information input device including an electrode array unit that can be attached to a side portion of a portable device that is held by a user and that includes a plurality of electrodes arranged in a row, a switching unit that can sequentially select one electrode from the plurality of electrodes, and a control unit, the control unit detecting a holding state of the portable device by the user by outputting a current to the electrode selected by the switching unit with the electrode array unit attached to the side portion of the portable device; The control unit transmits information regarding the user's gripping state to the portable device as an operation input signal. How to enter information.

Images