User interface apparatus and method of controlling haptic cues

Abstract

The invention relates to a user interface apparatus and a method of controlling a haptic cue. The user interface device includes: a knob device having a mechanical structure that prompts a tactile stimulus in response to each predetermined amount of knob operation; a panel, wherein the knob equipment is arranged on the panel; a controller; and a vibration generator configured to vibrate the knob device, the vibration generator being controllable by the controller. The controller controls the vibration generator to vibrate the knob device in synchronization with the predetermined amount of knob operation such that the knob device prompts tactile stimuli other than tactile stimuli of the mechanical structure.

Description

Technical Field

[0001] The present invention relates to tactile cues. Background Art

[0002] In recent years, touch panels have replaced mechanical switches or buttons in various devices. However, touch panels do not have uneven surfaces like mechanical switches or buttons; users must visually check the process from finding the position to be operated to completing the operation. Although touch panels with tactile cue functions have been developed to solve this problem, they are not general-purpose due to their limited usage environment.

[0003] In view of the above, a device has been proposed that has various functions of a touch panel with a tactile cue function while providing certainty through knob operation. Such a device is expected to be used for devices to be operated in parallel with other operations, such as vehicle components operated while driving, test equipment in the medical field, and devices used in the video / audio field. Summary of the Invention

[0004] A technique is needed to achieve diversity in tactile stimuli to be prompted from an interface operated by a user.

[0005] A user interface device according to an aspect of the present invention includes: a knob device having a mechanical structure that prompts a tactile stimulus in response to each predetermined amount of knob operation; a panel on which the knob device is mounted; a controller; and a vibration generator configured to vibrate the knob device, the vibration generator being controllable by the controller. The controller controls the vibration generator to vibrate the knob device in synchronization with the predetermined amount of knob operation so that the knob device prompts a tactile stimulus in addition to the tactile stimulus of the mechanical structure.

[0006] An aspect of the present invention is a method of controlling tactile cues of a user interface device, the user interface device including: a knob device having a mechanical structure that prompts a tactile stimulus in response to each predetermined amount of knob operation; a panel on which the knob device is mounted; and a vibration generator configured to vibrate the knob device. The method includes controlling the vibration generator to vibrate the knob device in synchronization with the predetermined amount of knob operation so that the knob device prompts a tactile stimulus in addition to the tactile stimulus of the mechanical structure.

[0007] An aspect of the present invention achieves diversity in tactile stimuli presented to a user.

[0008] It should be understood that the above general description and the following detailed description are both exemplary and explanatory and do not limit the present invention. Brief Description of the Drawings

[0009] Figure 1 Shows a configuration example of a user interface device in an embodiment of the present invention.

[0010] Figure 2 Schematically shows an example of adjusting the air volume setting of an air conditioner through a user interface device.

[0011] Figure 3 Schematically shows a configuration example of a knob device.

[0012] Figure 4 Schematically shows a configuration example of a knob device.

[0013] Figure 5 Is a diagram for explaining a method of determining the rotation angle of a rotary knob member based on detection points of two conductor sheets provided on the lower surface of the rotary knob member.

[0014] Figure 6 Schematically shows a configuration example of a part of a user interface device.

[0015] Figure 7 Is a diagram for explaining an example of the behavior of a user interface device when the rotary knob member rotates at a constant angular velocity ω.

[0016] Figure 8 Shows an example of the relationship between mechanical vibration pulses and rotation event pulses in a knob device.

[0017] Figure 9 Shows an example of an actuator drive signal.

[0018] Figure 10 Is a flowchart of a processing example of a user interface device.

[0019] Figure 11 Shows the operation of adjusting the temperature setting of an air conditioner through a user interface device and the resulting change in the image displayed on the user interface device.

[0020] Figure 12 Is for explaining with reference to Figure 11 Described the behavior of the user interface device.

[0021] Figure 13 Shows for performing with reference to Figure 11 And Figure 12 Described the logical configuration example of the process of reducing the temperature setting of the air conditioner.

[0022] Figure 14 Shows the operation of adjusting the air volume setting of an air conditioner through a user interface device and the resulting change in the image displayed on the user interface device.

[0023] Figure 15 is a diagram for explaining the behavior of the user interface device described with reference to Figure 14 the description.

[0024] Figure 16 shows an example of the logical configuration for performing the process of increasing the air volume setting of an air conditioner described with reference to Figure 14 and Figure 15 the description. Detailed Description

[0025] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that these embodiments are merely examples for implementing the present invention and do not limit the technical scope of the present invention.

[0026] One embodiment of the present invention applies programmable vibration to a knob device mounted on a panel, and the knob device has a structure for prompting a tactile stimulus to add a tactile stimulus. This configuration provides a variety of tactile stimuli to the fingers of the user who operates the knob, thereby realizing tactile augmented reality (AR).

[0027] Figure 1 shows an example of the configuration of the user interface device in one embodiment of the present invention. The user interface device 1 can be mounted on an automotive dashboard. Figure 1 The user interface device 1 in the configuration example of Figure 1 enables the user to operate a radio, a map, a phone, a music player, and an air conditioner.

[0028] The user interface device 1 includes a touch-sensing display panel 10 and a mechanical knob device 15 fixed to the touch-sensing display panel 10. Examples of the touch-sensing display panel 10 include a touch panel and a display panel stacked on top of each other. The knob device 15 is fixed on the viewing side (front side) of the touch panel, and the display panel is provided behind the touch panel. In addition, a lateral actuator ( Figure 1 not shown in

[0029] In Figure 1 the configuration example, the lateral actuator vibrates along the Y axis. The number of actuators, the position of the actuators, the direction of vibration, and the manner of vibration are not limited as long as the actuator can prompt a tactile stimulus in response to the user's knob operation. The actuator can vibrate one or both of the position and form of the touch-sensing display panel 10, or only vibrate the knob device 15 for tactile prompting. In addition, a vibration generator different from the actuator can be adopted.

[0030] In Figure 1In this case, the horizontal axis in the plane of the paper is the X-axis, and the vertical axis in the plane of the paper is the Y-axis. The X-axis and the Y-axis are perpendicular to each other. The Z-axis is perpendicular to the X-axis and the Y-axis; in other words, the Z-axis is perpendicular to the plane defined by the X-axis and the Y-axis. The user views the image displayed on the user interface device 1 along the Z-axis.

[0031] The touch-sensing display panel 10 displays a predetermined object image in response to the operation of the user. Figure 1 In the example of, the touch-sensing display panel 10 displays button images 11A to 11D. The images 11A to 11D correspond to buttons for selecting listening to the radio, making a call, displaying a map, and listening to music. A tactile stimulus may or may not be prompted in response to a touch on one of the button images.

[0032] The touch-sensing display panel 10 also displays the person images 14A and 14B on the left and right seats and the temperatures 13A and 13B of the air-conditioning settings. In response to a touch on the image of the temperature 13A or 13B, the temperature setting function of the left or right seat is assigned to the knob device 15. In Figure 1 the example of, the temperature setting of the left seat is selected.

[0033] The knob device 15 includes a rotatable knob member; for example, the user can rotate the knob member with a finger 20. In Figure 1 the configuration example of, the knob member can be freely rotated clockwise and counterclockwise. In other words, the knob member can continuously maintain a clockwise or counterclockwise rotation. As will be described later, the knob device 15 has a structure that provides a tactile click at each predetermined rotation angle (rotation angle unit).

[0034] In Figure 1 the example of, when the display temperature 13A of the left seat is tapped with an indicator such as a finger, the knob device 15 is assigned the use of setting the temperature of the left seat to the air conditioner. For example, with each tactile click generated by the knob device 15, the temperature to be set changes by 1°C. Counterclockwise rotation of the knob device 15 causes each tactile click generated by the knob device 15 to lower the temperature by 1°C, and clockwise rotation causes each tactile click generated by the knob device 15 to raise the temperature by 1°C. The controllable temperature range is predetermined. When the knob member maintains a counterclockwise rotation, the temperature stops changing at the predetermined minimum temperature. When the knob member maintains a clockwise rotation, the temperature stops changing at the predetermined maximum temperature.

[0035] Figure 2 An example of adjusting the air volume setting of the air conditioner through the user interface device 1 is schematically shown. When the left person image 14A is tapped with an indicator such as a finger, the knob device 15 is assigned the use of setting the air volume of the left seat to the air conditioner. The object image 17 represents the current air volume.

[0036] For example, for each haptic click generated by the knob device 15, the air volume to be set changes by one level. Counterclockwise rotation of the knob device 15 causes each haptic click generated by the knob device 15 to decrease the air volume by one level, and clockwise rotation causes each haptic click generated by the knob device 15 to increase the air volume by one level. The controllable air volume range is predetermined. When the knob member is kept rotating counterclockwise, the air volume stops changing at the predetermined minimum level (e.g., 0). When the knob member is kept rotating clockwise, the air volume stops changing at the predetermined maximum level.

[0037] In addition to the function of operating the air conditioner as described above, the knob device 15 can also be assigned various other functions, such as the function of adjusting the volume when listening to music or making a call and the function of selecting an item from a list.

[0038] For a knob device that provides haptic clicks through a mechanical structure, various configurations are known. Figure 3 and Figure 4 An exemplary configuration of the knob device 15 is schematically shown. Figure 3 and Figure 4 An exemplary configuration of the dial-type knob device 15 is shown in. Figure 3 shows a cross-section of the knob device 15 observed along the Y axis, Figure 4 shows a cross-section of a part of the knob device 15 observed along the Z axis.

[0039] The features of the present invention are applicable to other types of knob devices, such as a slide-type knob device. For example, the dial-type knob device 15 includes a knob member that can rotate on a fixed base member, and the slide-type knob device includes a slide knob member that can linearly slide along the fixed base member. The slide knob member provides mechanical haptic clicks (haptic stimuli) at each predetermined displacement amount in one direction. The structure of the knob device is not limited as long as it has a mechanism for providing haptic clicks. The movable range of the knob member is not particularly limited.

[0040] Referring to Figure 3 the dial-type knob device 15 includes a base member 160 and a rotary knob member 150. The base member 160 is an inner race member, and the rotary knob member 150 is an outer race member. The base member 160 is fixed to the mounting surface of the touch-sensing display panel 10, for example, by an adhesive. The dial-type knob device 15 has a bearing structure that includes balls 158; however, the structure for making the knob member movable in the knob device 15 is not particularly limited.

[0041] The rotary knob component 150 includes a plurality of conductor sheets 151, which are circumferentially spaced apart from each other on its lower surface or on the front surface facing the touch-sensing display panel 10. The user interface device 1 can detect the position of the conductor sheet 151 in contact with the surface of the touch-sensing display panel 10.

[0042] Referring Figure 4 , the rotary knob component 150 has a plurality of recesses 152, which are equidistantly arranged from each other on its inner peripheral wall. The outer peripheral end and the inner peripheral end of the rotary knob component 150 are circular. The base component 160 includes a spring member 163. The spring member 163 has a protrusion 161 on its outer wall or on the surface facing the inner wall of the rotary knob component 150. The spring member 163 has elasticity along the X-axis and presses the protrusion 161 against the area between the recesses 152.

[0043] When the rotary knob component 150 rotates, the protrusion 161 enters and exits the recesses 152 one by one to provide a tactile click to the finger operating the rotary knob component 150. As noted from this description, the tactile click generated by the knob device 15 can be a tactile stimulus generated by storing strain energy in the elastic member and releasing the strain energy from the elastic member.

[0044] The user interface device 1 detects the rotation angle of the rotary knob component 150 and controls the image to be displayed according to the detected angle. The user interface device 1 also generates a signal for controlling other electronic components (such as an air conditioner or a media player) according to the detected angle. The rotation angle of the rotary knob component 150 can be determined according to the change in the position of the plurality of conductor sheets 151 embedded in the rotary knob component 150.

[0045] Figure 5 is a diagram for explaining a method of determining the rotation angle of the rotary knob component 150 according to the detection points of two conductor sheets 151 provided on the lower surface of the rotary knob component 150. Figure 5 The coordinates on the surface of the touch-sensing display panel 10 and the positions of the rotary knob component 150 and the conductor sheets 151 embedded therein in the coordinate plane are provided.

[0046] In Figure 5 's example, the rotary knob component 150 includes two conductor sheets 151, namely conductor sheet A 151A and conductor sheet B 151B. The conductor sheet A 151A and the conductor sheet B 151B are arranged to be spaced 180 degrees from each other on the circumference of the rotary knob component 150 around the rotation center 156. The user interface device 1 determines the rotation angle of the rotary knob component 150 according to the positions of the conductor sheet A 151A and the conductor sheet B 151B.

[0047] When determining the coordinates (x A , y A ) of the conductor sheet A 151A and the coordinates (x B , y B ) of the conductor sheet B 151B through the touch-sensing display panel 10, the coordinates (x C , y C ) of the rotation center of the rotary knob member 150 can be obtained as follows:

[0048] (x A , y A ) = (5, 3) (x B , y B ) = (5, 7)

[0049]

[0050] The angle at which the line connecting the conductor sheet A 151A and the conductor sheet B 151B intersects the horizontal line along the X-axis passing through the rotation center at the rotation center 156 of the rotary knob member 150 can be obtained by the following formula:

[0051]

[0052] The rotation angle of the rotary knob member 150 can also be determined based on the positions of three or more conductor sheets. For example, assume that the rotary knob member 150 has three conductor sheets A, B, and C. These conductor sheets are arranged at intervals from each other on the circumference around the rotation center of the rotary knob member 150.

[0053] Let (X A , Y A ), (X B , Y B ) and (X C , Y C ) be the initial coordinates of the conductor sheet A, the conductor sheet B, and the conductor sheet C respectively, and (X A ’, Y A ’), (X B ’, Y B ’) and (X C , Y C ’) be the coordinates of the conductor sheet A, the conductor sheet B, and the conductor sheet C after rotation respectively. The rotation angle of the rotary knob member 150 can be obtained by one of the following formulas based on the initial coordinates and the coordinates after rotation of the conductor sheets A, B, and C:

[0054]

[0055] where, atan2(Y B - YA , X B -X A ) represents the angle of the line connecting the initial positions of conductor sheets A and B with respect to the X-axis, atan2(Y C -Y A , X C -X A ) represents the angle of the line connecting the initial positions of conductor sheets A and C with respect to the X-axis, atan2(Y B ’ - Y A ’, X B ’ - X A ) represents the angle of the line connecting the rotated positions of conductor sheets A and B with respect to the X-axis, atan2(Y C ’ - Y A ’, X C ’ - X A ) represents the angle of the line connecting the rotated positions of conductor sheets A and C with respect to the X-axis. Each of the above two formulas provides the rotation angle For example, even if one of the three conductor sheets A, B, and C, namely conductor sheet C, cannot be located for some reason and the coordinates X C and Y C are not determined, as long as the remaining conductor sheets A and B can be located, the rotation angle This rotation angle represents the rotation angle of the rotary knob member 150.

[0056] Compared with using two conductor sheets, using three conductor sheets arranged asymmetrically can improve the accuracy of determining rotations of approximately 90 degrees and 270 degrees.

[0057] As described above, the method using two conductor sheets estimates the rotation angle based on the inclination of the line connecting two points. The method using three conductor sheets estimates the rotation angle based on the inclinations of two lines connecting two of the three points. Since more information can be used to estimate the rotation angle, the method using three conductor sheets is expected to achieve higher accuracy than the method using two conductor sheets.

[0058] More specifically, an increase in the number of lines means an increase in clues, thereby improving accuracy. It is desirable that the three conductor sheets be arranged at the vertices of a non - equilateral triangle rather than at symmetric positions such as the vertices of an equilateral triangle. Compared with a symmetric arrangement, an asymmetric arrangement makes the difference in the distances between the two lines obvious, thus facilitating the detection of changes in the relative positions.

[0059] To estimate the rotation angle, the user interface device 1 identifies each conductor sheet and detects their positions. For example, the user interface device 1 assigns IDs to the conductor sheets and determines the initial positions of the conductor sheets in the initial setup. The user interface device 1 tracks the positions of the identified conductor sheets to determine their current positions. Well-known multi-touch detection functions can be applied to locate the conductor sheets.

[0060] Meanwhile, by predetermining the positional relationships of the multiple conductor sheets in the rotary knob member 150, the rotation angle for the next tactile click can be calculated based on the current positions of the multiple conductor sheets according to the mechanical structure. For example, Figure 4 the positional relationships among the shown protrusions 161, the multiple recesses 152, and the multiple conductor sheets 151 are predetermined. Each conductor sheet can be identified by detecting the touch waveform unique to the conductor sheet with one or more conductor sheets.

[0061] Hereinafter, a method for tactile prompting of the user interface device 1 is described. In addition to the mechanical tactile clicks prompted by the knob device 15, the user interface device 1 in an embodiment of this specification also provides diversity in the tactile feedback provided to the user by applying programmable vibrations to the rotary knob member 150. The configuration example described below provides tactile feedback to the user by vibrating the touch-sensing display panel 10 through the rotary knob member 150 by actuating the actuator, but the manner of adding tactile stimuli is not particularly limited.

[0062] Figure 6 A configuration example of a part of the user interface device 1 is schematically shown. The user interface device 1 includes a cover glass 103 of the touch-sensing display panel 10 and a knob device 15 mounted on the surface of the cover glass 103. Figure 6 The rotary knob member 150 of the knob device 15 is shown. The surface of the cover glass 103 is the mounting surface of the knob device 15. The actuator 105 is attached to the surface of the cover glass 103 opposite to the mounting surface of the knob device 15. The actuator 105 vibrates, for example, in the normal direction of the mounting surface of the knob device 15 or in the in-plane direction.

[0063] In an embodiment of this specification, the user interface device 1 vibrates the cover glass 103 on which the knob device 15 is mounted through the electro-controlled actuator 105 to superimpose the electro-controlled vibration of the knob device 15 on the mechanical vibration (tactile click) of the knob device 15. The user interface device 1 obtains diversity in the tactile feedback to the user by synchronizing the programmable vibration applied from the outside with the mechanical vibration of the knob device 15 and realizes tactile augmented reality (AR).

[0064] In one embodiment of this specification, the knob device 15 provides a periodic tactile stimulus (a tactile click prompted by its mechanical structure) when the rotary knob member 150 rotates. Specifically, when the rotary knob member 150 rotates a predetermined angle from the position where the previous tactile click was prompted, it prompts the next tactile click. The angle (the amount of rotation) between adjacent positions that each prompt a tactile click is consistent, but the angle does not have to be uniform.

[0065] The user interface device 1 controls the actuator 105 to vibrate in synchronization with the periodic vibration (repeated tactile clicks) generated by the structure of the knob device 15. Since the tactile clicks automatically provided by the knob device 15 are generated by the mechanical structure of the knob device 15, these tactile clicks cannot be electrically programmed. However, since the programmable vibration from the outside is synchronized with the tactile clicks, the user feels as if the feeling of the tactile clicks has changed seamlessly.

[0066] The tactile clicks generated by the mechanical structure of the knob device 15 change the shear force f acting on the fingertip in the rotational direction t . For example, the programmable vibration applied from the outside is a vibration in the normal direction of the surface on which the knob device 15 is installed or in a direction within its plane. In Figure 6 , the normal direction is the Z-axis direction.

[0067] The Pacinian corpuscles of the mechanoreceptors for detecting vibration cannot distinguish the vibration direction. Therefore, the user feels as if the shear force f acting on the fingertip in the rotational direction t has changed. Therefore, the user interface device 1 can make the user feel as if the tactile clicks generated by the mechanical structure of the knob device 15 have changed.

[0068] The rotary knob member 150 in one embodiment of this specification is designed to be able to rotate continuously. This configuration allows for diverse uses. For example, when the temperature setting has reached the upper limit, the user interface device 1 can apply a programmable external vibration in synchronization with the vibration of the mechanical structure. Then, even if the user does not look at the user interface device 1, the user will know this fact without feeling uncomfortable.

[0069] The rotary knob member 150 can be provided with an electrically controlled vibration through a device different from the actuator 105. For example, by controlling the signal supplied to the electrodes in the touch-sensing display panel 10, the reaction force supplied from the electrode to the conductive sheet of the rotary knob member 150 can be controlled. Like this example, various vibration generators can apply a programmable vibration to the rotary knob member 150.

[0070] Figure 7It is a diagram for explaining an example of the behavior of the user interface device 1 when the rotary knob member 150 rotates at a constant angular velocity ω. The curve graph 310 shows the shear force f t in the direction tangent to the circumference, and this shear force f t is caused by the mechanical vibration generated by the rotation of the rotary knob member 150 at the angular velocity ω and acts on the fingertip. The periodically appearing pulses 311 indicate that the shear force f greater than 0 t acts on the fingertip periodically. The shear force f t is generated each time the rotary knob member 150 rotates . The width of the pulse 311 with respect to the rotation angle (the pulse width with respect to the horizontal axis representing the rotation angle) is less than Figure 7 . Each of the pulses 311 consists of a sine wave for one period.

[0071] The curve graph 320 shows the rotation detection signal output in the user interface device 1 in response to the rotation amount of the rotary knob member 150 rotating to cause periodic tactile stimulation. The rotation amount is predetermined. Each periodically appearing rectangular pulse 321 indicates the occurrence event of a mechanical tactile click. In the example of Figure 7 , the user interface device 1 generates an event pulse 321 each time it detects the rotation of the rotary knob member 150 through the conductor sheet 151 .

[0072] In the example of Figure 7 , the interval between consecutive pulses 321 is equal to the interval between the pulses 311 of the rotary knob member 150, and the interval between the click pulse 311 and the immediately following event pulse 321 is consistent. The frequencies of the pulses 311 and the pulses 321 are the same, but the phases are different.

[0073] The curve graph 330 shows the drive signal supplied to the actuator 105. The user interface device 1 supplies a drive pulse (drive voltage pulse) 331 to the actuator 105 in response to the event pulse 321 of the rotation detection signal. In response, the actuator 105 vibrates for a short duration. Figure 7 The drive pulse 331 in the example of

[0074] consists of a plurality of consecutive isolated sine waves (sine wave pulses). Compared with the common angular velocity of the rotary knob member 150 rotated by the user, the duration (pulse width) of the drive pulse 331 is much shorter. ZThe time variation. The vibration pulse 341 corresponds to the shearing force applied to the finger by the tactile stimulus of one prompt. The duration of the vibration pulse 341 is the duration during which the actuator 105 vibrates, and it is substantially equal to the duration of the drive pulse 331.

[0075] As Figure 7 shown, the user interface device 1 vibrates the rotary knob member 150 by vibrating the actuator 105 in such a way that the vibration of the actuator 105 is synchronized with the periodic tactile clicks (tactile stimuli) generated by the mechanical structure of the knob device 15. In Figure 7 the example, the angular period of the rotation event pulse 321 is equal to the angular period of the tactile clicks of the mechanical structure of the knob device 15 This means that the angular period of the tactile cue generated by the vibration of the actuator 105 is equal to the angular period of the tactile clicks generated by the mechanical structure of the knob device 15

[0076] The way of synchronizing the rotation event pulse 321 with the vibration pulse 311 by the mechanical tactile clicks of the knob device 15 is not limited to the above example. The angular period of the rotation event pulse 321 can be an integer multiple or an integer fraction of the angular period of the mechanical vibration pulse 311 of the knob device 15 For example, the angular period of the rotation event pulse 321 can be or In addition, the phase difference between the vibration pulse 311 and the rotation event pulse 321 is not limited.

[0077] Figure 8 shows an example of the relationship between the mechanical vibration pulse 311 of the knob device 15 and the rotation event pulse 321. The curve graph 360 represents the relationship between the angle of the rotary knob member 150 and the shearing force f t in the tangential direction to the circumference acting on the fingertip generated by the mechanical vibration. The horizontal axis represents the angle of the rotary knob member 150, and the vertical axis represents the shearing force f t . The periodically occurring pulse 361 represents the shearing force f greater than 0 t acting on the finger periodically. Each time the rotary knob member 150 rotates , a shearing force f t is generated. The rotation amount is predetermined.

[0078] The curve graph 370 represents the rotation detection signal output in the user interface device 1 in response to the rotation of the rotary knob member 150 by the rotation amount . The periodically occurring pulse 371 represents the occurrence event of the mechanical tactile click. The horizontal axis represents the angle of the rotary knob member 150, and the vertical axis represents the amplitude of the event pulse 371.

[0079] In Figure 8 the example, the user interface device 1 controls the rotation angle period of the event pulse 371 The user interface device 1 can calculate, according to the coordinates of the plurality of conductor sheets 151, the angle from the current position to the position where the next mechanical haptic click will be generated in both the clockwise and counterclockwise rotations of the rotary knob member 150.

[0080] In Figure 8 the example, the rotation angle period of the event pulse 371 is equal to the rotation angle period of the haptic click of the rotary knob member 150 The user interface device 1 generates the event pulse 371 each time the rotary knob member 150 rotates by an amount which is detected based on the positions of the plurality of conductor sheets 151. The rotation angle period of the event pulse 371 is determined to be synchronized with the rotation angle period of the haptic click of the rotary knob member 150 ; The value of is an integer multiple or an integer fraction of

[0081] In Figure 8 the example, the phase difference between the angle at which the rotary knob member 150 provides a haptic click and the angle at which the rotation event pulse 371 is generated is 180 degrees. That is, the generation of the event pulse 371 is delayed from the previous mechanical haptic click by This configuration naturally changes the haptic stimulus presented to the user. The phase difference is not limited to 180 degrees. For example, the phase difference can be in the range of 180° ± 90° or 180° ± 180°. The same description applies to the phase difference between the angle at which the rotary knob member 150 provides a haptic click and the angle at which the actuator drive pulse is generated.

[0082] Figure 9 shows an example of the drive signal of the actuator 105. An example of the actuator 105 is a piezoelectric (PZT) actuator. For example, when, according to the rotation operation of the rotary knob member 150, the value to be set, such as the temperature or air volume of the air conditioner, or the volume, has reached the limit (maximum or minimum) of a predetermined range, but the rotary knob member 150 continues to rotate, the user interface device 1 supplies a drive voltage to the actuator 105 to cause the actuator 105 to start vibrating. Thus, the user can know that the value being set has reached the limit without looking at the display on the user interface device 1.

[0083] Referring to Figure 9 , the graph 310 represents the time variation of the shear force f t in the direction tangent to the circumference, where the shear force f tis caused by mechanical vibrations generated by rotating the rotary knob member 150 at an angular velocity ω and acts on the fingertips. The graph 320 shows the amount of rotation of the rotary knob member 150 in the user interface device 1 in response to rotation to cause periodic tactile stimulation that is output as a rotation detection signal. The graph 330 shows the drive signal to be supplied to the actuator 105. These have been described with reference to Figure 7 this.

[0084] The drive pulse 331 consists of a plurality of consecutive isolated sine waves. Specifically, it consists of 11 isolated sine waves with a period of 40 milliseconds (msec). The pulse width of each isolated sine wave (pulse) is 10 milliseconds, and the pulse voltage is 60V. The plurality of isolated sine waves provide appropriate tactile stimulation to the user. There is a period without vibration between two consecutive drive pulses 331, and the two consecutive drive pulses 331 are intermittent.

[0085] Figure 10 is a flowchart of a processing example of the user interface device 1. First, the user interface device 1 performs an initial setting step (S11). Specifically, the user interface device 1 acquires a set of predetermined angles Θ[], at which the structure of the knob device 15 will generate a tactile click. The user interface device 1 clears the previously acquired angle of the rotary knob member 150 to zero. The user interface device 1 sets the state "undetermined" as the state S representing the current state of the rotary knob member 150.

[0086] Next, the user interface device 1 performs an angle acquisition step (S12). Specifically, the user interface device 1 acquires the current rotation angle of the rotary knob member 150 rotation angle calculated based on the positions of the plurality of conductor sheets 151.

[0087] Next, the user interface device 1 performs a rotation state acquisition step (S13). Specifically, the user interface device 1 calculates and the difference between them, and changes the current state S based on the value of the difference as follows: If the difference is 0, the user interface device 1 changes the state S to "stopped"; if the difference is positive, the user interface device 1 changes the state S to "forward rotation"; if the difference is negative, the user interface device 1 changes the state S to "reverse rotation".

[0088] Next, the user interface device 1 performs an event output step (S14). Specifically, if the state S is "stopped", the user interface device 1 does nothing; if the state S is not "stopped", the user interface device 1 searches for the rotation angle of the rotary knob member 150 acquired in the set of angles Θ[] If a match exists, the user interface device 1 outputs an event of a haptic click occurring in the structure of the knob device 15.

[0089] Next, the user interface device 1 performs an angular temporary storage step (S15). Specifically, the user interface device 1 replaces the previously acquired angle with the current rotation angle After that, the user interface device 1 returns to step S12.

[0090] In the following, an example of adjusting the temperature setting of an air conditioner is described. Figure 11 An operation of adjusting the temperature setting of an air conditioner by the user interface device 1 and a change in the image displayed on the user interface device 1 resulting therefrom are shown. The user interface device 1 is configured to change the temperature setting by 1 °C when the knob device 15 generates one mechanical haptic click. A clockwise (rightward) rotation of the rotary knob member 150 increases the temperature, and a counterclockwise (leftward) rotation decreases the temperature.

[0091] When the user taps on the displayed temperature 13A of the left seat in Figure 11 the state 410, the knob device 15 is assigned the use of adjusting the temperature setting of the air conditioner. For example, the user rotates the rotary knob member 150 counterclockwise. When the knob device 15 generates one mechanical haptic click, the user interface device 1 decreases the temperature setting by 1 °C. The current temperature setting of the left seat is 22 °C; a counterclockwise rotation generating two haptic clicks decreases the temperature setting by 2 °C. The state 420 depicts a state where the temperature setting of the left seat has been decreased to 20 °C.

[0092] The user further rotates the rotary knob member 150 counterclockwise from the state 420 where the temperature setting of the left seat is 20 °C. The state of the user interface device 1 changes to the state 430 where the temperature setting of the left seat has been decreased to 18 °C.

[0093] Assume that after setting the temperature of the left seat to 18 °C, the user further rotates the rotary knob member 150 counterclockwise. The user interface device 1 drives the actuator 105 in synchronization with the haptic click generated by the structure of the knob device 15. The unique haptic stimulus obtained by adding (superimposing) the haptic stimulus of the actuator 105 to the haptic stimulus of the structure enables the user to perceive that the temperature setting has reached the lower limit. The temperature setting of the left seat remains at 18 °C.

[0094] For example, when the user rotates the rotary knob member 150 clockwise by one tactile click after perceiving a unique tactile stimulus, the temperature changes from 18 °C to 19 °C. In this case, the vibration of the actuator 105 is not added. A user who knows that the temperature at which the unique tactile stimulus is caused is set to 18 °C can set the desired temperature without looking at the screen of the user interface device 1 based on the number of tactile clicks counted when rotating the knob clockwise.

[0095] Figure 12 is a diagram for explaining the behavior of the user interface device 1 described with reference to Figure 11 As in the example described with reference to Figure 7 it is assumed that the user rotates the rotary knob member 150 counterclockwise at a constant angular velocity ω. Although it is assumed for the sake of explanation that the rotary knob member 150 rotates at a constant angular velocity ω, the actual angular velocity can take any value.

[0096] The graph 310 represents the time variation of the shear force f t in the direction tangent to the circumference, which shear force f t is caused by the mechanical vibration generated by the rotation of the rotary knob member 150 at an angular velocity ω and acts on the fingertip. The graph 320 represents the rotation detection signal output in the user interface device 1 in response to the amount of rotation of the rotary knob member 150 rotating to cause a periodic tactile stimulus rotation. The graph 330 represents the drive signal to be supplied to the actuator 105. The graph 340 represents the shear force in the Z-axis direction caused by the movement of the actuator 105 and acting on the fingertip. These have been described with reference to Figure 7 already.

[0097] The user interface device 1 monitors the angular position of the rotary knob member 150 based on the positions of the plurality of conductive sheets 151 of the rotary knob member 150. When the rotary knob member 150 rotates counterclockwise after the last temperature setting reduction the user interface device 1 reduces the temperature setting of the left seat by 1 °C. Unless the temperature setting reaches the lower limit, the user interface device 1 keeps the actuator 105 from vibrating. That is, the fingertip only receives the shear force f t caused by the mechanical structure.

[0098] In response to the counterclockwise rotation of the rotary knob member 150, the temperature set for the air conditioner reaches the lowest temperature (e.g., 18 °C) at time T1. After time T1, the user interface device 1 starts haptic prompting via the actuator 105 in response to the counterclockwise rotation of the rotary knob member 150. Specifically, the user interface device 1 outputs a drive signal for vibrating the actuator 105 synchronously with the rotation events that continuously occur in response to the continued counterclockwise rotation of the rotary knob member 150 after time T1. Accordingly, a haptic stimulus is presented to the user such that the haptic stimulus generated by the vibration of the actuator 105 is superimposed on the haptic stimulus generated by the structure of the knob device 15.

[0099] In the above example, the temperature setting of the air conditioner is decreased according to the user's operation. The process of increasing the temperature setting of the air conditioner is almost the same as the process of decreasing the temperature setting. The difference between the process of increasing the temperature setting and the process of decreasing the temperature setting lies in the rotation direction of the rotary knob member 150 and the upper limit of the reference temperature setting.

[0100] Specifically, in response to the clockwise rotation of the rotary knob member 150, the temperature set for the air conditioner reaches the highest temperature (e.g., 30 °C) at time T2. After time T2, the user interface device 1 starts haptic prompting via the actuator 105 in response to the clockwise rotation of the rotary knob member 150. Specifically, the user interface device 1 outputs a drive signal for vibrating the actuator 105 synchronously with the rotation events that continuously occur in response to the continued clockwise rotation of the rotary knob member 150 after time T2. Accordingly, a haptic stimulus is presented to the user such that the haptic stimulus generated by the vibration of the actuator 105 is superimposed on the haptic stimulus generated by the structure of the knob device 15.

[0101] In the above example, when the temperature set for the air conditioner is between the lower limit and the upper limit, the user interface device 1 suppresses the vibration of the actuator 105. In another example, when the temperature is neither the upper limit nor the lower limit, the user interface device 1 may vibrate the actuator 105 in a manner different from the vibration at the upper and lower limits. For example, the amplitude or frequency of the drive pulse 331 may be made different, or a phase different from the mechanical haptic click may be provided for the drive pulse 331. In yet another example, the vibration pattern when the temperature is at the upper limit may be different from the vibration pattern when the temperature is at the lower limit.

[0102] Figure 13 Illustrated is for performing with reference to Figure 11 and Figure 12An example of the logical configuration of the process for lowering the temperature setting of an air conditioner. In addition to the knob device 15, the touch-sensing display panel 10, and the actuator 105, the user interface device 1 also includes a controller 50. The controller 50 executes the processes to be performed by the user interface device 1 while controlling other devices. In this example, the controller 50 instructs the temperature regulator 61 of the temperature setting according to the user's operation.

[0103] The controller 50 may include one or more arithmetic devices and one or more storage devices. Examples of arithmetic devices include a processor, a graphics processing unit (GPU), and a field-programmable gate array (FPGA). The storage device stores programs and data to be used by the controller 50. The storage device may include volatile or non-volatile memory. The storage device includes a working area to be used by the program.

[0104] The controller 50 functions as a functional unit (module) for controlling external devices including the touch-sensing display panel 10, the actuator 105, and the temperature regulator 61. In this example, the controller 50 functions as a display controller 51, a touch panel controller 52, an operation assignment unit 55, and a vibration controller 57. The touch panel controller 52 includes a knob operation recognition unit 53 and a rotation detection signal output unit 54.

[0105] The operation assignment unit 55 assigns a specific operation function to the user interface device 1 in response to a specific operation of the user. In the example described with reference to Figure 11 and Figure 12 , the operation assignment unit 55 assigns the air conditioner temperature setting function to the user interface device 1. The display controller 51 generates a specific image according to the assigned operation function and the user's operation, and displays the image on the touch-sensing display panel 10.

[0106] The touch panel controller 52 detects the touch point of the user on the touch-sensing display panel 10 and the touch points of a plurality of conductor sheets 151 attached to the rotary knob member 150 (see Figure 3 ), and forwards the information obtained from the detection results to the operation assignment unit 55 and the vibration controller 57. Specifically, the knob operation recognition unit 53 detects the touch points of the plurality of conductor sheets 151 attached to the rotary knob member 150.

[0107] The rotation detection signal output unit 54 calculates the angle of the rotary knob member 150 based on the touch points of the plurality of conductor sheets 151 of the rotary knob member 150, and forwards this information to the operation assignment unit 55. The operation assignment unit 55 performs angle-temperature conversion 56, determines the temperature to be set (temperature setting change) according to the angle (angle change) of the rotary knob member 150, and notifies the temperature regulator 61 and the display controller 51 of it. The temperature regulator 61 controls the air conditioner to adjust the room temperature to the specified temperature. The display controller 51 generates an image based on the specified temperature.

[0108] The rotation detection signal output unit 54 generates a rotation event pulse in response to a predetermined amount of rotation of the rotary knob member 150 As described above, the rotation event pulse generated in the rotation detection signal output unit 54 is synchronized with the predetermined amount of rotation In addition, the rotation detection signal output unit 54 obtains information about the current temperature setting from the operation assignment unit 55.

[0109] When a predetermined condition is satisfied, the rotation detection signal output unit 54 outputs a rotation event pulse (rotation event signal) for controlling the actuator 105 to the vibration controller 57. In this example, when the temperature setting has reached the lower limit or the upper limit and the rotary knob member 150 is rotated counterclockwise or clockwise, the rotation detection signal output unit 54 outputs a rotation event pulse to the vibration controller 57. The rotation event pulse is synchronized with the predetermined amount of rotation The vibration controller 57 outputs a drive signal to the actuator 105 according to the received rotation event pulse to make the actuator 105 vibrate.

[0110] Next, an example of adjusting the air volume setting of the air conditioner is described. Figure 14 An operation of adjusting the air volume setting of the air conditioner by the user interface device 1 and a change in the image displayed on the user interface device 1 resulting therefrom are shown. The user interface device 1 is configured to change the air volume setting by one level when the knob device 15 generates one mechanical tactile click. A clockwise (rightward) rotation of the rotary knob member 150 increases the air volume, and a counterclockwise (leftward) rotation decreases the air volume.

[0111] When the user taps the human image 14A of the left seat displayed in Figure 14 in the state 710, the knob device 15 is assigned the use of adjusting the air volume setting of the air conditioner. For example, the user rotates the rotary knob member 150 clockwise. When the knob device 15 generates one mechanical tactile click, the user interface device 1 increases the air volume setting by one level. The current air volume setting of the left seat is level 1; a clockwise rotation that generates two tactile clicks increases the air volume setting by two levels. The state 720 depicts the state where the air volume setting of the left seat has been increased to level 3.

[0112] The user further rotates the rotary knob member 150 clockwise from the state 720 where the air volume setting of the left seat is at level 3. The state of the user interface device 1 changes to the state 730 where the air volume setting of the left seat is increased to level 5 (the maximum level).

[0113] Assume that after setting the air volume of the left seat to level 5, the user further rotates the rotary knob member 150 clockwise. The user interface device 1 drives the actuator 105 in synchronization with the tactile click generated by the structure of the knob device 15. The unique tactile stimulus obtained by superimposing the tactile stimulus of the actuator 105 on the tactile stimulus of the structure enables the user to perceive that the air volume setting has reached the upper limit. The air volume setting of the left seat remains at level 5.

[0114] For example, when the user rotates the rotary knob member 150 counterclockwise by one tactile click after perceiving the unique tactile stimulus, the air volume changes from level 5 to level 4. In this case, the vibration of the actuator 105 is not added. The user who knows that the air volume setting that causes the unique tactile stimulus is level 5 can set the desired air volume without looking at the screen of the user interface device 1 based on the number of tactile clicks counted when rotating the knob counterclockwise.

[0115] Figure 15 is for explaining with reference to Figure 14 the behavior of the user interface device 1 described. Assume that the user rotates the rotary knob member 150 clockwise at a constant angular velocity ω. Although for the sake of explanation, the rotary knob member 150 rotates at a constant angular velocity ω, the actual angular velocity can take any value.

[0116] The graph 310 represents the time variation of the shear force f t in the direction tangent to the circumference, and this shear force f t is caused by the mechanical vibration generated by the rotation of the rotary knob member 150 at the angular velocity ω and acts on the fingertip. The graph 320 represents the rotation detection signal output by the user interface device 1 in response to the amount of rotation of the rotary knob member 150 that causes periodic tactile stimulation rotation. The graph 330 represents the drive signal to be provided to the actuator 105. The graph 340 represents the shear force in the Z-axis direction caused by the movement of the actuator 105 and acting on the fingertip.

[0117] The user interface device 1 monitors the angular position of the rotary knob member 150 based on the positions of the plurality of conductor sheets 151 of the rotary knob member 150. When the rotary knob member 150 rotates clockwise after the last increase in the air volume setting When the above occurs, the user interface device 1 increases the air volume setting of the left seat by one level. Unless the air volume setting reaches the upper limit, the user interface device 1 keeps the actuator 105 from vibrating. That is, the fingertip only receives the shear force f caused by the mechanical structure t .

[0118] In response to the clockwise rotation of the rotary knob member 150, the air volume set for the air conditioner reaches the maximum air volume level (e.g., level 5) at time T5. After time T5, the user interface device 1 starts tactile prompting through the actuator 105 in response to the clockwise rotation of the rotary knob member 150. Specifically, the user interface device 1 outputs a drive signal for vibrating the actuator 105 synchronously with the rotation events that continuously occur in response to the rotary knob member 150 still continuing to rotate clockwise after time T5. Therefore, a tactile stimulus is presented to the user such that the tactile stimulus generated by the vibration of the actuator 105 is superimposed on the tactile stimulus generated by the structure of the knob device 15

[0119] The above example increases the air volume setting of the air conditioner according to the user's operation. The process of decreasing the air volume setting of the air conditioner is almost the same as the process of increasing the air volume setting. The difference between the process of decreasing the air volume setting and the process of increasing the air volume setting lies in the rotation direction of the rotary knob member 150 and the lower limit of the reference air volume setting

[0120] Specifically, in response to the counterclockwise rotation of the rotary knob member 150, the air volume set for the air conditioner reaches the minimum level (e.g., level 0) at time T6. After time T6, the user interface device 1 starts tactile prompting through the actuator 105 in response to the counterclockwise rotation of the rotary knob member 150. Specifically, the user interface device 1 outputs a drive signal for vibrating the actuator 105 synchronously with the rotation events that continuously occur in response to the rotary knob member 150 still continuing to rotate counterclockwise after time T6. Therefore, a tactile stimulus is presented to the user such that the tactile stimulus generated by the vibration of the actuator 105 is superimposed on the tactile stimulus generated by the structure of the knob device 15

[0121] In the above example, when the air volume set for the air conditioner is between the lower limit and the upper limit, the user interface device 1 suppresses the vibration of the actuator 105. In another example, when the air volume is neither the upper limit nor the lower limit, the user interface device 1 can vibrate the actuator 105 in a manner different from the vibration at the upper and lower limits. For example, the amplitude or frequency of the drive pulse 331 can be made different, or a different phase can be provided for the drive pulse 331 from the mechanical tactile click. In yet another example, the vibration mode when the air volume is at the upper limit can be different from the vibration mode when the air volume is at the lower limit

[0122] Figure 16 Shows for performing with reference to Figure 14And Figure 15 An example of the logical configuration for the process of increasing the air volume setting of an air conditioner is described. The following mainly describes the differences from the configuration example in Figure 13 . Compared with the logical configuration in Figure 13 , the process of the operation allocation unit 55 changes from angle-temperature conversion 56 to angle-air volume conversion 59, and the temperature regulator 61 is replaced by the air volume regulator 63.

[0123] In the example described with reference to Figure 14 and Figure 15 , the operation allocation unit 55 assigns the air conditioner air volume setting function to the user interface device 1. The display controller 51 generates a specific image based on the assigned operation function and the user's operation, and displays the image on the touch-sensing display panel 10. The operation allocation unit 55 performs angle-air volume conversion 59, determines the air volume to be set (air volume setting change) according to the angle (angle change) of the rotary knob member 150, and notifies it to the air volume regulator 63 and the display controller 51. The air volume regulator 63 controls the air conditioner to adjust the air volume to the specified level. The display controller 51 generates an image based on the specified air volume.

[0124] The rotation detection signal output unit 54 generates a rotation event pulse in response to a predetermined rotation amount of the rotary knob member 150 . As described above, the rotation event pulse generated in the rotation detection signal output unit 54 is synchronized with the predetermined rotation amount . In addition, the rotation detection signal output unit 54 obtains information about the current air volume setting from the operation allocation unit 55.

[0125] When a predetermined condition is satisfied, the rotation detection signal output unit 54 outputs a rotation event pulse (rotation event signal) for controlling the actuator 105 to the vibration controller 57. In this example, when the air volume setting has reached the lower limit or the upper limit and the rotary knob member 150 rotates counterclockwise or clockwise, the rotation detection signal output unit 54 outputs a rotation event pulse to the vibration controller 57. The rotation event pulse is synchronized with the predetermined rotation amount . The vibration controller 57 outputs a drive signal to the actuator 105 according to the received rotation event pulse to make the actuator 105 vibrate.

[0126] Similar to the above temperature setting adjustment and air volume setting adjustment, the controller 50 can be configured to provide a drive signal to the actuator 105 to make it vibrate only during a period when a predetermined condition is satisfied. Thus, the user can know the current state without looking at the image displayed on the user interface device 1. The predetermined condition can be determined differently for each function assigned to the knob device 15.

[0127] For the function assigned to the knob device 15, a combination of the range of values changed by the operation of the user and the operation direction of the rotary knob member 150 is predetermined. When the value to be changed is within the predetermined range, the controller 50 causes the actuator 105 to vibrate in response to the operation of the rotary knob member 150 in a predetermined direction.

[0128] In the above example, when the value to be changed (such as temperature or air volume) has reached the upper limit and the knob is operated in the direction of further increasing the value, or when the value to be changed has reached the lower limit and the knob is operated in the direction of further decreasing the value, the controller 50 adds vibration through the actuator 105. The controller 50 can always cause the actuator to vibrate in the same manner in response to the operation of the knob.

[0129] As described above, embodiments of the present invention have been described; however, the present invention is not limited to the above embodiments. Those skilled in the art can easily modify, add, or convert each element in the above embodiments within the scope of the present invention. A part of the configuration of one embodiment can be replaced with the configuration of another embodiment, or the configuration of one embodiment can be incorporated into the configuration of another embodiment.

Claims

1. A user interface device, comprising: A knob device having a mechanical structure for prompting a tactile stimulus in response to each predetermined amount of knob operation; a panel on which the knob device is mounted; Controller; as well as a vibration generator configured to vibrate the knob device, the vibration generator being controllable by the controller, The controller controls the vibration generator to vibrate the knob device in synchronization with the predetermined amount of knob operation, so that the knob device prompts tactile stimulation in addition to the tactile stimulation of the mechanical structure.

2. The user interface device according to claim 1, wherein: The vibration of the knob device synchronized with the predetermined amount of knob operation is generated every time the predetermined amount of knob operation occurs.

3. The user interface device according to claim 1, wherein: The driving pulse for the vibration generator to vibrate the knob device in synchronization with the predetermined amount of knob operation is composed of a plurality of isolated sine waves.

4. The user interface device according to claim 1, wherein: The vibration generator is an actuator mounted on the panel and driven by a driving signal.

5. The user interface device according to claim 1, wherein: The vibration of the knob device caused by the vibration generator is generated only when a predetermined condition is satisfied.

6. The user interface device according to claim 5, in, The controller is configured to assign a function selected from a plurality of functions to the knob device, wherein, for a selected function, a combination of a range of values ​​to be adjusted by a user operation and an operation direction of the knob device is predetermined, and Wherein, the controller is configured to cause the vibration generator to vibrate in response to the operation of the knob device in the predetermined direction when the value to be changed is within the predetermined range.

7. The user interface device according to claim 5, in, The knob device is assigned the function of adjusting the temperature setting of the air conditioner, Wherein, the controller is configured to cause the vibration generator to vibrate in response to an operation of the knob device to increase the temperature setting when the temperature setting has reached an upper limit or an operation of the knob device to decrease the temperature setting when the temperature setting has reached a lower limit.

8. A method for controlling tactile cues of a user interface device, the user interface device comprising: A knob device having a mechanical structure for prompting a tactile stimulus in response to each predetermined amount of knob operation; a panel on which the knob device is mounted; and a vibration generator configured to vibrate the knob device, The method comprises: The vibration generator is controlled to vibrate the knob device in synchronization with the predetermined amount of knob operation so that the knob device prompts tactile stimulation in addition to the tactile stimulation of the mechanical structure.

Images