User interface apparatus, and haptic sense presentation control method

Abstract

To provide a variety of haptic sense presentation to a user.SOLUTION: A user interface apparatus according to the present invention has a knob device having a mechanical structure presenting haptic sense for each of predetermined knob operation amounts, a panel provided with the knob device, a control device, and a vibration generator which can be controlled by the control device for vibrating the knob device. The control device controls the vibration generator to vibrate the knob device in synchronization with the predetermined knob operation amounts, thereby presenting haptic sense in addition to haptic sense generated by a mechanical structure.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to tactile presentation.

Background Art

[0002] In recent years, touch panels have come to be used in many devices in place of mechanical switches, buttons, etc. However, since touch panels do not have surface irregularities like mechanical switches or buttons, it is necessary to visually confirm a series of processes from confirmation of the operation position to completion of the operation. As a countermeasure, there are touch panels having a tactile presentation function, but there are restrictions in the usage environment and they are not universal.

[0003] Therefore, devices having the multifunction of a touch panel with a tactile presentation function and the certainty by knob operation have been proposed. Such devices are assumed to be used, for example, in car component operations performed during driving of an automobile, inspection devices used in the medical field, devices used in the video / audio field, etc., in devices that are operated in parallel with other operations.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] In an interface operated by a user, a technique capable of diversifying tactile presentation to the user is desired.

Means for Solving the Problems

[0006] A user interface device according to an aspect of the present disclosure includes a knob device having a mechanical structure that presents a sense of touch for each predetermined knob operation amount, a panel on which the knob device is installed, a control device, and a vibration generator that can be controlled by the control device and vibrates the knob device. The control device controls the vibration generator to vibrate the knob device in synchronization with the predetermined knob operation amount, thereby presenting a sense of touch in addition to the sense of touch by the mechanical structure.

[0007] An aspect of the present disclosure is a method for controlling haptic presentation in a user interface device. The user interface device includes a knob device having a mechanical structure that presents a sense of touch for each predetermined knob operation amount, a panel on which the knob device is installed, and a vibration generator that vibrates the knob device. The haptic presentation control method controls the vibration generator to vibrate the knob device in synchronization with the predetermined knob operation amount, thereby presenting a sense of touch in addition to the sense of touch by the mechanical structure.

Advantages of the Invention

[0008] An aspect of the present disclosure can diversify haptic presentation to the user.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be noted that this embodiment is merely an example for realizing the present disclosure and does not limit the technical scope.

[0011] One embodiment of the present disclosure adds a tactile sensation by applying a programmable vibration to a knob device having a mechanism for presenting a tactile sensation and installed on a panel. Thereby, the tactile sensation presented to the finger of the user operating the knob can be diversified, and tactile AR (augmented reality) can be realized.

[0012] FIG. 1 shows a configuration example of a user interface device according to an embodiment of the present disclosure. The user interface device 1 is mounted on, for example, an instrument panel of an automobile. In the configuration example shown in FIG. 1, the user interface device 1 enables a user to operate a radio, a map, a phone, music, and an air conditioner. FIG. 1 shows a state where the user adjusts the set temperature of the air conditioner for the left seat.

[0013] The user interface device 1 includes a display panel 10 with a touch detection function and a mechanical knob device 15 fixed on the display panel 10 with the touch detection function. The display panel 10 with the touch detection function includes, for example, a stacked touch panel and a display panel. The knob device 15 is fixed on the visual side (front side) of the touch panel, and the display panel is disposed on the back side of the touch panel. Further, a lateral actuator (not shown in FIG. 1) is fixed to the front surface or the back surface of the display panel 10 with the touch detection function.

[0014] In the configuration example shown in FIG. 1, the lateral actuator vibrates in the Y-axis direction. Note that the number, position, vibration direction, and vibration mode of the actuator are not limited as long as a tactile sensation can be presented in response to a knob operation by the user. Further, one or both of the position and the shape of the display panel 10 with the touch detection function may be vibrated for tactile presentation, or only the knob device 15 may be vibrated. A vibration generator different from the actuator may be used.

[0015] In FIG. 1, the horizontal axis in the plane is the X-axis, the vertical axis in the plane is the Y-axis, and these are perpendicular. The Z-axis is perpendicular to the X-axis and the Y-axis, that is, perpendicular to the plane defined by the X-axis and the Y-axis. The user visually recognizes the display image of the user interface device 1 in the Z-axis direction.

[0016] The display panel 10 with a touch detection function displays a predetermined object image according to a user operation. In the example shown in FIG. 1, the display panel 10 with a touch detection function displays button images 11A to 11D. The images 11A to 11D are buttons for selecting radio voice output, phone call, map display, and music playback, respectively. Tactile sensation may or may not be presented for touching any of the button images.

[0017] The display panel 10 with a touch detection function further displays the person images 14A and 14B of the left and right seats and the set temperatures 13A and 13B of the air conditioner. When the user touches the image 13A or 13B of the set temperature, the function of setting the temperature of the left seat or the right seat is assigned to the knob device 15. In the example of FIG. 1, the temperature setting of the left seat is selected.

[0018] The knob device 15 includes a rotatable knob component, and the user can rotate the knob component, for example, by the finger 20. In the configuration example shown in FIG. 1, the knob component can rotate freely in both the left and right rotation directions. That is, the knob component can continue to rotate without stopping in both the left and right directions. As will be described later, the knob device 15 has a mechanism that gives a click feeling for each certain rotation angle (rotation angle unit) of the knob component.

[0019] In the example shown in FIG. 1, when the displayed set temperature 13A of the left seat is tapped by an indicator such as a finger, the use of setting the temperature of the air conditioner for the left seat is assigned to the knob device 15. For example, for each click feeling generated by the knob device 15, the set temperature changes by 1°C. When the knob component is rotated to the left, the set temperature decreases by 1°C for each click feeling generated by the knob device 15, and when the knob component is rotated to the right, the set temperature increases by 1°C for each click feeling generated by the knob device 15. The settable temperature range is defined. When the knob component continues to rotate to the left, the set temperature stops at the defined minimum temperature. When the knob component continues to rotate to the right, the set temperature stops at the defined maximum temperature.

[0020] Figure 2 schematically shows an example of the air volume setting of the air conditioner by the user interface device 1. When the left person image 14A is tapped by an indicator such as a finger, the use of the air volume setting of the air conditioner for the left seat is assigned to the knob device 15. The object image 17 shows the current set air volume.

[0021] For example, for each click feeling generated by the knob device 15, the set air volume changes by one level. When the knob part is rotated to the left, the set air volume decreases by one level for each click feeling generated by the knob device 15, and when the knob part is rotated to the right, the set air volume increases by one level for each click feeling generated by the knob device 15. The range of adjustable air volume is defined. When the knob part continues to rotate to the left, the set air volume stops at the defined minimum air volume (for example, zero). When the knob part continues to rotate to the right, the set temperature stops at the defined maximum air volume.

[0022] In addition to the above-described air conditioner operations, various other functions can be assigned to the knob device 15. For example, functions such as music playback, volume adjustment on the phone, and selection of items from a list can be assigned.

[0023] Various configurations of knob devices that give a click feeling by mechanical mechanisms are known. FIGS. 3 and 4 schematically show a configuration example of the knob device 15. FIGS. 3 and 4 show a configuration example of a dial-type knob device 15. FIG. 3 shows a cross-section viewed in the Y-axis direction, and FIG. 4 shows a partial cross-section viewed along the Z-axis.

[0024] The features of the present disclosure can also be applied to other types of knob devices such as slider-type knob devices. For example, the dial-type knob device 15 includes a knob part that rotates on a fixed base part, and the slider-type knob device includes a slide knob part that slides in a straight line direction on a fixed base part. A mechanical click feeling (tactile) is presented for each predetermined slide amount in a certain direction. The structure of the knob device is not limited as long as it has a mechanism for presenting a click feeling. The movable range of the knob part is not particularly limited.

[0025] Referring to FIG. 3, the dial-type knob device 15 includes a base component 160 and a rotary knob component 150. The base component 160 is an inner ring component, and the rotary knob component 150 is an outer ring component. The base component 160 is fixed to the installation surface of the display panel 10 with a touch detection function, for example, adhered by an adhesive. The dial-type knob device 15 has a bearing structure including balls 158. Note that the structure enabling the movement of the knob component in the knob device 15 is not particularly limited.

[0026] On the bottom surface of the rotary knob component 150, that is, the surface facing the front surface of the display panel 10 with a touch detection function, a plurality of conductors 151 spaced apart in the circumferential direction are included. The user interface device 1 can detect the position of each conductor 151 in contact with the surface of the display panel 10 with a touch detection function.

[0027] Referring to FIG. 4, the rotary knob component 150 has recesses 152 formed at equal intervals on its inner peripheral surface. The outer and inner circumferences of the rotary knob component 150 are circular. The base component 160 includes a spring portion 163. A protrusion 161 is disposed on the outer surface of the spring portion 163, that is, the surface facing the inner peripheral surface of the rotary knob component 150. The spring portion 163 has elasticity in the X-axis direction and presses the protrusion 161 against the region between the recesses 152.

[0028] In response to the rotation of the rotary knob component 150, the protrusion 161 is sequentially inserted into and removed from the recess 152, giving a clicking feeling to the finger operating the rotary knob component 150. Thus, the clicking feeling by the knob device 15 is, for example, a tactile sensation generated by the accumulation and release of strain energy in an elastic body.

[0029] The user interface device 1 detects the rotation angle of the rotary knob component 150, controls the display image according to the detected angle, and also generates a signal for controlling other electronic components such as an air conditioner and a media player according to the detected angle. The rotation angle of the rotary knob component 150 can be determined from the position change of a plurality of conductors 151 built in the rotary knob component 150.

[0030] FIG. 5 is a diagram for explaining a method of determining a rotation angle from detection positions of two conductors 151 disposed on the bottom surface of the rotary knob component 150. FIG. 5 shows the coordinates on the surface of the display panel 10 with touch detection function, and the positions of the knob component 150 and its built-in conductors 151 on the coordinate surface.

[0031] In the example shown in FIG. 5, the rotary knob component 150 includes two conductors 151, which are conductor A 151A and conductor B 151B. Conductor A 151A and conductor B 151B are arranged 180 degrees apart on a circumference centered on the rotation center 156 of the knob component 150. The user interface device 1 determines the rotation angle of the rotary knob component 150 from the positions of conductor A 151A and conductor B 151B.

[0032] The coordinates (x A , y A ) of conductor A 151A and the coordinates (x B , y B ) of conductor B 151B are detected by the display panel 10 with touch detection function, the coordinates (x C , y C ) of the rotation center of the rotary knob component 150 can be obtained by the following formula.

[0033]

Equation

[0034] The angle φ at which the straight line connecting conductor A 151A and conductor B 151B intersects the horizontal line along the X-axis passing through the rotation center at the rotation center 156 of the rotary knob component 150 can be obtained by the following formula.

[0035]

Equation

[0036] The rotational angle of the rotary knob component 150 can be determined from the positions of three or more conductors. As an example, assume that three conductors A, B, and C are built into the rotary knob component 150. These are arranged spaced apart on the circumference of the center of rotation of the rotary knob component 150.

[0037] Let the initial coordinates of conductor A be (X A , Y A ), the initial coordinates of conductor B be (X B , Y B ), and the initial coordinates of conductor C be (X C , Y C ). Let the coordinates of conductor A after rotation be (X A ´, Y A ´), the coordinates of conductor B after rotation be (X B ´, Y B ´), and the coordinates of conductor C after rotation be (X C ´, Y C ´). From the initial coordinates and the coordinates after rotation of conductors A, B, and C, the rotational angle φ of the rotary knob component 150 can be obtained by the following formula.

[0038]

Equation

[0039] atan2(Y B - Y A , X B - X A ) indicates the angle between the straight line connecting the positions of conductor A and conductor B in the initial state and the X-axis. atan2(Y C - Y A , X C - X A ) indicates the angle between the straight line connecting the positions of conductor A and conductor C in the initial state and the X-axis. atan2(Y B ´ - Y A ´, X B ´ - X A ´) indicates the angle between the straight line connecting the positions of conductor A and conductor B after rotation and the X-axis. atan2(Y C ´ - Y A ´, X C ´ - X A“) indicates the angle between the straight line connecting the positions of the conductor A and the conductor C after rotation and the X-axis. The above two equations each indicate the rotation angle φ. For example, among the three conductors A, B, and C, for some reason, if one conductor C cannot be detected and X C and Y C become indeterminate, the rotation angle φ can still be obtained as long as the two conductors A and B can be detected. This rotation angle φ indicates the rotation angle of the knob part 150.

[0040] By using three conductors in asymmetric positions, an improvement in accuracy near 90 degrees or 270 degrees can be expected compared to using two conductors.

[0041] As described above, the method of using two conductors infers the rotation angle from the slope of the straight line between two points. On the other hand, the method of using three conductors infers the rotation angle from the slopes of the two straight lines formed among three points. Since there is more information for inferring the rotation angle when using three conductors compared to two conductors, an improvement in accuracy can be expected.

[0042] That is, an increase in the number of straight lines means an increase in clues, thus increasing the accuracy. Also, it is desirable that the three conductors are arranged at the vertices of a non - symmetric triangle rather than at symmetric positions such as an equilateral triangle. Compared to a symmetric arrangement, in an asymmetric arrangement, the distance difference between the two line segments becomes clear, and the relative position change is easier to perceive.

[0043] To infer the rotation angle, the user interface device 1 identifies each conductor and detects their positions. The user interface device 1, for example, in the initial setting, assigns an ID to each conductor and determines their initial positions. The user interface device 1 can determine the current position of each conductor by tracking the position of each identified conductor. A widely known multi - touch detection function can be applied to the identification of conductors.

[0044] Also, by presetting the relationship between the positions of the plurality of conductors and the position of the rotary knob component 150 that gives a click feeling, it is possible to estimate the rotation angle from the current position of the plurality of conductors to the presentation of the click feeling by the next mechanical structure. For example, the positional relationships among the protrusion 161, the plurality of recesses 152, and the plurality of conductors 151 shown in FIG. 4 are preset. Note that a unique touch waveform may be detected by some or all of the conductors, and each conductor may be identified.

[0045] Hereinafter, a method of tactile presentation by the user interface device 1 will be described. In one embodiment of the present specification, the user interface device 1 diversifies the tactile feedback to the user by applying a programmable vibration to the knob component 150 in addition to the mechanical click feeling presented by the knob device 15. The configuration example described below vibrates the display panel 10 with a touch detection function by an actuator to give the user tactile feedback via the knob component 150, but the method is not particularly limited.

[0046] FIG. 6 schematically shows a configuration example of a part of the user interface device 1. The user interface device 1 includes a cover glass 103 of a display panel with a touch detection function and a knob device 15 attached to the surface of the cover glass 103. FIG. 6 shows the rotary knob component 150 of the knob device 15. The surface of the cover glass 103 is the mounting surface of the knob device 15. The actuator 105 is attached to the surface of the cover glass 103 on the side opposite to the surface where the knob device 15 is attached. For example, the actuator 105 vibrates in the normal direction or in-plane direction of the knob device mounting surface.

[0047] In one embodiment of the present specification, the user interface device 1 superimposes an electrically controllable vibration of the knob device 15 on a mechanical vibration (click feeling) of the knob device 15 by vibrating the surface on which the knob device 15 is installed using an electrically controllable actuator 105. The user interface device 1 synchronizes a programmable vibration externally applied with the mechanical vibration of the knob device 15 to diversify the tactile feedback to the user and realize tactile AR (augmented reality).

[0048] In one embodiment of the present specification, the knob device 15 presents a periodic tactile sensation (click feeling presented by the mechanism) with respect to the rotational position of the rotary knob component 150. That is, when the rotary knob component 150 rotates by a specific angle from the previous click feeling presentation position, the next click feeling is presented. The angle (rotation amount) Δφ between adjacent positions presenting the click feeling is constant. Note that Δφ does not have to be constant.

[0049] The user interface device 1 controls the vibration of the actuator 105 so as to synchronize with the periodic vibration (repeated click feeling) due to the structure of the knob device 15. Since the click feeling mechanically presented by the knob device 15 is generated by its mechanical structure, it cannot be electronically programmed. However, since the externally applied programmable vibration is synchronized, the user is mislead into feeling that the tactile sensation of the click feeling changes seamlessly.

[0050] The click feeling generated by the mechanical structure of the knob device 15 changes the shearing force fz in the rotational direction acting on the fingertip. The externally applied programmable vibration is, for example, a vibration in the normal direction or in-plane direction of the surface on which the knob device 15 is attached. In FIG. 6, the normal direction is the Z-axis direction.

[0051] Since the Pacinian corpuscles of the mechanical receptor that detects vibration cannot distinguish the direction of vibration, the user perceives as if the shear force fz in the rotational direction acting on the fingertip is changing. As a result, it is possible to make the user perceive that the click feeling generated by the mechanical structure of the knob device 15 is changing.

[0052] In one embodiment of the present specification, the rotary knob component 150 is designed to rotate infinitely without stopping. Thereby, the applications can be diversified. By presenting vibrations synchronized with vibrations caused by a programmable mechanical structure applied from the outside, for example, when the upper limit of the temperature setting is reached, the user can know the fact without giving a sense of discomfort to the user and without the user seeing the state of the operation target.

[0053] Depending on a configuration different from that of the actuator 105, electrically controllable vibrations may be applied to the rotary knob component 150. For example, by controlling the signal applied to the electrodes included in the display panel 10 with a touch detection function, the resistance applied from the electrodes to the conductor of the rotary knob component 150 can be controlled. In this way, programmable vibrations can be applied to the rotary knob component 150 by various vibration generators.

[0054] FIG. 7 is a diagram for explaining an operation example of the user interface device 1 when the knob component 150 is rotated at a constant angular velocity ω. The graph 310 shows the time change of the shear force ft in the tangential direction of the circumference acting on the fingertip due to the mechanical vibration generated by the rotation at the angular velocity ω of the rotary knob component 150. The periodically appearing pulse 311 indicates that a shear force ft greater than 0 is acting on the finger. The shear force ft is generated every time the rotary knob component 150 rotates by Δφ. The width of the pulse 311 with respect to the rotation angle (pulse width with the rotation angle as the horizontal axis) is smaller than Δφ. In FIG. 7, the pulse 311 is composed of a sine wave of one cycle.

[0055] Graph 320 shows the rotational detection signal output within the user interface device 1 in response to the rotational amount Δφ of the rotary knob component 150 corresponding to the periodic tactile sensation. The rotational amount Δφ is preset. The periodically appearing rectangular pulse 321 indicates the occurrence event of a mechanical click feeling. In the example shown in FIG. 7, the user interface device 1 generates an event pulse 321 for each rotation of Δφ of the rotary knob component 150 detected by the conductor 151.

[0056] In the example shown in FIG. 7, the interval between consecutive pulses 321 is the same as the interval of the pulses 311 of the rotary knob component 150, and the interval between the event pulse 321 immediately after the click pulse 311 is constant. The frequency of the pulse 311 and the frequency of the pulse 321 are the same, but the phases are different.

[0057] Graph 330 shows the drive signal applied to the actuator 105. The user interface device 1 applies a drive pulse 331 (drive voltage pulse) to the actuator 105 in response to the event pulse 321 of the rotational detection signal. As a result, the actuator 105 vibrates for a short time. In the example of FIG. 7, the drive pulse 331 is composed of a plurality of consecutive isolated sine waves (sine wave pulses). Compared with the general angular velocity of the rotary knob component 150 by the user, the period (pulse width) of the drive pulse 331 is much shorter.

[0058] Graph 340 shows the time change of the shear force fz in the tangential direction of the circumference acting on the fingertip due to the vibration of the actuator 105. The vibration pulse 341 indicates the shear force applied to the finger by one tactile presentation. The period of the vibration pulse 341 is the vibration period of the actuator 105 and substantially coincides with the period of the drive pulse 331.

[0059] As shown in FIG. 7, the user interface device 1 vibrates the rotary knob part 150 by the vibration of the actuator 105 so as to synchronize with the periodic click feeling (tactile sensation) due to the mechanical structure of the knob device 15. In the example shown in FIG. 7, the angular period of the rotation event pulse 321 is the same as the angular period Δφ of the click feeling due to the mechanical structure of the knob device 15. That is, the angular period of the tactile presentation due to the vibration of the actuator 105 is the same as the angular period Δφ of the click feeling due to the mechanical structure of the knob device 15.

[0060] The synchronization mode of the vibration pulse 311 of the mechanical click feeling of the knob device 15 and the rotation event pulse 321 is not limited to the above example. The angular period of the rotation event pulse 321 may be an integer multiple or a fraction of the angular period Δφ of the mechanical vibration pulse 311 of the knob device 15. For example, the angular period of the rotation event pulse 321 may be Δφ / 2 or 2*Δφ. Also, the phase difference between the vibration pulse 311 and the rotation event pulse 321 is not limited.

[0061] FIG. 8 shows an example of the relationship between the mechanical vibration pulse 311 of the knob device 15 and the rotation event pulse 321. The graph 360 shows the relationship between the angle of the rotary knob part 150 and the shearing force ft in the tangential direction of the circumference acting on the fingertip due to the mechanical vibration. The horizontal axis represents the angle of the rotary knob part 150, and the vertical axis represents the shearing force ft. The periodically appearing pulse 361 indicates that a shearing force ft greater than 0 acts on the finger. The shearing force ft is generated every time the rotary knob part 150 rotates by Δφ. The rotation amount Δφ is preset.

[0062] The graph 320 shows the rotation detection signal output within the user interface device 1 according to the rotation amount Δφ of the rotary knob part 150. The periodically appearing pulse 371 indicates the occurrence event of the mechanical click feeling. The horizontal axis represents the angle of the rotary knob part 150, and the vertical axis represents the magnitude of the event pulse 371.

[0063] In the example shown in FIG. 8, the user interface device 1 controls the rotation angle period Δφ1 of the event pulse 371. The user interface device 1 can know, from the coordinates of the plurality of conductors 151, the angles from the current position to the positions that give a mechanical click feeling in the respective left and right rotation directions.

[0064] In the example shown in FIG. 8, the rotation angle period Δφ1 of the event pulse 371 is the same as the rotation angle period Δφ of the click feeling of the rotary knob component 150. The user interface device 1 generates an event pulse 321 for each rotation of Δφ of the rotary knob component 150 detected from the positions of the plurality of conductors 151. The rotation angle period Δφ1 of the event pulse 371 is determined to be synchronized with the rotation angle period Δφ of the click feeling of the rotary knob component 150, and Δφ1 is an integer multiple or a fraction of 1 of Δφ.

[0065] In the example shown in FIG. 8, the phase difference α between the angle giving the click feeling of the rotary knob component 150 and the angle of the rotation event pulse 321 is 180 degrees. That is, the event pulse 371 is generated with a delay of Δφ / 2 from the previous mechanical click feeling. Thereby, the tactile sensation presented to the user can be changed more naturally. Note that the phase difference is not limited to 180 degrees. For example, the phase difference may be within the range of 180°±90° or within the range of 180°±180°. The same explanation applies to the phase difference between the angle giving the click feeling of the rotary knob component 150 and the drive pulse to the actuator.

[0066] FIG. 9 shows an example of a drive signal for the actuator 105. The actuator 105 is, for example, a piezo (PZT) actuator. For example, when the rotation operation of the rotary knob component 150 causes the value of the setting target such as the air conditioner temperature, air volume, or volume to reach the boundary value (maximum value or minimum value) of the specified range, and further, when the rotation operation of the rotary knob component 150 is performed, the user interface device 1 applies a drive voltage to the actuator 105 to start its vibration. Thereby, the user can know that the setting target has reached the specified value without visually recognizing the display of the user interface device 1.

[0067] Referring to FIG. 9, graph 310 shows the time variation of the tangential shear force ft in the circumferential direction acting on the fingertip due to the mechanical vibration generated by rotation at the angular velocity ω of the rotary knob component 150. Graph 320 shows the rotation detection signal output within the user interface device 1 in response to the rotation amount Δφ of the rotary knob component 150 corresponding to the periodic tactile sensation. Graph 330 shows the drive signal applied to the actuator 105. These are described with reference to FIG. 7.

[0068] The drive pulse 331 is composed of a plurality of consecutive isolated sine waves. Specifically, the drive pulse 331 is composed of 11 isolated sine waves with a period of 40 msec. The pulse width of each isolated sine wave (pulse) is 10 msec, and the pulse voltage is 60V. A plurality of isolated sine waves can present a suitable tactile sensation to the user. There is a vibration-free period between consecutive drive pulses 331, and the plurality of consecutive drive pulses 331 are intermittent.

[0069] FIG. 10 shows a flowchart of a processing example by the user interface device 1. First, the user interface device 1 executes an initial setting step (S11). More specifically, the user interface device 1 acquires an array Θ[] of angles at which a click feeling is generated by the mechanism of the knob device 15, which is preset. The user interface device 1 clears the previously acquired angle φn-1 to 0. The user interface device 1 sets the state S indicating the current state of the rotary knob component 150 to "indeterminate".

[0070] Next, the user interface device 1 executes an angle acquisition step (S12). More specifically, the user interface device 1 acquires the rotation angle φn of the current rotary knob component 150. The rotation angle φn is calculated from the positions of the plurality of conductors 151.

[0071] Next, the user interface device 1 executes an acquisition step of the rotation state (S13). More specifically, the user interface device 1 obtains the difference between φn and φn-1, and changes the current state from the value of the difference as follows. When the difference is 0, the user interface device 1 sets the state S to "stopped". When the difference is positive, the user interface device 1 sets the state S to "clockwise rotation". When the difference is negative, the user interface device 1 sets the state S to "counterclockwise rotation".

[0072] Next, the user interface device 1 executes an event output step (S14). More specifically, when the state S is "stopped", the user interface device 1 does nothing. When the state S is other than "stopped", the user interface device 1 searches the array Θ[], and when the rotation angle φn of the rotary knob component 150 matches any value, outputs an event that generates a click feeling by the mechanism of the knob device 15.

[0073] Next, the user interface device 1 executes an angle temporary storage step (S15). More specifically, the user interface device 1 substitutes the current rotation angle φn for the angle φn-1 acquired one time before. Then, the flow returns to step S12.

[0074] Hereinafter, an example of temperature adjustment of the air conditioner will be described. FIG. 11 shows the temperature setting operation of the air conditioner in the user interface device 1 and the corresponding image change. The user interface device 1 is configured such that the set temperature changes by 1°C for each occurrence of a mechanical click feeling by the knob device 15. The clockwise rotation (right rotation) of the rotary knob component 150 raises the set temperature, and the counterclockwise rotation (left rotation) lowers the set temperature.

[0075] In state 410, when the user taps the set temperature 13A of the left seat that is displayed, it is assigned to the knob device 15 for the purpose of setting the temperature of the air conditioner. The user, for example, rotates the rotary knob part 150 counterclockwise. The user interface device 1 decreases the set temperature by 1°C for each occurrence of the mechanical click feeling caused by the knob device 15. The current set temperature of the left seat is 22°C, and by rotating counterclockwise to generate two click feelings, the set temperature drops by 2°C. State 420 indicates the state where the set temperature of the left seat has dropped to 20°C.

[0076] From state 420 where the set temperature of the left seat is 20°C, the user further rotates the rotary knob part 150 counterclockwise. The state of the user interface device 1 changes to state 430, and the set temperature of the left seat drops to 18°C.

[0077] After the set temperature of the left seat is set to 18°C, assume the user further rotates the rotary knob part 150 counterclockwise. The user interface device 1 drives the actuator 105 in synchronization with the click feeling generated by the mechanism of the knob device 15. The unique tactile feeling where the tactile feeling by the actuator 105 is added (superimposed) to the tactile feeling by the mechanism makes the user perceive that the set temperature is at the lower limit. At this time, the set temperature of the left seat is maintained at 18°C.

[0078] For example, after the user perceives the unique tactile feeling and rotates the rotary knob part 150 clockwise by one click, the set temperature changes from 18°C to 19°C. At this time, the vibration by the actuator 105 is not applied. A user who knows that the set temperature at which the unique tactile feeling occurs is 18°C can set the temperature to the desired value without looking at the screen of the user interface device 1 relying on the number of click feelings when rotating the knob clockwise.

[0079] FIG. 12 is a diagram for explaining the operation of the interface device 1 described with reference to FIG. 11. As in the example described with reference to FIG. 7, it is assumed that the user rotates the rotary knob component 150 counterclockwise at a constant angular velocity ω. Here, for the sake of explanation, it is assumed that the rotary knob component 150 is rotated at a constant angular velocity ω, but the actual rotational angular velocity is arbitrary.

[0080] Graph 310 shows the time change of the tangential shear force ft in the circumferential direction acting on the fingertip due to the mechanical vibration generated by the rotation of the rotary knob component 150 at the angular velocity ω. Graph 320 shows the rotation detection signal output within the user interface device 1 according to the rotation amount Δφ of the rotary knob component 150 corresponding to the periodic tactile sensation. Graph 330 shows the drive signal applied to the actuator 105. Graph 340 shows the shear force in the z direction acting on the fingertip due to the operation of the actuator 105. These are described with reference to FIG. 7.

[0081] The user interface device 1 monitors the angular position of the rotary knob component 150 based on the positions of the plurality of conductors 151 of the rotary knob component 150. When the rotary knob component 150 rotates counterclockwise by Δφ from the previous set temperature decrease, the set temperature of the left seat is decreased by 1°C. When the set temperature has not reached the lower limit value, the user interface device 1 maintains the vibration of the actuator 105 in a stopped state. That is, only the shear force fz due to the mechanism acts on the fingertip of the rotary knob component 150.

[0082] Due to the counterclockwise rotation of the rotary knob component 150, the set temperature of the air conditioner reaches the lowest temperature (e.g., 18 °C) at time T1. After time T1, the user interface device 1 starts the tactile presentation by the actuator 105 in response to the counterclockwise rotation of the rotary knob component 150. Specifically, the user interface device 1 outputs a drive signal that vibrates the actuator 105 in synchronization with the rotation event caused by the counterclockwise rotation of the rotary knob component 150 that continues after time T1. Thereby, a tactile sensation in which the tactile sensation by the mechanism of the knob device 15 and the tactile sensation by the vibration of the actuator 105 are superimposed is presented to the user.

[0083] In the above example, the set temperature of the air conditioner is decreased according to the user operation. The process of increasing the set temperature of the air conditioner is substantially the same as the process of decreasing the set temperature. The process of increasing the set temperature is different from the process of decreasing the set temperature in that the rotation direction of the rotary knob component 150 and the set temperature upper limit are referred to.

[0084] That is, due to the clockwise rotation of the rotary knob component 150, the set temperature of the air conditioner reaches the highest temperature (e.g., 30 °C) at a specific time T2. After time T2, the user interface device 1 starts the tactile presentation by the actuator 105 in response to the clockwise rotation of the rotary knob component 150. Specifically, the user interface device 1 outputs a drive signal that vibrates the actuator 105 in synchronization with the rotation event caused by the clockwise rotation of the rotary knob component 150 that continues after time T2. Thereby, the tactile sensation by the mechanism of the knob device 15 and the tactile sensation by the vibration of the actuator 105 are superimposed and presented to the user.

[0085] In the above example, while the air conditioner set temperature is between the lower limit and the upper limit, the vibration of the actuator 105 is stopped. In other examples, when the set temperature is not the upper limit value and the lower limit value, the user interface device 1 may vibrate in a manner different from the vibration modes at the upper limit value and the lower limit value. For example, a phase difference with respect to the amplitude, frequency, or click feeling of the drive pulse 331 by the mechanism may be provided. Also, the vibration mode when the set temperature is at the upper limit value and the vibration mode when it is at the lower limit value may be different.

[0086] FIG. 13 shows an example of a logical configuration for executing the air conditioner set temperature decrease process described with reference to FIGS. 11 and 12. The user interface device 1 includes a control device 50 in addition to the knob device 15, the display panel 10 with a touch detection function, and the actuator 105. The control device 50 executes the processes that the user interface device 1 should execute while controlling other devices. In this example, the control device 50 instructs the temperature adjustment device 61 of the set temperature in response to a user operation.

[0087] The control device 50 can include one or more arithmetic devices and one or more storage devices. The arithmetic device can include, for example, a processor, a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The storage device stores the programs and data used by the control device 50. The storage device can include a volatile or non-volatile memory. The storage device includes a work area used by the program.

[0088] The control device 50 operates as a functional unit (module) for controlling external devices including the display panel 10 with a touch detection function, the actuator 105, and the temperature adjustment device 61. In this example, the control device 50 operates as a display control unit 51, a touch panel control unit 52, an operation assignment unit 55, and a vibration control unit 57. The touch panel control unit 52 includes a knob operation determination unit 53 and a rotation detection signal output unit 54.

[0089] The operation assignment unit 55 assigns a specific operation function to the interface device 1 according to the user operation. In the example described with reference to FIGS. 11 and 12, the operation assignment unit 55 assigns the air conditioner temperature setting function to the interface device 1. The display control unit 51 generates a specific image according to the assigned operation function and the user operation, and displays it on the display panel 10 with a touch detection function.

[0090] The touch panel control unit 52 detects the touch position by the user on the display panel 10 with a touch detection function and the touch position by a plurality of conductors 151 (see FIG. 3) attached to the rotary knob device 15, and passes the information obtained from the detection result to the operation assignment unit 55 and the vibration control unit 57. Specifically, the knob operation determination unit 53 detects the touch position by a plurality of conductors 151 attached to the rotary knob device 15.

[0091] The rotation detection signal output unit 54 calculates the angle of the rotary knob component 150 from the touch positions of the plurality of conductors 151 of the rotary knob component 150, and passes the information to the operation assignment unit 55. The operation assignment unit 55 executes the angle-temperature conversion process 56 to determine the set temperature (set temperature change) corresponding to the angle (angle change) of the rotary knob component 150, and transmits it to the temperature adjustment device 61 and the display control unit 51. The temperature adjustment device 61 controls the air conditioner so as to reach the specified temperature. The display control unit 51 generates an image corresponding to the specified temperature.

[0092] The rotation detection signal output unit 54 generates a rotation event pulse corresponding to a predetermined rotation amount Δφ of the rotary knob component 150 for the vibration control unit 57. The rotation event pulse generated in the rotation detection signal output unit 54 is synchronized with the predetermined rotation amount Δφ as described above. Further, the rotation detection signal output unit 54 acquires the information of the current set temperature from the operation assignment unit 55.

[0093] The rotation detection signal output unit 54 outputs a rotation event pulse (rotation event signal) for actuator control to the vibration control unit 57 under predetermined conditions. In this example, the rotation detection signal output unit 54 outputs a rotation event pulse to the vibration control unit 57 when the set temperature is the lower limit value or the upper limit value and the rotation of the rotary knob component 150 is counterclockwise or clockwise. The rotation event pulse is synchronized with a predetermined rotation amount Δφ. The vibration control unit 57 outputs a drive signal to the actuator 105 to vibrate it according to the received rotation event pulse.

[0094] Next, an example of air volume adjustment of the air conditioner will be described. FIG. 14 shows the air volume setting operation of the air conditioner in the user interface device 1 and the corresponding image change. The user interface device 1 is configured such that the set air volume changes by one level for each occurrence of a mechanical click feeling by the knob device 15. The clockwise rotation (right rotation) of the rotary knob component 150 increases the set air volume, and the counterclockwise rotation (left rotation) decreases the set air volume.

[0095] In state 710, when the user taps the person image 14A of the left seat, the function of setting the air volume of the air conditioner is assigned to the knob device 15. The user, for example, rotates the rotary knob component 150 clockwise. The user interface device 1 increases the set air volume by one level for each occurrence of a mechanical click feeling by the knob device 15. The current set air volume level of the left seat is 1, and the set air volume rises by two levels by a clockwise rotation that generates two click feelings. State 720 shows the state where the set air volume of the left seat has risen to level 3.

[0096] From state 720 where the set air volume of the left seat is level 3, the user further rotates the rotary knob component 150 clockwise. The state of the user interface device 1 changes to state 730, and the set air volume of the left seat rises to level 5 (maximum level).

[0097] After the set air volume of the left seat is set to level 5, assume that the user further rotates the rotary knob part 150 clockwise. The user interface device 1 drives the actuator 105 in synchronization with the clicking feeling generated by the mechanism of the knob device 15. The unique tactile feeling in which the tactile feeling by the actuator 105 is superimposed on the tactile feeling by the mechanism makes the user perceive that the set air volume is at the upper limit. At this time, the set air volume of the left seat is maintained at level 5.

[0098] For example, after the user perceives the unique tactile feeling, if the rotary knob part 150 is rotated counterclockwise by one click, the set air volume changes from level 5 to level 4. At this time, the vibration by the actuator 105 is not applied. A user who knows that the set air volume at which the unique tactile feeling occurs is level 5 can set the air volume to the desired value without looking at the screen of the user interface device 1 relying on the number of clicking feelings when rotating the knob counterclockwise.

[0099] FIG. 15 is a diagram for explaining the operation of the interface device 1 described with reference to FIG. 14. Assume that the user rotates the rotary knob part 150 clockwise at a constant angular velocity ω. Here, for the sake of explanation, it is assumed that the rotary knob part 150 is rotated at a constant angular velocity ω, but the actual rotation angular velocity is arbitrary.

[0100] Graph 310 shows the time change of the tangential shear force ft in the circumferential direction acting on the fingertip due to the mechanical vibration generated by the rotation of the rotary knob part 150 at the angular velocity ω. Graph 320 shows the rotation detection signal output within the user interface device 1 according to the rotation amount Δφ of the rotary knob part 150 corresponding to the periodic tactile feeling. Graph 330 shows the drive signal applied to the actuator 105. Graph 340 shows the shear force in the z direction acting on the fingertip by the operation of the actuator 105.

[0101] The user interface device 1 monitors the angular position of the rotary knob component 150 based on the positions of the plurality of conductors 151 of the rotary knob component 150. When the rotary knob component 150 rotates clockwise by Δφ from the previous set air volume increase, the set air volume of the left seat is increased by one level. The user interface device 1 maintains the vibration of the actuator 105 in a stopped state when the set air volume has not reached the upper limit value. That is, only the shear force fz by the mechanism acts on the fingertip of the rotary knob component 150.

[0102] Due to the clockwise rotation of the rotary knob component 150, the set air volume of the air conditioner reaches the maximum air volume level (for example, level 5) at time T5. After time T5, the user interface device 1 starts the tactile presentation by the actuator 105 in response to the clockwise rotation of the rotary knob component 150. Specifically, the user interface device 1 outputs a drive signal for vibrating the actuator 105 in synchronization with the rotation event caused by the clockwise rotation of the rotary knob component 150 continued after time T5. Thereby, the tactile sensation by the mechanism of the knob device 15 and the tactile sensation by the vibration of the actuator 105 are superimposed and presented to the user.

[0103] The above example increases the set air volume of the air conditioner according to the user operation. The process of decreasing the set air volume of the air conditioner is substantially the same as the process of increasing the set air volume. The process of decreasing the set air volume is different from the process of increasing the set air volume in that the rotation direction of the rotary knob component 150 and the lower limit of the set air volume are referred to.

[0104] That is, due to the counterclockwise rotation of the rotary knob component 150, the set air volume of the air conditioner reaches the lowest level (for example, level 0) at a specific time T6. After the time T6, the user interface device 1 starts the tactile presentation by the actuator 105 in response to the counterclockwise rotation of the rotary knob component 150. Specifically, the user interface device 1 outputs a drive signal for vibrating the actuator 105 in synchronization with the rotation event caused by the counterclockwise rotation of the rotary knob component 150 continued after the time T6. Thereby, the tactile sensation by the mechanism of the knob device 15 and the tactile sensation by the vibration of the actuator 105 are superimposed and presented to the user.

[0105] In the above example, the vibration of the actuator 105 stops while the set air volume of the air conditioner is between the lower limit and the upper limit. In other examples, the user interface device 1 may vibrate in a manner different from the vibration modes at the upper limit value and the lower limit value when the set air volume is not the upper limit value and the lower limit value. For example, the amplitude, frequency of the drive pulse 331, or the phase difference from the click feeling by the mechanism may be provided. Also, the vibration mode when the set air volume is at the upper limit value and the vibration mode when it is at the lower limit value may be different.

[0106] FIG. 16 shows an example of a logical configuration for executing the air volume increase process of the air conditioner described with reference to FIGS. 14 and 15. Hereinafter, the differences from the configuration example shown in FIG. 13 will be mainly described. Compared with the logical configuration shown in FIG. 13, the process of the operation assignment unit 55 changes from the angle-temperature conversion process 56 to the angle-air volume conversion process 59, and the temperature adjustment device 61 is replaced by the air volume adjustment device 63.

[0107] In the example described with reference to FIGS. 14 and 15, the operation assignment unit 55 assigns the air conditioner air volume setting function to the interface device 1. The display control unit 51 generates a specific image according to the assigned operation function and the user operation, and displays it on the display panel 10 with the touch detection function. The operation assignment unit 55 executes the angle - air volume conversion process 59, determines the set air volume (set air volume change) according to the angle (angle change) of the rotary knob component 150, and transmits it to the air volume adjustment device 63 and the display control unit 51. The air volume adjustment device 63 controls the air conditioner so as to achieve the specified air volume. The display control unit 51 generates an image according to the specified air volume.

[0108] The rotation detection signal output unit 54 generates a rotation event pulse corresponding to a predetermined rotation amount Δφ of the rotary knob component 150 for the vibration control unit 57. The rotation event pulse generated in the rotation detection signal output unit 54 is synchronized with the predetermined rotation amount Δφ as described above. Further, the rotation detection signal output unit 54 acquires information on the current set air volume from the operation assignment unit 55.

[0109] The rotation detection signal output unit 54 outputs a rotation event pulse (rotation event signal) for actuator control to the vibration control unit 57 under a predetermined condition. In this example, the rotation detection signal output unit 54 outputs a rotation event pulse to the vibration control unit 57 when the set air volume is the lower limit value or the upper limit value and the rotation of the rotary knob component 150 is counterclockwise or clockwise. The rotation event pulse is synchronized with the predetermined rotation amount Δφ. The vibration control unit 57 outputs a drive signal to the actuator 105 to vibrate according to the received rotation event pulse.

[0110] Similar to the above temperature adjustment and air volume adjustment, the control device 50 may give a drive signal to the actuator 105 to vibrate only during the period when the preset conditions are satisfied. Thereby, the user can know the current state without visually recognizing the display image of the user interface device 1. The preset conditions can be set for each function assigned to the knob device 15.

[0111] For values that can be adjusted by user operation in the functions assigned to the knob device 15, a combination of a preset range and the operation direction of the rotary knob part 150 is preset. The control device 50 vibrates the actuator 105 in response to the operation of the rotary knob part 150 in the preset operation direction within the preset range.

[0112] In the above example, for knob operations in the direction of increasing the value when the adjustable value is the upper limit, such as the set temperature or the set air volume, and for knob operations in the direction of decreasing the value when the adjustable value is the lower limit, vibration by the actuator 105 is added. Note that the actuator vibration may always be given in the same manner for the knob operation.

[0113] As described above, the embodiments of the present application have been described, but the present disclosure is not limited to the above embodiments. A person skilled in the art can easily change, add, or convert each element of the above embodiments within the scope of the present disclosure. It is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment.

Description of Reference Numerals

[0114] 15 Knob device 10 Display panel with touch detection function 50 Control device 61 Temperature control device 63 Air volume control device 105 Actuator 151 Conductor 150 Rotary knob part

Claims

1. A user interface device, comprising: a knob device having a mechanical structure that presents a sense of touch for each predetermined knob operation amount; a panel on which the knob device is installed; a control device; a vibration generator that can be controlled by the control device and vibrates the knob device; and the control device controls the vibration generator to vibrate the knob device in synchronization with the predetermined knob operation amount, thereby presenting a sense of touch in addition to the sense of touch by the mechanical structure. A user interface device.

2. The user interface device according to claim 1, wherein the vibration of the knob device synchronized with the predetermined knob operation amount is given for each predetermined knob operation amount. A user interface device.

3. The user interface device according to claim 1, wherein the drive pulse of the vibration generator for the vibration of the knob device synchronized with the predetermined knob operation amount is composed of a plurality of isolated sine waves. A user interface device.

4. The user interface device according to claim 1, wherein the vibration generator is an actuator disposed on the panel and driven by a drive signal. A user interface device.

5. The user interface device according to claim 1, wherein the vibration of the knob device by the vibration generator is given only when a preset condition is satisfied. A user interface device.

6. The user interface device according to claim 5, wherein the control device assigns one function selected from a plurality of functions to the knob device, for a value adjustable by a user operation in the one function, a combination of a preset range and an operation direction of the knob device is preset, and the control device vibrates the vibration generator in response to the operation of the knob device in the preset operation direction within the preset range. A user interface device.

7. The user interface device according to claim 5, wherein an adjustment function of a set temperature by an air conditioner is assigned to the knob device, and the control device vibrates the vibration generator in response to the operation of the knob device to increase the set temperature when the set temperature is an upper limit value and the operation of the knob device to decrease the set temperature when the set temperature is a lower limit value. User interface device.

8. A tactile presentation control method in a user interface device, wherein the user interface device includes a knob device having a mechanical structure that presents a tactile sensation for each predetermined knob operation amount, a panel on which the knob device is installed, and a vibration generator that vibrates the knob device, and the tactile presentation control method presents a tactile sensation in addition to the tactile sensation by the mechanical structure by controlling the vibration generator to vibrate the knob device in synchronization with the predetermined knob operation amount. Tactile presentation control method.

Images