Method for detecting an operating input and operating device

Description

[0001] The invention relates to a method for detecting an operating input carried out with at least one finger according to the type defined in more detail in the preamble of claim 1. Furthermore, the invention relates to an operating device for carrying out the method according to the invention.

[0002] A sensor system for detecting an operating input, as can be used in the method according to the invention, is known, for example, from DE 10 2011 112 567 A1 by the applicant. The operating device for a motor vehicle described therein comprises a transmitting unit for transmitting a signal to the finger and at least one receiving unit for receiving a signal reflected by the finger. The type of signals is, in principle, irrelevant as long as the corresponding signals can be transmitted, reflected by the finger, and received again by the sensor system. They can, in particular, be in the form of optical signals, which are then recorded accordingly by a camera, both in the visible range and in the infrared range. Ultrasonic signals, etc. are also conceivable. The sensor system is designed such that it essentially detects various contact points of a contour of the finger.Such a sensor technology is also called optical finger navigation (OFN) or is designed, for example, as a fingerprint sensor.

[0003] The main advantage is that the finger is not moved across a larger surface like with a touchscreen, but rather is moved and rolled across a relatively small surface. The actual sensor is either point- or line-shaped, for example, a straight line in a one-dimensional sensor or two intersecting lines in a two-dimensional sensor.

[0004] The disadvantage of such sensors is essentially that feedback to the operator is not possible or only difficult to realize, so that the feedback is typically only realized through the reaction of, for example, a computer to the corresponding input, which makes it necessary for the person using the sensor to observe a screen if they want feedback.

[0005] For further prior art, reference can also be made to US 2019 / 0354185 A1. This document describes the generation of a standing wave that runs across the entire surface of, for example, a touchscreen, thereby counteracting the otherwise smooth surface of the touchscreen with varying resistance for a finger as it sweeps across the surface. The otherwise smooth surface can thus be set into a standing oscillation such that a user feels a kind of rippling of alternating wave troughs and crests with their finger surface, making it easier to assign a specific path to their movement. The aforementioned document then additionally uses deformations in the standing wave to detect the position of the finger on the touchscreen surface.

[0006] US2014 / 0118127 A1 discloses an operating device that generates haptic feedback, such as a periodic vibration, on a user interface when touched. The frequency of the vibration changes depending on the position touched on the user interface.

[0007] US2007 / 0236450 A1 also discloses an operating device that generates haptic feedback with virtual texture sensations by modulating the friction between the operating finger and the user interface. The haptic feedback is determined based on the touch position; in addition, derivatives of the detected finger position, such as speed and / or acceleration, detected finger pressure, and / or detected direction of finger movement, can also be included.

[0008] US 2011 / 0248916 A1 discloses an operating device in which tactile feedback with a first characteristic is provided when the touch occurs at a first position associated with a first selection option. Tactile feedback with a second characteristic is provided as soon as the touch is detected at a second location not associated with the first selection option. The tactile feedback, such as a vibration, can be modified depending on further criteria, such as information types, distances of an operating device from a selection option, presets, user profiles, historical user data, operating forces, or operating type.

[0009] Ma Ping ET AL: "56-2: Optical Touch Screen Integrated With Fingerprint Recognition" (SID Symposium Digest of Technical Papers, Vol. 48, No. 1, May 1, 2017) describes a possible integration of an optical touchpad with a fingerprint sensor.

[0010] The object of the present invention is to further improve a method for detecting an operating input with a sensor in the manner mentioned at the beginning and also to provide an operating device suitable for this purpose.

[0011] According to the invention, this object is achieved by a method having the features of claim 1, and in particular in the characterizing part of claim 1. Advantageous embodiments and further developments of the method emerge from the dependent claims. An operating device for carrying out the method according to the invention is specified in claim 6. Advantageous embodiments and further developments with regard to the operating device also emerge from the dependent claims.

[0012] The method according to the invention serves to detect an operating input performed with at least one finger by detecting various contact points of a finger contour via a sensor system on a sensor surface. This sensor system, designed as a fingerprint sensor, has at least one transmitting unit for transmitting a signal to the finger and at least one receiving unit for receiving a signal reflected from the finger. The sensor is therefore not designed as a large-area touch-sensitive surface, but rather as a largely point-based sensor that detects a rolling movement and / or sliding movement of a finger across the sensor, as is the case, for example, with fingerprint sensors or so-called optical finger navigation sensors.This has the advantage that the sensor can be made extremely small and can detect movement in one or two dimensions, which can in particular be perpendicular to each other and thus allow a planar movement resolution.

[0013] According to the invention, the sensor surface is set into surface vibrations by at least one actuator, which changes the frictional resistance between the finger and the sensor surface depending on the changing contact points of the finger's contour, i.e., ultimately, the rolling movement on and / or sliding movement of the finger across the point-like sensor surface. This enables haptic feedback to the user for the first time, since the different frictions can be sensed or felt by the finger, enabling reliable feedback to the user independent of optical or acoustic signals.

[0014] According to an advantageous development of the idea, however, this can additionally be supported by optical and / or acoustic signals to generate feedback, whereby the haptic feedback is in the foreground in the method according to the invention and is realized by introducing vibrations into the sensor surface.

[0015] In the method according to the invention, the surface vibration is varied over time such that a sequence of larger and smaller vibration amplitudes and / or frequencies results in the area below the contact points of the finger on the sensor surface. The frictional resistance between the finger and the sensor surface can be varied accordingly via the frequencies or, in particular, the vibration amplitudes, since a sensor surface moved further towards the finger by the vibrations offers greater frictional resistance to the finger than a surface further away from the finger. This effect can be achieved directly by controlling vibration amplitudes and, additionally or alternatively, by changing the frequencies and, for example, a resulting resonance behavior of the sensor surface or parts thereof.

[0016] A very advantageous embodiment of the method according to the invention can further provide for the change in the surface vibrations in the region below the finger support points to occur abruptly, continuously, or according to a predetermined function, in each case to achieve the representation of a desired haptic effect. The haptic sensation is achieved by a varying sequence of different frictional resistances or forces during the rolling of the finger on the sensor surface, associated with a sliding or sliding movement. Depending on the desired haptic effect, the transition between the individual frictional resistances and thus ultimately between the vibrations they cause can be designed in different ways.For example, sudden transitions can be created that simulate the click of a button, the movement of a cylindrical rotary switch by a notch, or similar desired haptic effects. Other effects can be more easily realized using ramps or curves that change the vibration amplitudes and / or frequencies, such as simulating the movement of a trackball or a joystick with increasing resistance the further the movement proceeds, or similar effects.

[0017] According to the present invention, the surface vibration is temporally varied such that the change in the surface vibrations occurs in such a way that the increase and decrease in vibration amplitudes and / or frequencies changes the frictional resistance or frictional forces in such a way that the effect of a roller rotating in one or two dimensions with a detent in its rotational movement is haptically simulated. This leads to a very simple and efficient application, such as can be used in the cockpit of a vehicle. A roller can thus be easily and efficiently simulated in the minimal installation space required by the sensor surface.A user can, for example, while concentrating on the traffic situation, operate this roller using only the haptic feedback, preferably with a detent of its rotational movement, in order to adjust brightness, headlight range, adjust a setting on a multimedia device, select a menu, or similar functions. The mechanical design is extremely simple and robust, as no moving parts are required. Nevertheless, the haptic feedback made possible for the first time by the constitutional process allows for an extraordinarily realistic representation of such a rotating roller, which the user perceives exactly as they would perceive a mechanical roller.This can be done in one dimension, i.e. with a direction of rotation in only one plane, or as a two-dimensional roller, which can be moved in both directions, for example perpendicular to it.

[0018] An exceptionally advantageous development of the method according to the invention can also provide for adaptive adjustment of the change in vibration amplitudes and / or frequencies based on a distance-time diagram of the movement of the contact points of the finger contour on the sensor surface. Such adaptive adjustment, for example, of the magnitude of the vibration amplitudes is possible because the sensor essentially records a distance-time diagram of the sliding and / or rolling movement of the finger contour on the sensor surface. If this distance-time diagram corresponds to the expected movement, for example, the movement of a rasterized roller, then this distance-time diagram will approximately follow the desired haptic feedback and thus the sequence generated by the surface vibration.If this is not the case, and the movement recorded via the distance-time diagram is not what would be expected based on the desired haptic effect, in particular if the movement of the finger is significantly more uniform than expected, then the desired haptic feedback has obviously not been sufficiently perceived by the user. Therefore, the haptic feedback can be made even more pronounced by increasing the change in the vibration amplitudes and / or frequencies. If, on the other hand, the movement is excessively strong or choppy, the strength of the haptic feedback can also be reduced. To do this, for example, the change in the vibration amplitude is reduced.The change is adaptive in such a way that, ultimately, depending on the user, an adjustment can be made in such a way that the user feels roughly the same haptic feedback depending on the sensitivity of his or her fingers, in order to always provide different people with roughly the same haptic feedback after a short period of use through the adaptive adjustment.

[0019] As already mentioned above, in addition to the haptic feedback, at least some of the changes in the vibration amplitudes and / or frequencies can be accompanied by acoustic and / or optical signals. These can be presented, for example, via a loudspeaker in the case of acoustic signals or via optical means such as a display, lighting elements or the like. Preferably, for example, the engagement of the aforementioned roller feedback simulated by the friction pattern is supported by an engagement sound corresponding to a mechanical roller. With the optical signals, it is also conceivable in principle to use this light simultaneously for optical feedback in the case of optical detection of the finger via the emission of visible light, for example by means of additional emission windows in the area of ​​the sensor surface or in areas adjacent to the sensor surface.

[0020] The operating device according to the invention for carrying out the aforementioned method in one of the embodiments now comprises such a sensor system for detecting various contact points of a finger contour on the sensor surface, which sensor system has a transmitting unit for transmitting a signal to the finger and at least one receiving unit for receiving a signal reflected from the finger. According to the invention, it is provided that at least two side edges of the sensor surface are each at least indirectly connected to at least one piezo actuator. Via these at least two piezo actuators, vibrations can thus be introduced into the surface of the sensor, which, according to a very advantageous development of the operating device according to the invention, is designed as an optical finger navigation sensor or as a fingerprint sensor.In an extremely small installation space, an operating device can be constructed very efficiently, which can be used for menu control or the like, for example, whereby simple, efficient and largely intuitive operation is possible thanks to the direct haptic feedback.

[0021] According to a very advantageous embodiment of the operating device according to the invention, it can be provided that two opposite side edges and two perpendicularly opposite side edges of the sensor surface are each at least indirectly connected to at least one piezo actuator. This arrangement, in which the perpendicularly opposite side edges of the sensor surface can be excited to oscillate via the actuator, now allows the realization of the structure already mentioned in the method according to the invention, for example, a two-dimensional roller of a joystick or the like, since the corresponding oscillations are generated in both surface directions of the sensor surface and different frictional resistances or frictional forces can be applied to the finger.

[0022] According to a very advantageous development of the invention, the operating device can also provide devices for at least indirectly outputting acoustic and / or optical signals, so that acoustic or optical feedback is possible in addition to the haptic feedback. This can occur directly in the area of ​​the sensor, as already described above for optical feedback via light signals. However, it can also occur indirectly, with the sensor indirectly providing the appropriate output, for example, by outputting signals for generating acoustic feedback to a media system, which then plays them back via the connected loudspeakers, particularly when used in a vehicle.

[0023] The vehicle, already mentioned several times, is also the preferred application of such an operating device and thus of the method according to the invention described above in one or more of the illustrated embodiments. In particular, a small operating device that provides haptic feedback is very advantageous, allowing the user to continue to concentrate on the traffic during operation.

[0024] Further advantageous embodiments of the method and the operating device also emerge from the exemplary embodiment, which is described in more detail below with reference to the figures.

[0025] Showing: Fig. 1 shows a schematic diagram of a sensor and its operation; Fig. 2 shows a representation of the sensor surface with actuators for generating a surface vibration; Fig. 3 shows a schematic representation of an operation of a roller with the associated force curve; and Fig. 4 shows the implementation of the embodiment from Fig. 3 into the control element according to the invention.

[0026] In the presentation of the Figure 1a schematic representation of a surface 1 with a sensor 2 can be seen, which detects various contact points of a contour of a finger 3, which is essentially guided in a partly stroking / sliding and partly rolling movement over the sensor 2, as indicated by the arrow and the dotted representations of the finger at a later point in the movement. Such a sensor 2 is often referred to as an optical finger navigation sensor (OFN) or a fingerprint sensor, or is used as such. The design and operation correspond to the state of the art. The advantage lies in the possibility of building the sensor 2 accordingly small, for example, in a linear manner for one-dimensional detection or, when detecting a movement perpendicular to the plane of the page, accordingly from two intersecting linear parts. This enables a very small sensor surface.

[0027] In the presentation of the Figure 2 The surface 1 of such an operating device 10 is again schematically indicated in a plan view. This surface, referred to below as the sensor surface 1, is provided on at least two, preferably all four, of its side edges with corresponding actuators 4—preferably piezo actuators 4—which can introduce a surface vibration 5, indicated here schematically, into the sensor surface 1 in order to generate different frictional resistances or forces FR through the different vibration states when the sensor surface 1 is touched with a finger 3. This enables haptic feedback of the actuation of such a sensor in the structure referred to overall as the operating element 10.

[0028] In the following, this will be illustrated using the example of the actuation of a roller, purely as an example in one dimension, whereby with the arrangement of piezo actuators 4 on all four sides of the sensor surface 1, this can also be extended accordingly in two dimensions.

[0029] In the presentation of the Figure 3 Now, by way of example, a roller 6 is indicated in a surface 7. This is moved accordingly by the finger 3, for example by rotating it through an angle of rotation α, in particular in the representation of the Figure 3 in the direction of the arrow. The roller 6 itself should have a mechanical detent, so that the movement of the roller 6 occurs in individual, successive movements, which are interrupted in between by the respective detent and a concomitant increase in friction. The force F occurring in this case is shown in the illustration of the Figure 3The diagram on the right shows the angle of rotation α for three detents, for example. The mechanical structure of roller 6 with its detents thus results in the angle of rotation α shown in Figure 3 The force curve shown on the right, which in a mechanical design provides immediate feedback to the finger 3 and thus ultimately to the user, who operates the roller 6 with his finger 3.

[0030] This structure and the haptic feedback can now be easily and efficiently replicated by the operating device 10, without mechanical components that could wear out. In the illustration of the Figure 4Finger 3 can again be seen, which is located above the operating device 10 and actuates it in the manner customary in optical finger navigation, in that the finger 3 is again moved, for example, in one dimension above the sensor surface 1, in particular partly stroked and partly rolled. Of the actual sensor technology, a transmitting unit 8 for transmitting a signal and a receiving unit 9 for receiving the signal reflected by the finger 3 can be seen below the sensor surface 1. The detailed structure will not be discussed further; it can be found, for example, in the applicant's generic document mentioned at the beginning.

[0031] In addition to the known sensor design, the piezo actuators 4, two of which can be seen here, are now present. These are directly or indirectly connected to the sensor surface 1 and can introduce the surface vibration 5 already indicated above into the sensor surface 1. Depending on the desired haptic feedback from the sensor surface 1, this surface vibration 5 is now adjusted accordingly. For example, if the haptic feedback of the roller 6, as shown in the illustration of the Figure 3 explained, the vibrations are generated, for example, according to the Figure 4The diagram shown top right is shown. In this diagram, the amplitude x of the surface oscillation 5 is shown over the path s of the stroking and / or rolling of the finger 3 or its fingertip over the sensor surface 1. In this case, oscillations with different amplitudes are alternated in such a way that the friction force FR of the finger 3 on the sensor surface 1 during its movement changes in the Figure 4 as shown below right. This allows the mechanical behavior of the roller 6 to be realized relatively precisely as haptic feedback from the operating device 10 to the finger 3.

[0032] In addition to the purely haptic feedback via the surface vibration 5 of the sensor surface 1, a change in the surface vibration 5, for example a change in the vibration amplitudes, can also be indicated via acoustic and / or optical signals, which supplement or accompany the haptic feedback. To indicate this, Figure 4 Below the operating device 10, purely by way of example, a loudspeaker 11 is indicated as a symbol for the additional possible acoustic feedback, and an LED 12 (light-emitting diode) is indicated as a symbol for possible additional optical feedback. In addition to pure light signals via the indicated LED 12, the optical feedback can also be provided via a display on a screen, for example, by advancing a menu located there and displaying the next menu item as selected accordingly, or the like. All of this would fall under optical feedback for the purposes of this description.

[0033] With minimal installation space and without the need to install mechanical parts whose rotation or other movements must be detected, an optical finger navigation sensor can now be very easily combined with reliable haptic feedback. The haptic feedback can be adjusted almost arbitrarily by adjusting the oscillation amplitude x and / or the frequency of the oscillation, for example to adjust the aforementioned roller 6 or any other type of mechanical movement by varying the friction force FR between the finger 3 and the sensor surface 1. This allows, for example, a roller 6, a button, a joystick, a trackball, or the like to be simulated with regard to its feedback by the haptic feedback generated by the surface oscillation 5 in the sensor surface 1.

Claims

1. Method for detecting an operating input performed by at least one finger (3) by identifying different contact points of a contour of the finger (3) using a sensor system on a sensor surface (1), with at least one transmission unit (8) for transmitting a signal to the finger (3) and at least one receiving unit (9) for receiving a signal reflected by the finger (3), wherein a surface vibration (5) is introduced into the sensor surface (1) of the sensor system designed as a fingerprint sensor with at least one actuator (4), by means of which vibration the frictional resistance (FR) between the finger (3) and the sensor surface (1) changes depending on the changing contact points of the contour of the finger (3), wherein the surface vibration (5) is changed over time such that a sequence of larger and smaller vibration amplitudes (x) and / or frequencies is produced in the region below the contact points of the finger (3) on the sensor surface (1).

2. Method according to claim 1, characterized in that the change in the surface vibration (5) in the region below the contact points of the finger (3) on the sensor surface (1) is carried out abruptly, continuously or according to a predetermined function, in each case to produce a desired haptic effect.

3. Method according to either claim 1 or claim 2, characterized in that the surface vibration (5) is changed in such a way that the frictional resistance (FR) is changed by the rise and fall of vibration amplitudes (x) and / or frequencies in such a way that the effect of a roller (6) rotatable in one or two dimensions with a detent of its rotational movement (α) is haptically reproduced.

4. Method according to any of claims 1 to 3, characterized in that the change in the vibration amplitudes (x) and / or frequencies is adaptively adjusted on the basis of a path-time diagram of the movement of the contact points of the contour of the finger (3) on the sensor surface (1), for which purpose the change is increased at least in the case of a movement that does not correspond to the desired haptic effect.

5. Method according to any of claims 1 to 4, characterized in that at least some of the changes in the vibration amplitudes (x) and / or frequencies are additionally accompanied by acoustic and / or optical signals.

6. Operating device (10) for carrying out the method according to any of claims 1 to 5, comprising a sensor system for identifying different contact points of a contour of the finger (3) on the sensor surface (1), including a transmission unit (8) for transmitting a signal to the finger (3) and at least one receiving unit (9) for receiving a signal reflected by the finger (3), wherein at least two side edges of the sensor surface (1) of the sensor system designed as a fingerprint sensor are each at least indirectly connected to at least one piezo actuator (4), wherein the piezo actuators (4) introduce a surface vibration such that the frictional resistance (FR) between the finger (3) and the sensor surface (1) changes depending on the changing contact points of the contour of the finger (3), wherein the surface vibration (5) is changed over time such that a sequence of larger and smaller vibration amplitudes (x) and / or frequencies is produced in the region below the contact points of the finger (3) on the sensor surface (1).

7. Operating device (10) according to claim 6, characterized in that two opposite side edges and two perpendicularly opposite side edges of the sensor surface (1) are each at least indirectly connected to at least one piezo actuator (4).

8. Operating device (10) according to either claim 6 or claim 7, characterized in that apparatuses (11, 12) for at least indirectly outputting acoustic and / or optical signals are provided.