Method and apparatus for providing a user interface
The method and device provide haptic feedback by simulating physical properties of graphical objects on a touch-sensitive display, addressing the lack of tactile confirmation in vehicle systems and improving driver interaction safety.
Patent Information
- Application Number
- DE102009032068
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2009-07-07
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2029-07-07
AI Technical Summary
Existing vehicle operating systems lack haptic feedback, which is crucial for safe and intuitive operation by drivers, especially when using touchscreens that do not provide tactile confirmation of actions.
A method and device that assign virtual properties like mass, friction, and elasticity to graphical objects on a touch-sensitive display, simulating physical interactions through haptic feedback by moving the display surface in the plane or perpendicular to it, providing tactile cues based on the virtual properties.
Enhances user interaction by simulating physical properties of displayed objects, offering tactile confirmation and enhancing the sense of touch, making operations more intuitive and safer for drivers.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The present invention relates to a method for providing a user interface. The user interface comprises a display area and a touch-sensitive surface. This surface is characterized in that it can be moved back and forth in the plane of the surface in order to exert haptic feedback on an actuating element that touches the surface. In the method, a graphical object is displayed on the display area. Furthermore, the position at which the touch-sensitive surface is touched by the actuating element is detected.In the method, if the actuating element touches the touch-sensitive surface at a position corresponding to the position at which the graphical object is displayed and the actuating element is subsequently moved on the touch-sensitive surface, the position at which the graphical object is displayed is moved together with the position at which the actuating element touches the touch-sensitive surface.
[0002] The invention further relates to a device for providing a user interface. The device has a display device with a display area. It also has a touch-sensitive surface. Furthermore, the device comprises a control device with which a position can be detected at which the touch-sensitive surface is touched with an actuating element. Furthermore, the control device can control the display device such that a graphic object can be displayed on the display area, wherein the position at which the graphic object is displayed can be displaced together with the position at which the actuating element touches the touch-sensitive surface.Finally, the device comprises a feedback device controllable by the control device, with which the touch-sensitive surface can be displaced back and forth in the plane of the surface in order to exert haptic feedback on the actuating element that touches the surface. The method and the device according to the invention are used in particular for providing a user interface in a vehicle.
[0003] A vehicle typically contains various control elements for controlling vehicle features. For example, mechanical pushbutton switches may be provided to turn a window heater or air conditioning system on and off. Furthermore, mechanical rotary switches may be provided to adjust, for example, the temperature in the vehicle's interior and the strength of the ventilation fan. Furthermore, a variety of other control elements for a navigation system, driver assistance systems, and communication and multimedia applications may be provided in the vehicle.
[0004] Due to the increase in electronic devices in vehicles, it is no longer possible to provide separate mechanical controls for all devices. Therefore, multifunction control systems are often used in vehicles to control the devices using fewer controls. Such multifunction control systems typically include a multifunction display and a central input device. The central input device can, for example, comprise a rotary push-button switch, which can be used to operate buttons displayed on the display device.
[0005] Furthermore, it was proposed to equip the display surface itself with a touch-sensitive surface, thus providing a so-called touchscreen. With such a touchscreen, operation is achieved by the user touching the touchscreen with an actuating element, such as their fingertip. The position of the touch is recorded, evaluated, and assigned to an operating process. Virtual operating elements, such as mechanical pushbuttons, slide switches, or rotary switches, can be displayed on the display surface for the operating processes. A display device with a touch-sensitive surface, used in conjunction with a navigation system, is described, for example, in DE 10 2005 020 155 A1.
[0006] Very specific requirements arise for control systems in a vehicle, as operation is carried out, among other things, by the driver. The mechanical controls originally used could be operated blindly by the driver, meaning the driver didn't have to take their eyes off the road to operate the system. They knew where the mechanical controls were located. Furthermore, haptic feedback was provided when operating the mechanical controls, allowing the driver to determine whether they had performed the desired operation. Control systems that use a touchscreen as an input device do not provide such haptic feedback.
[0007] US Pat. No. 7,148,875 B2 discloses a device for a computer user interface that provides haptic feedback to the user. To provide the haptic feedback, a touch-sensitive surface can be moved back and forth, particularly in a direction perpendicular to the surface. If the user performs certain operations on the touch-sensitive surface, specific haptic feedback is provided, which can also depend on the content displayed on a display.
[0008] Furthermore, devices are known from US 6,219,032 B1 and US 7,199,790 B2 in which haptic feedback is provided for external control elements, such as a joystick or a control mouse.
[0009] From WO 2008 / 037275 A1 a method and a device according to the preamble of claim 1 and claim 10 respectively are known.
[0010] US 2006 / 0209037 A1 describes a transparent layer arranged over a display surface. This layer is movable, and a mechanical force can be applied to the layer by an actuator. The position of a user's touch on the layer is detected by a position sensor, and the force required to move it varies depending on the position.
[0011] The present invention is based on the object of providing a method and a device of the type mentioned above, with which a user is given haptic feedback which supports him in an operating process which is carried out in connection with a graphic object which is displayed on the display surface.
[0012] This object is achieved according to the invention by a method having the features of claim 1 and a device having the features of claim 10. Advantageous embodiments and further developments emerge from the dependent claims.
[0013] In the method according to the invention, a virtual mass is assigned to the graphic object, and when the graphic object is moved, haptic feedback is applied to the actuating element, which feedback depends on the virtual mass of the graphic object. The display area, with the touch-sensitive surface, can be moved back and forth in the plane of the surface in order to apply the haptic feedback to an actuating element that touches the surface. The haptic feedback is designed in particular such that forces are applied to the actuating element that correspond to forces that occur when, instead of the graphic object, a physical object with a mass corresponding to the virtual mass of the graphic object is moved.
[0014] Through the haptic feedback in the method according to the invention, the user is given the feeling that the graphical object displayed on the display surface is a physical object that can be moved. The haptic feedback simulates the movement of a mass. Physical properties of the mass, such as inertia, are conveyed through the haptic feedback. If a physical object is moved in a direction, it must be accelerated. During the acceleration process, a counterforce is exerted on the actuating element. This counterforce is simulated in the method according to the invention by the haptic feedback. Corresponding forces that would occur when braking a physical object are simulated by the haptic feedback during the corresponding movement of the graphical object.
[0015] According to a further embodiment of the method according to the invention, a virtual friction coefficient is assigned to the touch-sensitive surface. In this case, the haptic feedback is configured such that forces are exerted on the actuating element that correspond to forces that occur when, instead of the graphic object, a physical object with a mass corresponding to the virtual mass of the graphic object is moved over a surface with a friction coefficient corresponding to the virtual friction coefficient. In this case, haptic feedback is provided not only during an accelerated movement of the graphic object, but also during an unaccelerated movement of the graphic object on the display surface.
[0016] According to a further embodiment of the method according to the invention, a virtual modulus of elasticity is assigned to the graphical object. Furthermore, obstacles are displayed on the display surface. In this case, the haptic feedback is designed such that forces are exerted on the actuating element that correspond to forces that occur when, instead of the graphical object, a physical object with a modulus of elasticity corresponding to the virtual modulus of elasticity of the graphical object is scattered on real obstacles arranged near the obstacles displayed on the display surface. In this case, not only the inertia of the mass and the friction on a surface are simulated by the haptic feedback, but also the reflection behavior of an elastic body.
[0017] According to a further embodiment of the method according to the invention, the touch-sensitive surface can further be displaced back and forth in a direction perpendicular to the plane of the surface in order to exert haptic feedback on the actuating element that touches the surface. In this case, for example, positions of the display surface can be assigned virtual height information depending on the display content. Upon movement of the actuating element on the touch-sensitive surface, haptic feedback is exerted on the actuating element, which depends on the virtual height information at the position of the display surface that corresponds to the position at which the actuating element touches the surface. In this way, a user can, for example, feel the height profile of a digital map of a navigation system using the haptic feedback.
[0018] According to a further embodiment of the method according to the invention, an operating object is displayed on the display surface, which can be actuated by moving the actuating element on the touch-sensitive surface. When the operating object is actuated, three-dimensional feedback is exerted on the actuating element. When the actuating element is actuated, in particular, a graphical rotation of the operating object is reproduced. The operating object can, for example, be a graphically represented roller, a rotary wheel, or a carousel, which are rotated in three dimensions. In addition to a counterforce acting in the direction of the plane of the touch-sensitive surface, the rotational movement in three dimensions can be simulated by raising and lowering the touch-sensitive surface.
[0019] The touch-sensitive surface is formed, in particular, on the display surface. This creates a so-called touchscreen. In this case, the actuating element is, in particular, the user's fingertip. Furthermore, the haptic feedback can also be transmitted to the user's hand via another actuating element.
[0020] In the device according to the invention, a virtual mass is assigned to the graphic object, and when the graphic object is moved by means of the feedback device, haptic feedback can be applied to the actuating element, which feedback depends on the virtual mass of the graphic object. Using a feedback device controllable by the control device, the display area with the touch-sensitive surface can be moved back and forth in the plane of the surface in order to apply the haptic feedback to the actuating element that touches the surface.
[0021] With the device according to the invention, the steps of the above method can be carried out partially or completely. In particular, the control device is designed to generate the haptic feedback of the method described above.
[0022] According to a further development of the device according to the invention, the touch-sensitive surface can further be displaced back and forth by means of the feedback device in a direction which is perpendicular to the plane of the surface in order to exert haptic feedback on the actuating element which touches the surface.
[0023] The invention will now be explained using an embodiment with reference to the drawing.
[0024] The single figure shows schematically an embodiment of the device according to the invention.
[0025] The device described below for providing a user interface is housed in a vehicle, in particular a motor vehicle. However, it can also be incorporated into any other device intended to provide the user with haptic feedback.
[0026] The device comprises a display device 1 with a display surface 2. The display surface 2 is arranged in the interior of a vehicle in such a way that it is clearly visible and easily accessible to at least one vehicle occupant, in particular the driver. The display surface 2 can be provided by a display, in particular a liquid crystal display, of any design.
[0027] The display device 1 is coupled to a control device 3. The control device 3 generates graphic data that can be displayed by the display device 1. For example, a graphic object 5 can be displayed on the display surface 2 of the display device 1.
[0028] A touch-sensitive surface is provided on the display surface 2 for user inputs. For example, a film can be arranged over the display surface 2, with which the position of a touch of an actuating element can be detected. The actuating element is in particular the user's fingertip 4. The film can be designed, for example, as a resistive touch film, capacitive touch film, or piezoelectric film. Furthermore, the film can be designed such that a heat flow emanating, for example, from the user's fingertip 4 is measured. Various inputs can be obtained from the temporal development of the touch of the film. For example, in the simplest case, touching the film at a specific position can be detected and assigned to a graphic object 5 displayed on the display surface 2. Furthermore, sliding movements of the fingertip 4 across the film can be interpreted.In particular, the user can define a line on the display surface 2 by touching the film at one position, sliding to another point on the film, and removing the fingertip 4 from the film at another position. The display device 1 is thus designed as a touchscreen.
[0029] The display area 2 with the touch-sensitive surface, which is touched by the user's fingertip 4, is also movable back and forth in the plane of the surface, which in this case coincides with the plane of the display area 2. In the coordinate system shown in the figure, the touch-sensitive surface is thus movable back and forth in the xy plane. Furthermore, the touch-sensitive surface can also be movable back and forth in the z direction, i.e., in a direction perpendicular to the xy plane.
[0030] To move the touch-sensitive surface, the display device 1 can include a feedback device with multiple actuators. The actuators cause the display surface 2 with the touch-sensitive surface to move relative to a fixed base of the display device 1. This movement exerts a force on the fingertip 4, which touches the touch-sensitive surface. The force is perceived by the user as haptic feedback. The actuators can be piezoelectrically operated, for example. Further details and possibilities for moving the touch-sensitive surface back and forth are described in US Pat. No. 7,148,875 B2.
[0031] The control device 3 detects, on the one hand, the position at which a user's fingertip 4 touches the touch-sensitive surface on the display area 2. Furthermore, it detects a movement of the fingertip 4 on the touch-sensitive surface. On the other hand, the control device 3 controls the actuators of the feedback device, which move the touch-sensitive surface in the xy plane or additionally in the z direction to provide haptic feedback.
[0032] The following describes in detail how the control device 3 controls the haptic feedback depending on an actuation of the touch-sensitive surface: For example, a graphic object 5 is displayed on the display surface 2 by means of the control device 3. The user can perform various operations with this graphic object 5. Among other things, they can move the graphic object 5 on the display surface 2, for example in the direction of arrow A, by touching the touch-sensitive surface at a position at which the graphic object 5 is displayed on the display surface 2. The user then moves their fingertip 4 on the touch-sensitive surface in the direction of arrow A. If such an operation is detected by the control device 3, it controls the display surface 2 such that the position of the graphic object 5 on the display surface 2 is moved together with the position at which the fingertip 4 touches the touch-sensitive surface.When the fingertip 4 is moved on the touch-sensitive surface, starting at the position of the graphic object 5, the graphic object 5 is always located at the fingertip 4. During this movement, the user is given the following haptic feedback:. The graphical object 5 is assigned a virtual mass, which is stored, for example, in the control device 3. If the user performs operations using the touch-sensitive surface that result in the position of the graphical object 5 on the graphical display area 2 being changed, haptic feedback is exerted on the user's fingertip 4, which depends on the virtual mass of the graphical object 5. If the graphical object 5 is accelerated in the direction of arrow A due to an accelerated movement of the fingertip 4 in this direction, a force is exerted on the fingertip 4 that corresponds to the force that would occur if the user were to move a physical object with a mass corresponding to the virtual mass of the graphical object 5 with their fingertip 4 instead of the graphical object 5.For this purpose, the display area 2 with the touch-sensitive surface executes a movement in the xy plane opposite to the direction of arrow A. The user is thus given the impression of moving a physical object in the direction of arrow A. The inertial mass of such a physical object is simulated by the haptic feedback. For each accelerated movement of the graphic object 5, i.e., both a positive acceleration and a deceleration, haptic feedback is provided in the form of a force exerted on the fingertip 4.
[0033] Furthermore, the touch-sensitive surface itself can be assigned a virtual friction coefficient, which is stored, for example, in the control device 3. In this case, the haptic feedback can be configured such that, in addition to the forces corresponding to the virtual mass of the graphical object 5, forces are simulated that occur when a physical object with a mass corresponding to the virtual mass of the graphical object 5 is moved over a surface with a friction coefficient that corresponds to the virtual friction coefficient of the touch-sensitive surface. For the sum of these forces, a force is exerted on the user's fingertip 4 as haptic feedback by means of the movement of the touch-sensitive surface.
[0034] Finally, a virtual modulus of elasticity can be assigned to the graphical object 5. Obstacles which can reflect the graphical object 5 can be displayed on the display surface 2. For example, the boundaries of the display surface 2 can represent an obstacle. In this case, if the user moves the graphical object 5 towards an obstacle by moving the fingertip 4 on the touch-sensitive surface until it finally touches it, forces are exerted on the user's fingertip 4 as haptic feedback. These forces correspond to forces that occur when, instead of the graphical object 5, a physical object with a modulus of elasticity corresponding to the virtual modulus of elasticity of the graphical object 5 is scattered against real obstacles. Due to the haptic feedback, the user thus feels the graphical object 5 bouncing off an obstacle.This haptic feedback is also provided by means of the control device 3 and the feedback device by a movement of the display area 2 with the touch-sensitive surface in the xy plane.
[0035] If the display area 2 with the touch-sensitive surface can also be moved back and forth in the z-direction, the user can also be given haptic feedback in this direction. For example, if a geographical map is displayed on the display area 2, each position of the display area 2 can be assigned virtual height information that depends on the height information of the geographical map. If the user now moves their fingertip 4 over the touch-sensitive surface, haptic feedback is applied to the fingertip 4, which depends on this virtual height information at the position of the display area 2 that corresponds to the position at which the fingertip 4 touches the surface. For example, the display area 2 with the touch-sensitive surface moves away from the fingertip 4 when the fingertip 4 sweeps over a depression in the geographical map.However, if the fingertip 4 moves over a hill in the geographical map, the display area 2 is moved towards the fingertip 4.
[0036] Finally, by means of the control device 3, an operating object can be displayed on the display surface 2, which can be actuated by a movement of the fingertip 4 on the touch-sensitive surface. For example, by moving the fingertip 4 on the touch-sensitive surface, the user can rotate a graphically represented roller, a rotary wheel, or a carousel in perspective in space. During this movement, a virtual mass can be assigned to the operating object, as described above, so that a counterforce is exerted on the user's fingertip 4 as haptic feedback during the rotation. Furthermore, haptic feedback in the z-direction can be exerted on the user's fingertip 4 for the perspectively displayed three-dimensional movement of the operating object. LIST OF REFERENCE SYMBOLS 1 display device 2 display area 3 Control device 4 fingertips 5 graphic object
Claims
[1] A method for providing a user interface with a display area (2) and with a touch-sensitive surface, wherein the method comprises: - a graphic object (5) is displayed on the display area (2), - the position is detected at which the touch-sensitive surface is touched with an actuating element (4), and, - when the actuating element (4) touches the touch-sensitive surface at a position corresponding to the position at which the graphic object (5) is displayed, and the actuating element (4) is subsequently moved on the touch-sensitive surface, the position at which the graphic object is displayed is moved together with the position at which the actuating element (4) touches the touch-sensitive surface, wherein - a virtual mass is assigned to the graphic object (5) and - when the graphic object (5) is moved, a haptic feedback is applied to the actuating element (4), which depends on the virtual mass of the graphic object (5), characterized by , that the display surface (2) with the touch-sensitive surface can be moved back and forth in the plane of the surface in order to exert the haptic feedback on an actuating element (4) which touches the surface. [2] Method according to claim 1, characterized by that the haptic feedback is designed in such a way that forces are exerted on the actuating element (4) which correspond to forces which occur when, instead of the graphic object (5), a physical object with a mass which corresponds to the virtual mass of the graphic object (5) is moved. [3] Method according to claim 1 or 2, characterized bythat a virtual friction coefficient is assigned to the touch-sensitive surface and that the haptic feedback is designed such that forces are exerted on the actuating element (4) which correspond to forces which occur when, instead of the graphic object (5), a physical object with a mass which corresponds to the virtual mass of the graphic object (5) is moved over a surface with a friction coefficient which corresponds to the virtual friction coefficient. [4] Method according to one of the preceding claims, characterized bythat a virtual modulus of elasticity is assigned to the graphic object (5), that obstacles are displayed on the display surface (2) and that the haptic feedback is designed such that forces are exerted on the actuating element (4) which correspond to forces which occur when, instead of the graphic object (5), a physical object with a modulus of elasticity which corresponds to the virtual modulus of elasticity of the graphic object (5) is scattered on real obstacles which are arranged near the obstacles displayed on the display surface. [5] Method according to one of the preceding claims, characterized by that the touch-sensitive surface is further displaceable back and forth in a direction (z) which is perpendicular to the plane of the surface in order to exert haptic feedback on the actuating element (4) which touches the surface. [6] Method according to claim 5, characterized bythat positions of the display surface (2) are assigned virtual height information depending on the display content and that upon movement of the actuating element (4) on the touch-sensitive surface, haptic feedback is exerted on the actuating element (4), which depends on the virtual height information at the position of the display surface (2), which corresponds to the position at which the actuating element (4) touches the surface. [7] Method according to claim 5 or 6, characterized by that an operating object is displayed on the display surface (2), which can be actuated by a movement of the actuating element (4) on the touch-sensitive surface and that when the operating object is actuated, a three-dimensional feedback is exerted on the actuating element (4). [8] Method according to claim 7, characterized bythat when the actuating element (4) is actuated, a graphical rotation of the operating object is displayed. [9] Method according to one of the preceding claims characterized by that the touch-sensitive surface is formed on the display surface (2). [10] Device for providing a user interface with - a display device (1) having a display surface (2), - a touch-sensitive surface, and - a control device (3) with which a position can be detected at which the touch-sensitive surface is touched with an actuating element (4), and with which the display device (1) can be controlled such that a graphic object (5) can be displayed on the display surface (2), wherein the position at which the graphic object (5) is displayed can be displaced together with the position at which the actuating element (4) touches the touch-sensitive surface, wherein - a virtual mass is assigned to the graphic object (5) and - when the graphic object (5) is moved by means of the feedback device, a haptic feedback can be exerted on the actuating element (4), which depends on the virtual mass of the graphic object, characterized by , that with a feedback device controllable by the control device (3), the display surface (2) with the touch-sensitive surface can be moved back and forth in the plane of the surface in order to exert the haptic feedback on the actuating element (4) which touches the surface. [11] Device according to claim 10, characterized by that by means of the feedback device the touch-sensitive surface can further be displaced back and forth in a direction (z) which is perpendicular to the plane of the surface in order to exert haptic feedback on the actuating element (4) which touches the surface. [12] Device according to claim 10 or 11, characterized by that the touch-sensitive surface is formed on the display surface (2).
Citation Information
Patent Citations
Vehicle`s information e.g. time, displaying method, for use with navigation system, involves displaying information and / or data with geo position in section of geographical map, where relationship exists to geo position covered by section
DE102005020155A1
Method and system for providing haptic effects
US20060209037A1
Method for providing force feedback to a user of an interface device based on interactions of a controlled cursor with graphical elements in a graphical user interface
US6219032B1
Haptic feedback for touchpads and other touch controls
US7148875B2
Providing force feedback to a user of an interface device based on interactions of a user-controlled cursor in a graphical user interface
US7199790B2
