Method and device for controlling a user interface

Abstract

Method for controlling at a user interface comprising an input device with means for controlling coordinates of a display area (2), wherein the input device comprises a touch-sensitive surface (5) of the display area (2) and, for controlling coordinates of the display area (2), the touch-sensitive surface (5) is touched by an actuating element, which is a user's fingertip, in which - a button (9, 10, 11) is displayed on the display area (2), with several interaction options assigned to the button (9, 10, 11), - wherein in an initial state a first interaction possibility is visualized by a graphic design of the button (9, 10, 11) in such a way that this first interaction possibility corresponds to a virtual push button, - by means of the input device, an initiating action is performed in conjunction with the button (9, 10, 11), in which the fingertip touches the touch-sensitive surface (5) at this virtual push button and the fingertip is not immediately removed, but is moved in a direction starting from the displayed button (9, 10, 11), and - the design of the button (9, 10, 11) changes so that a slide switch (13, 20, 23) is graphically displayed and, as an interaction option depending on the detected action, a continuous change of the controlled coordinates of the display area (2) is activated.

Description

[0001] The present invention relates to a method for controlling a user interface comprising an input device with means for controlling the coordinates of a display area. In the method according to the invention, a button is displayed on the display area, which can be controlled and actuated by means of an input device. The invention further relates to a device for controlling a user interface, comprising a display device with a display area, an input device comprising means for controlling the coordinates of the display area, and a control device. The display area can be controlled by means of the control device such that a button can be displayed, and the button can be controlled by means of the input device. Furthermore, several interaction options are assigned to the button. Finally, the control device can detect an initiating action performed in conjunction with the button.The method and device according to the invention are used in particular in a vehicle.

[0002] A vehicle typically displays various pieces of information, primarily vehicle-related data, which are presented visually to the driver. Traffic-related data can also be displayed. Previously, analog mechanical gauges were used to display this information. These were typically located within the instrument cluster, close to the driver's primary field of vision behind the steering wheel. The instrument cluster primarily displays the vehicle's speed, fuel level, coolant temperature, and other operational information.

[0003] With the increase in electronic systems in vehicles, it became necessary to display a greater amount of information within the vehicle. Modern vehicles, for example, include a variety of driver assistance systems, the information from which must be displayed. Furthermore, many vehicles are equipped with a navigation system. Such a navigation system can display digital geographic maps with a route and, if applicable, a wide range of additional information. Finally, modern vehicles often include communication and multimedia applications, including a mobile phone interface and systems for playing music and speech. The vehicle must also be able to display information for these applications.

[0004] To flexibly display the diverse information within the vehicle, a freely programmable display, for example, is used as the display surface. This display can be located in the area of ​​the conventional instrument cluster. Alternatively, it can be positioned, for instance, above the vehicle's center console, providing a multifunction display.

[0005] A multifunction display is typically operated in conjunction with an operator control system. The input device for the operator control system can include remote controls such as pushbuttons, rotary switches, or rotary push-buttons. A touch-sensitive surface can also be provided. This can be separate from the display area to provide a so-called touchpad. Alternatively, the touch-sensitive surface can be integrated into the display area to provide a so-called touchscreen. With a touchscreen, operation is achieved by the user touching the touchscreen, for example, with their hand. The position of the touch is detected, evaluated, and assigned to an operating action. To assist the user during operation, virtual switches can be displayed on the screen as graphical buttons.A display device with a touch-sensitive surface, which is used in conjunction with a navigation system, is described, for example, in DE 10 2005 020 155 A1.

[0006] From DE 10 2004 053 950 A1 a method for controlling a display device is known in which a control menu of a device of a vehicle is displayed by means of a display device if a touch of a control element of the device is identified.

[0007] DE 10 2007 029 618 A1 discloses a contactless display and control device for a vehicle. The device described in this publication comprises a display surface that can be operated in a display mode and an operating mode. Furthermore, a position-sensitive sensor unit is provided with which the position of an actuating element relative to a control element can be detected. When the control element is actuated, the user moves the actuating element into an activation volume in front of the actuating element. The operating mode of the display device then changes from display mode to operating mode. In operating mode, interactive control elements are displayed. DE 10 2006 061 778 A1 discloses another display and control device in which it is possible to switch back and forth between a display mode and an operating mode.

[0008] German patent DE 20 2007 018 368 U1 describes a multimedia communication device with a touchscreen that responds to gestures for controlling, manipulating, and editing media files. The device can detect the touch of a finger and assign it to a displayed image object. In this case, a selection action is performed. Furthermore, a double-tap or a finger touch that lasts longer than a predetermined duration can trigger a double-click action.

[0009] US 2009 / 0 058 822 A1 describes a user interface for handling video data.

[0010] DE 10 2007 045 967 A1 describes a method for the contactless input of characters.

[0011] US 2008 / 0 211 779 A1 describes a method that enables the reconfigurable control of vehicles.

[0012] Displaying information and operating the various vehicle systems presents very specific requirements. The driver is one of the key users of this information and its functions. Therefore, the information should be presented in such a way that it does not distract the driver while driving. The displayed information should be intuitive and quickly understandable, requiring only a very brief moment of distraction. Similarly, operating the vehicle's systems should be as simple and intuitive as possible, allowing the driver to use them even while driving.If operation is supported or guided by a display, the display should be designed so that the driver only needs to look at the display very briefly to perform the operation.

[0013] The present invention is based on the objective of providing a method and a device of the type mentioned at the outset in which the operation of buttons displayed on the display surface is simplified.

[0014] According to the invention, this problem is solved by a method with the features of claim 1 and a device with the features of claim 4. Advantageous embodiments and further developments are described in the dependent claims.

[0015] In the method according to the invention, a button is displayed on the display surface, and several interaction options are assigned to the button. An initiating action performed in connection with the button is detected by means of the input device, and at least one of the interaction options is activated depending on the detected action.

[0016] For the purposes of this invention, a button is understood to be a control element of a graphical user interface. A button differs from elements and areas used solely for information display, so-called display elements or display areas, in that it is selectable. When a button is selected, its associated function is executed. This function can only result in a change to the information display. Furthermore, buttons can be used to control devices whose operation is supported by the information display. Buttons can thus replace conventional mechanical switches. Buttons can be created and displayed anywhere on a freely programmable display area. Additionally, it may be possible to select a button. In this case, the associated function is not yet executed.The highlighted button is displayed in a way that stands out from other buttons. Highlighting and / or selecting a button can be done using cursor control or, according to the invention, by directly operating a touch-sensitive surface of the display.

[0017] In the context of the invention, an initiating action is understood to mean that a connection to a specific button has been established during an operating process. The user then performs a specific operating action. This operating action is initiated, i.e., started, by an action of the user. This initiating action is analyzed. Depending on the result of the analysis, a specific interaction option is activated from the multitude of possible interaction options.

[0018] When a button has multiple interaction options assigned to it, one interaction option is typically activated via a separate operating step. For example, a tool menu might allow the user to select the desired interaction option for the displayed buttons. However, if speed is particularly important in a user interface, as is the case in a vehicle, for instance, selecting an interaction option in a separate operating step is disadvantageous. According to the invention, the selection and activation of the desired interaction with the button is integrated into the operating process for pressing the button. Specifically, the interaction option is activated at the beginning of the operating process by an initiating action performed in conjunction with the button.

[0019] In the method according to the invention, the input device comprises a touch-sensitive surface. The touch-sensitive surface is formed, in particular, on the display area, thus providing a so-called touchscreen. To control coordinates of the display area, the touch-sensitive surface is touched by an actuating element. If a touchscreen is provided, the touch-sensitive surface of the display area is thus touched at the coordinates on the display area that are to be controlled. According to the invention, the actuating element is the fingertip of a user.

[0020] According to another embodiment of the method according to the invention, the input device includes a cursor control. In this case, a cursor displayed on the display surface is moved to control coordinates of the display surface. The user then operates, for example, a mechanical control element such as a joystick.

[0021] The initiating action that leads to the activation of at least one interaction option can, for example, be pressing the button using the input device. An initiating action to activate at least one interaction option occurs, in particular, when the button is pressed for a specific time interval. If the button is a virtual push button and the input is via a touch-sensitive surface of the display, pressing the button for a specific time interval corresponds to a long press of the push button.

[0022] According to another embodiment, an initiating action is performed using the input device. This gesture is executed particularly when a connection to a specific button has already been established.

[0023] In this case, the initial gesture of the operation activates a specific interaction option. If the input device includes a touch-sensitive surface, the user can, for example, perform a specific characteristic movement on the surface with their fingertip to activate the desired interaction option. Such a gesture is, in particular, a movement originating from the button. This movement can be performed using a cursor or directly on the display area.

[0024] According to another embodiment of the method according to the invention, the approach of an actuating element to the input device is detected, and the initiating action for activating at least one interaction possibility is the approach of an actuating element to the input device. If the input device comprises, for example, a touch-sensitive surface, the approach of the fingertip to this surface is used to activate an interaction possibility. The approach to the input device can be detected, in particular, with spatial resolution; that is, the position at which the actuating element approaches the input device is detected.

[0025] According to one embodiment of the inventive method, the button's display is changed depending on a detected action and the activated interaction option. In an initial state, a first interaction option is visualized by the button's graphic design. This first interaction option corresponds to a virtual push button. If the user now touches the touch-sensitive surface of the display area at this push button with their fingertip and does not immediately remove the fingertip, but instead moves it in a direction from the displayed button, the button's design changes, for example, so that a slide switch is graphically displayed. An interaction option associated with the button can thus be a continuous change in the controlled coordinates of the display area.Furthermore, an interaction option can be the selection of a button from several displayed buttons. In this case, multiple selectable buttons are graphically represented.

[0026] According to a further development of the method according to the invention, the user performing the action is detected. For example, the user of the actuating element, which is, for instance, the user's fingertip, can be detected. Furthermore, the user's preferred interaction options are stored. In this case, the selectable interaction options can depend on the user's preferred interaction options. In particular, the selectable interaction options are the user's preferred interaction options. According to this further development of the method according to the invention, the interaction options can thus be individually adapted to each user.

[0027] Finally, it should be noted that different buttons may have different interaction options. Furthermore, the available interaction options for a button may depend on the context in which the button is displayed.

[0028] In the device according to the invention for controlling a user interface, at least one of the interaction possibilities of the button can be activated by means of the control device depending on a detected action.

[0029] The control device according to the invention makes the steps of the above method partially or completely feasible.

[0030] According to one embodiment of the device according to the invention, this device comprises a proximity sensor with which the approach of an actuating element to the input device can be detected. Preferably, the proximity sensor can also detect the position at which the actuating element approaches the input device.

[0031] The input device may further comprise, for example, a touch-sensitive surface and / or a cursor control, as explained above with reference to the method according to the invention.

[0032] The invention will now be explained using an exemplary embodiment with reference to the drawings. Fig. Figure 1 shows an embodiment of the device according to the invention and the coupling of this device with other equipment of a vehicle, and the Fig. Figures 2 to 5 show displays that were generated by an embodiment of the method according to the invention.

[0033] The device for controlling a user interface is located, in particular, in a vehicle. A vehicle occupant can operate various vehicle functions via this user interface. The control device comprises a display device 1 with a display area 2, which is arranged in the vehicle's interior so that it is clearly visible to at least one vehicle occupant, in particular the driver. The display area 2 can be provided by a display, in particular a liquid crystal display, of any design.

[0034] The display device 1 is coupled to a control device 3. The control device 3 generates graphical data that can be displayed by the display device 1. For this purpose, the control device 3 is coupled to a memory 4 for storing data. The data includes, in particular, buttons 8, which can be displayed on the display surface 2 and through which the user can interact with the control device. Several interaction options are assigned to the buttons 8, which are stored in the memory 4.

[0035] Furthermore, the control device 3 is coupled to a vehicle bus 7, via which data can be exchanged with various vehicle systems. The user interface serves in particular to control these vehicle systems. The user interface can also be used to control the display device 1 itself.

[0036] Furthermore, an input device is provided by means of which the user can generate control signals. In this case, a touch-sensitive surface 5 is provided on the display area 2. The display device 1 thus comprises a so-called touchscreen. By touching the touch-sensitive surface 5 with an actuating element, such as a fingertip, the user can control specific coordinates of the display area 2. The touch-sensitive surface 5 detects the position at which it is touched. A control signal can then be generated, which depends in particular on the display content at the position on the display area 2 at which the touch-sensitive surface 5 was touched.

[0037] Alternatively, the touch-sensitive surface can also be provided separately from display area 2. In this case, a so-called touchpad is provided. Furthermore, a mechanical control element can also be provided with which a cursor displayed on display area 2 can be controlled.

[0038] Finally, a proximity sensor 6 is provided, which detects the approach of an actuating element to the input device, i.e., in the present embodiment, to the touch-sensitive surface 5. The proximity sensor 6 is connected to the control device 3. It transmits a signal to the control device 3 when an actuating element has approached the touch-sensitive surface 5 to such an extent that a preset threshold value is undershot.

[0039] Finally, the input device or proximity sensor 6 can be designed to detect the user operating the input device. Details of how a user or the seating position of a user operating a vehicle feature can be determined are described in WO 2008 / 025 407 A1.

[0040] The following refers to the Fig. Sections 2 to 5 describe an embodiment of the method according to the invention. The method can, in particular, be carried out by the control device described above: Using the control device 3, several buttons 9 to 11 are displayed on the display surface 2, each with a specific function. Button 9 is used to access a defined submenu, button 10 allows the volume of an audio output in the vehicle's interior to be adjusted, and buttons 11 are used to set the temperature for the driver and passenger sides of the vehicle. Buttons 9 to 11 have several interaction options. For example, button 9 is represented as a virtual push button. The user can briefly tap the touch-sensitive surface 5 at the position of button 9. This action displays a submenu associated with button 9.Furthermore, button 9 has a virtual slider switch assigned to it as an interaction option, which the user can move by moving their fingertip on the touch-sensitive surface 5. The following explains how an interaction option is activated: The starting point is the in Fig. The second example shown depicts a virtual push button assigned to button 9 as an interaction option. A brief tap of button 9 opens a submenu. This submenu contains further buttons, which the user can select by tapping again.

[0041] However, if the user touches button 9 with their fingertip and does not immediately release it, but moves their fingertip on surface 5, e.g., horizontally, the graphical representation of button 9 changes. It no longer displays the image shown in the image. Fig. The two virtual pushbuttons shown are not represented, but rather a graphical element 12, as shown in Fig. Figure 3 shows that touching surface 5 at button 9 and subsequently moving the point of contact on surface 5 is interpreted by the control device 3 as an initiating action performed in conjunction with button 9. This initiating action, in which the fingertip is moved from button 9 onto surface 5, activates a different interaction option for button 9. For this new interaction option, the [function / function] shown in Figure 3 is used. Fig. 3 shown graphic element 12 with the virtual slide switch 13.

[0042] The slide switch 13 comprises a scale with a total of five divisions and a marker 14, which the user can position on the scale by moving their fingertip on the touch-sensitive surface 5. Above the scale are five indicator elements 15 to 19, which visualize functions that correspond to the respective positions of the marker 14 on the scale of the slide switch 13.

[0043] To access a telephone function, for example, visualized by display element 15, the user touches button 9 with their fingertip and moves it slightly in one direction. Graphic element 12 then appears, displaying marker 14 of the slider switch at the position where the user's fingertip touches surface 5. The user continues moving their finger on the touch-sensitive surface 5 until marker 14 is positioned on the scale below display element 15, which corresponds to the telephone function. Once marker 14 is correctly positioned on the slider switch 13, the user releases surface 5. A telephone function is then accessed, i.e., a telecommunications menu is displayed.

[0044] In the method according to the invention, the user thus has various options for interacting with a button 9. Which interaction option is activated depends on an initiating action of the operating process that is carried out in connection with the respective button 9.

[0045] Other initiating actions for activating a specific interaction option, which are not explained in detail here, include, for example, touching a button for a certain time interval without moving on the touch-sensitive surface 5, or performing a specific gesture on the touch-sensitive surface 5.

[0046] Furthermore, the mere approach to the touch-sensitive surface 5 at a specific position, or the manner of approaching this position, can be interpreted by the control device 3 as an initiating action to activate a specific interaction option. If the user performing the action is identified, the initiating actions for activating the interaction options can also be defined in a user-specific manner. Furthermore, the interaction options can depend on the user and, if applicable, the context of use.

[0047] The user touches during the in Fig. In the display shown in Figure 2, if the fingertip touches button 10 and then moves the fingertip slightly, e.g., horizontally, this is again interpreted by the control device 3 as an initiating action for activating an interaction option. The action shown in Figure 2 triggers the following: Fig. Figure 4 shows a display in which a slide switch 20 with a marker 21 and an indicator element 22 for volume control are shown. Using this slide switch 20, the user can adjust the volume for an acoustic output in the vehicle interior.

[0048] Accordingly, the user can use button 11 to adjust the temperature for the driver's or passenger's side. A short tap of button 11 brings up a menu for the vehicle's climate settings. If, however, the user touches button 11 and does not immediately release the touch-sensitive surface 5, but instead moves their finger in a certain direction, the menu appears. Fig. 5 Display shown with a slide switch 23 with a marker 24 and a display element 25 which shows the target temperature.

[0049] Another interaction option displays several differentiated buttons within a general button area. For example, a "Media" button can be subdivided into buttons for the functions "Forward," "Stop / Play," and "Rewind" for playing music tracks when initiating an operation.

[0050] At the end of the interaction with a button, as shown in the Fig. When numbers 3 to 5 are displayed, the display jumps back to the one shown. Fig. 2. Display shown. Furthermore, after a defined time after releasing a button, the display can revert to its original view. REFERENCE MARK LIST 1 Display device 2 Display area 3 Control device 4 storage 5 touch-sensitive surfaces 6 Proximity sensor 7-vehicle bus 8 button 9-11 buttons 12 graphic element 13 virtual slide switches 14 Marking 15-19 Display elements 20 virtual slide switches 21 Mark 22 Display element 23 virtual slide switches 24 Marking 25 Display element

Claims

Method for controlling a user interface comprising an input device with means for controlling coordinates of a display surface (2), wherein the input device comprises a touch-sensitive surface (5) of the display surface (2) and, for controlling coordinates of the display surface (2), the touch-sensitive surface (5) is touched by an actuating element, which is a user's fingertip, wherein: - a button (9, 10, 11) is displayed on the display surface (2), the button (9, 10, 11) being associated with several interaction possibilities; - wherein, in an initial state, a first interaction possibility is visualized by a graphic design of the button (9, 10, 11) such that this first interaction possibility corresponds to a virtual push button; - an initiating action performed in connection with the button (9, 10, 11) is detected by means of the input device.where the fingertip touches the touch-sensitive surface (5) of this virtual switch and the fingertip is not immediately removed, but is moved in a direction starting from the displayed button (9, 10, 11), and the design of the button (9, 10, 11) changes so that a slide switch (13, 20, 23) is graphically displayed and, as an interaction option depending on the detected action, a continuous change of the controlled coordinates of the display area (2) is activated. Method according to claim 1, characterized in that a second interaction option is a selection of a button from several displayed buttons and that several selectable buttons are graphically represented for this second interaction option. Method according to claim 1 or 2, characterized in that a user performing the action is detected, that interaction options preferred by the user are stored, and that the selectable interaction options depend on the interaction options preferred by the user. Device for controlling a user interface comprising: - a display device (1) with a display area (2); - an input device comprising a touch-sensitive surface (5) of the display area (2) and comprising means for controlling coordinates of the display area (2), wherein, for controlling coordinates of the display area (2), the touch-sensitive surface (5) is touched by an actuating element, which is a user's fingertip; and - a control device (3) by means of which the display area (2) can be controlled so that a button (9, 10, 11) can be displayed, wherein the button (9, 10, 11) can be controlled by means of the input device and the button (9, 10, 11) is assigned several interaction possibilities, wherein in an initial state a first interaction possibility is visualized by a graphic design of the button (9, 10, 11) in such a way thatthat this first interaction option corresponds to a virtual touch switch, and by means of which an initiating action carried out in conjunction with the button (9, 10, 11) can be detected, in which the fingertip touches the touch-sensitive surface (5) of this virtual touch switch and the fingertip is not immediately removed, but is moved in a direction starting from the displayed button (9, 10, 11) and the design of the button (9, 10, 11) changes so that a slide switch (13, 20, 23) is graphically displayed and - by means of the control device (3) as an interaction option of the button (9, 10, 11) a continuous change of the controlled coordinates of the display area (2) can be activated depending on the detected action.

Images