Control of a multimodal user interface

Abstract

Method (200) for controlling a user interface (110, 110'), the method comprising: providing (202) a control element (112i, 112i-2) within a representation (112) of the user interface (110, 110'); acquiring (204) an initial input from a user (60) of the user interface (110, 110'), wherein the user (60) controls a pointer element (114) based on a direction (68') defined by a pointing gesture directed at the representation (112) with a hand (68) of the user (60), and the initial input is configured to position the pointer element (114) within the representation (112); determining (206) a relation between the control element (112i, 112i-2) and the pointer element (114);and, if the relation includes at least partial coverage (208) of the control element (112i, 112i-2) by the pointer element (114), capture (210) of a second user input (60) associated with the control element (112i, 112i-2), characterized in that the capture of the second user input (60) comprises capturing a second text input based on a second interaction of the user (60) with an input device connected to the user interface (110, 110'), wherein the input device is configured to capture the second text input in the form of one or more individual characters based on one or more character gestures of the hand (68) of the user (60) on the input device.

Description

[0001] The disclosure relates to systems and methods for controlling a user interface. In particular, the disclosure relates to systems and methods for controlling a multimodal user interface in vehicles. State of the art

[0002] User interfaces (UIs) for computers, tablets, and mobile phones are known in the prior art. Such user interfaces are also known for vehicles, particularly motor vehicles, which are based on similar technologies (e.g., touchscreens, optical sensors for gesture detection and control) and have a similar or identical range of functions.

[0003] Familiar user interfaces provide a variety of interaction elements upon which a user's interactions with the interface are essentially based. Typical interaction elements include buttons, text input fields, sliders, selection lists, option elements, and the like.

[0004] Text input fields are typically provided for textual input, such as entering search terms, short text messages, or textual information in general. Generally, this requires clicking, activating, or selecting the corresponding text input field to enter data.

[0005] Especially in the case of mobile devices, or where a full keyboard is typically not available, voice input is often possible after selecting one or more corresponding controls. This can be done, for example, using a special button on the (display) keyboard (see, for example, the "dictate" button in iOS or Android), which a user must press after selecting a text input field to activate speech recognition.

[0006] For example, US Publication 8,160,883 describes such a well-known user interface. Specifically, it provides controls for a web server to create client-side markers that include detection and / or an audible prompt. The controls include elements of a dialogue such as question, answer, confirmation, command, or instruction, and allow the use of voice input.

[0007] A disadvantage of familiar user interfaces is that interaction typically involves numerous, sometimes cumbersome, input steps that must be performed in a specific sequence. This is particularly true when so-called natural input methods (e.g., based on capturing a user's gaze or pointing gestures) are to be used, as many interaction steps are usually inefficient and ergonomically demanding. In certain use cases, such as when using mobile devices on the go or interacting with user interfaces in vehicles, this can negatively impact the user's attention to their surroundings or the vehicle itself.

[0008] Another problem is that without the active selection of individual user interface elements, even speech recognition, which may be running temporarily or permanently in the background, cannot distinguish which control element (e.g., field) is active and / or intended to be filled when multiple controls (e.g., text input fields) are present. This also creates the risk of incorrect entries, requiring further actions (e.g., undoing). This, too, can negatively impact user attention, make use more difficult and cumbersome, and / or impair the user experience.

[0009] It can also be a disadvantage if the user interface is limited to one or a few modalities, especially if different modalities cannot be combined or can only be combined with difficulty. In the example described above of using a mobile device, for instance, the user might first have to touch the touchscreen to activate an input field, and then perform a further touch to activate speech recognition. Furthermore, this touch is typically digital, meaning that no intermediate state between inactive and active can be selected. This is familiar, for example, in the context of a desktop application through "OnMouseOver" or the so-called focus, where no triggering or activation occurs or is even necessary.This is not known in other contexts, for example with mobile devices or user stations in vehicles that have to do without classic input devices such as mouse and keyboard.

[0010] Therefore, there is a need for methods and systems for controlling a user interface that reduce and / or minimize the necessary interactions for user input, thereby making them more efficient and simpler.

[0011] There remains a need for methods and systems for controlling a user interface that, in particular, enable multimodal inputs and / or combine them in a way that allows for the most efficient or intuitive user input possible.

[0012] In particular, there remains a need for methods and systems for controlling a user interface in vehicles that reduce and / or minimize the necessary interactions for user input, thereby making input more efficient and easier, and improving the user's attention to driving the vehicle and road safety. Disclosure of the invention

[0013] It is a purpose of the present disclosure to provide systems and a user interface for controlling a user interface that avoid one or more of the described disadvantages and / or enable one or more of the described advantages.

[0014] This problem is solved by the subject matter of the independent claims. Advantageous embodiments are specified in the dependent claims.

[0015] In a first aspect according to embodiments of the present disclosure, a method for controlling a user interface is specified. The method comprises providing a control element within a representation of the user interface; acquiring a first input from a user of the user interface, wherein the first input is configured to position a pointer element within the representation; determining a relation between the control element and the pointer element; and, if the relation involves at least a partial covering of the control element by the pointer element ( 114 ) includes capturing a second user input associated with the control element.

[0016] In a second aspect following the preceding aspect 1, the control element includes a text input field and the user's second input is a textual input.

[0017] In a third aspect following one of the preceding aspects 1 or 2, capturing the user's second input includes capturing text input based on speech recognition.

[0018] In a fourth aspect following one of the preceding aspects 1 to 3, capturing the second user input comprises capturing an initial text input based on a first interaction of the user with a keyboard provided on the display. Preferably, the keyboard is displayed at least on a portion of the display and / or the user makes the initial text input via the keyboard using the pointer element and / or by operating a touchscreen.

[0019] In a fifth aspect following any of the preceding aspects 1 to 4, capturing the second user input comprises capturing a second text input based on a second interaction of the user with an input device connected to the user interface. Preferably, the input device is configured to capture the second text input in the form of one or more individual characters based on one or more hand gestures of the user on the input device.

[0020] In a sixth aspect following one of the preceding aspects 1 to 5, the at least partial overlap concerns an active sub-area of ​​the pointer element.

[0021] Preferably wherein the pointer element includes a mouse pointer, further preferably wherein the active area is defined by the tip of the mouse pointer.

[0022] In a seventh aspect following one of the preceding aspects 1 to 6, the first and second user inputs show different modalities.

[0023] In an eighth aspect following one of the preceding aspects 1 to 7, the different modalities include one or more of the following: input based on the user's gaze direction and / or gaze; input based on the user's pointing and / or hand gesture; user interaction based on a touchscreen; voice input or speech recognition; interaction based on the user operating a vehicle control element, preferably wherein the control element includes the input device.

[0024] In a ninth aspect according to embodiments of the present disclosure, a system for controlling a user interface is specified. The system comprises a user interface and a control unit configured to execute the method according to embodiments of the present disclosure, in particular according to one of the preceding aspects 1 to 8.

[0025] In a tenth aspect according to embodiments of the present disclosure, a vehicle is specified comprising a system according to embodiments of the present disclosure, in particular according to the preceding aspect 9.

[0026] The methods and systems disclosed herein offer one or more of the advantages listed below.

[0027] Methods and systems for controlling a user interface according to the present disclosure reduce and / or minimize the necessary interactions between user and user interface for inputting information, making input particularly more efficient and simpler. Multimodal inputs are enabled and / or combined in such a way as to allow for the most efficient or intuitive user input possible.

[0028] Methods and systems for controlling a user interface in vehicles according to the present disclosure reduce, minimize and / or optimize the interactions between user and user interface necessary for inputs, in particular improving the user's attention to driving the vehicle and road safety in general.

[0029] Based on systems and methods according to the present disclosure, a large number of different control elements, in particular text input fields, can be displayed simultaneously on a user interface and used by means of speech recognition. A control element to be filled (e.g., a text input field) can be easily selected and used by means of speech recognition without any further prior interactions (e.g., prior clicking). Optionally, alternative, robust input methods, such as an (on-screen) keyboard or a so-called graffiti input field (e.g., via an iDrive controller), are also available. List of characters

[0030] Exemplary embodiments of the disclosure are shown in the figures and are described in more detail below. Unless otherwise noted, the same reference symbols are used for identical and equivalent elements. Fig. Figure 1 shows a schematic representation of a system for controlling a user interface according to embodiments of the present disclosure; Fig. Figure 2 shows a schematic representation of various interactions between a user and a representation provided by the user interface according to embodiments of the present disclosure; Fig. 3A to Fig. Figure 3G schematically illustrates representations of different views of a user interface according to embodiments of the present disclosure in the course of various interactions between a user and the user interface; and Fig. Figure 4 shows a flowchart of a method for controlling a user interface according to embodiments of the present disclosure. Implementations of the revelation

[0031] Unless otherwise noted, the same reference symbols are used for identical and equivalent elements in the following.

[0032] Fig. Figure 1 shows a schematic representation of a system 100 for controlling a user interface according to embodiments of the present disclosure for use in a vehicle 80 The system can essentially be based on a single control unit. 120 of the vehicle 80 and / or on one or more backend components 150 (e.g., servers, services) are executed. The vehicle 80 includes, in addition to the control unit, 120 further a communication unit 130 , which are used for data communication with the vehicle 80 external components (e.g. mobile devices) 125 and / or backend 150 ) is configured, and a user interface 110 , for example as a touchscreen in the vehicle 80This can be implemented (e.g., on or in the instrument panel area of ​​the vehicle). 80 or as part of a rear-seat entertainment system in the back of the vehicle 80 ).

[0033] The user interface 110 in the vehicle 80 is used as an illustrative example to demonstrate the concepts underlying the present revelation. The system 100 However, it can also be partially or fully used on a mobile device 125 be implemented by a user and offer essentially the same functions and a corresponding user interface 110' provide as below with regard to the user interface 110 in the vehicle 80 is described. The user interface 110' can further implement all necessary components relating to the user interface 110These include, for example, voice input and output, cameras and screens, biometric sensors (e.g., fingerprint scanner, retina scanner) and the like.

[0034] The vehicle 80 further includes sensor technology 115 , which is configured to be a user 60 (not in Fig. 1 shown) in the vehicle 80 to detect. The sensors 115 is in Fig. 1 is only an example and may include several components, especially at different locations in or on the vehicle. 80 installed components.

[0035] The sensors 115 This may include, for example, one or more cameras configured to monitor the interior of the vehicle, particularly a user. 60 of the vehicle, to capture, for example, images of the user's face 60 to capture. To capture the user 60To capture the data, additional components can be provided, such as sensors that implement gesture control (e.g., pointing, indicating, swiping on the central information display (CID) without having to use a touchscreen). The sensors 115 It may also include one or more cameras that capture the area around the vehicle. Based on this, a user can 60 data is captured outside the vehicle (e.g., for comparing documents or images from the user). 60 As mentioned above, the sensors can 115 also wholly or partially on the mobile device 125 It includes.

[0036] The user interface 110 may include one or more multimodal user interfaces, in particular user interfaces used for operating the vehicle 80The user interface includes configured settings (e.g., communication, infotainment, climate control, seat settings, vehicle settings). 110 enables the multimodal capture of user input 60 , for example via a graphical user interface (e.g. touchscreen), via classic vehicle controls 80 (e.g. buttons, switches, iDrive controller), via voice control (e.g. using sensors) 115 ), via scanner (e.g., for fingerprints, retina scans), and the like. The user interface 110 further enables the multimodal output of information to a user 60, for example, via graphical display elements (e.g., touchscreen, head-up display, instrument cluster, central information display or CID), via tactile elements (e.g., vibration of the steering wheel or parts of the seat), via voice output through a speaker system in the vehicle (e.g., infotainment system) or acoustic signaling devices (e.g., gong, beeper), and the like. The user interface 110 Based on relevant configuration data, they can implement a graphical user interface that displays indicators and controls for the user. 60 for operating the vehicle 80 can be used. Additionally or alternatively, the user interface can include (further) display and control elements, such as switches, buttons and displays.

[0037] Via the communication unit 130 can the control unit 120 or the system 100to communicate with external components and services, for example with backend servers and / or services. 150 communicate. Alternatively or additionally, the control unit can 120 or the system 100 via the communication interface 130 with the mobile device 125 a user 60 engage in data communication and use data and / or sensors provided by the mobile device 125 be provided (as described above).

[0038] Furthermore, the system can 100 one for the vehicle 80 external backend component 150 or infrastructure that provides one or more resources (e.g., servers, services). The backend component 150 can be temporarily or permanently connected to the control unit 120 of the vehicle 80 and / or with the mobile device 125 in data communication 140Preferably, resource-intensive processing steps (e.g., natural language processing and / or image or video data processing, speech recognition, processing of large external datasets) can be offloaded to the external backend component. 150 outsourced, which are controlled by the control unit 120 in the vehicle 80 and / or through the mobile device 125 which could be difficult or even impossible to implement. Consideration can also be given to potential requirements regarding computing power, storage capacity, available bandwidth, connection to external data sources, and the like.

[0039] One on the control unit 120 The executed software component can establish a data connection 140 to one or more services 150, for example, search engines or messaging services. Such a connection can preferably be established directly if a service or platform (or the underlying protocol) is directly supported by the software component. Alternatively or additionally, such a connection can be established via a user-defined interface. 60 with the vehicle 80 connected mobile device 125 This can be done. The software component can then use all services, platforms, and protocols available to the mobile device. 125 be supported.

[0040] Fig. Figure 2 shows a schematic representation of various interactions between a user 60 and one through the user interface 110 provided representation 112 according to embodiments of the present disclosure. Fig. Figure 2 shows three different use cases (a), (b) and (c) in which different modalities of user input are used.

[0041] In case (a) the user controls 60 a pointer element 114 (cf. “mouse pointer” or similar) based on a viewing direction 66' , which are due to the representation 112 directed eyes 66 the user 60 is defined. The pointer element follows. 114 the direction of view 66' about the presentation 112 and remains at the corresponding location, for example on a control element (e.g. text input field).

[0042] Case (b) outlines a similar case based on a different modality. In case (b), the user controls 60 a pointer element 114 based on direction 68' , which are due to the representation 112 directed pointing gesture with one hand 68the user 60 is defined. The pointer element follows. 114 the pointing direction 68' about the presentation 112 and remains at the corresponding location, for example on a control element (e.g. text input field).

[0043] Case (c) outlines a case based on an additional modality. In case (c), the user controls 60 a pointer element 114 based on inputs made with one hand 68 the user 60 on an input device 108 (e.g., touchpad, touchscreen, iDrive controller). The pointer element follows this process. 114 the movement of the hand 68 (or one or more fingers) of the user 60 about the presentation 112 and remains at the corresponding location, for example on a control element (e.g. text input field).

[0044] In all the aforementioned cases, and in further cases, it can be assumed that the user 60 a control element (e.g. a text input field) on the display 112 want to select. The system 100 This allows this based on different modalities to provide the user with 60 to provide the option of using a modality (or modalities) that corresponds to their individual preferences and / or is suitable for a given usage context. The cases described above cover only a selection of exemplary modalities. Further modalities are conceivable, for example, input via touchscreen (e.g., directly onto the display). 112 ), voice input, operation of a dedicated (physical) control element (e.g. button on the vehicle's steering wheel) 80 ) and similar items.

[0045] The Fig. 3A to Fig. 3G schematically illustrates representations 112different views of a user interface 110 according to embodiments of the present disclosure in the course of various interactions between a user 60 and the user interface 110 .

[0046] Fig. 3A to Fig. 3G generally shows a representation 112 a user interface 110 , which are exemplary in areas 112r-1 and 112r-2 The display is divided, thus defining two different, approximately equal-sized display areas. Alternatively, the display can be... 112 It may include a single area or more than two areas, depending on the usage context and / or displayed content. The usage context shown is purely an example of user communication. 60 via text messages (see title "Messages"). Numerous other applications can be accessed similarly or identically via the user interface. 110to realize this without deviating significantly from what has been shown.

[0047] In the examples shown, in the area 112r-1 to see a list of possible communication partners, each represented by a corresponding control element. 112e , 112es be displayed, with the control element 112es is displayed as "selected" (see checkmark). A user-selected option. 60 controllable pointer element 114 is also shown. The area 112r-2 Displays any existing history for a selected communication partner, including incoming messages. 112mi and / or outgoing messages 112mo In addition, there are text input fields. 112i and 112i-2 shown, each of which is one of the areas 112r-1 and 112r-2 are assigned. Each text input field 112i , 112i-2 Furthermore, it displays a pictogram indicating the type of input field. 112ip and a help text displaying the context of the input field 112it up. The user 60 The pictogram indicates that it is a text input field that supports voice input (see pictogram). 112ip "Microphone") and can find out from the respective help text what the text input field is associated with (see search term or message). The Fig. 3B to Fig. 3G shows identical or similar elements unless otherwise explained or additionally.

[0048] As in the Fig. 3A to Fig. 3G is shown, which can be seen in the display. 112 multiple text input fields 112i , 112i-2 to be placed. Furthermore, it is planned that voice input will be possible in each respective text input field. 112i , 112i-2 is automatically activated when the pointer element 114 on the text input field 112i , 112i-2is located - without requiring a "click" or any other further interaction from the user 60 is required. In Fig. For example, the pointer element is located in 3A. 114 not over one of the text input fields 112i , 112i-2 , so that no voice input function is or will be activated. In Fig. 3B, however, contains the pointer element. 114 on (or above) the text input field 112i in the left area 112r-1 the representation 112 , which activates the voice input function for the text input field. 112i An activated voice input function can be indicated by a changed pictogram. 112ip' (see pictogram) 112ip in Fig. 3A and pictogram 112ip' in Fig. 3B) may be displayed, for example with additional pictogram elements, a changed size of the pictogram, different colors, and / or the like. Alternatively or additionally, a display of the text input field may be used. 112i The text input field can also be adjusted accordingly if the voice input function is activated (see size of the text input field). 112i in Fig. 3A and Fig. 3B).

[0049] As an alternative to the voice input function, other modalities are available, for example by displaying a (screen) keyboard (see below). Fig. 3D; further description below) or by means of graffiti / drawing gestures via iDrive controller and the like.

[0050] Fig. 3B shows a state of the representation 112 , in which the user 60 the pointer element 114 on (or above) the text input field 112i has positioned, for example via gaze control (see above). Fig. 2 (a)), via pointing gesture control (cf. Fig. 2 (b)) or via touchpad (cf. Fig. 2 (c)), and the voice input function has been activated as explained above. In this state, the user can 60 without further interaction via voice input, the text input field 112i Fill it, in this example with the search term "Dominique".

[0051] Fig. 3C shows a subsequent state of the representation 112 , in which the user 60 Enter the text input field using voice input with the example search term "Dominique". 112i has filled. Accordingly, the search term "Dominique", recognized by the voice input function, is entered into the text input field. 112i displayed as it would be seen when typing, so that the user 60The system can immediately recognize correct (or potentially incorrect) data entry. Based on the input, the list of possible communication partners is updated, and the corresponding controls for the entered search term are displayed. 112e has been hidden.

[0052] If, in the event of an incorrect input or recognition error, the entered search term is incorrect, the text input field may be... 112i by activating a corresponding control element 112ic (e.g., a button to close / reset the text input field) would be reset. In this case, the user would 60 for display in Fig. 3B returns and could make another (voice) input, activating the control element 112ic This can be done via touchscreen or by clicking. It is also conceivable that a keyword or command word (e.g., "Back" or "Undo") could activate the function via voice input recognition.

[0053] In the event of correct input and recognition, the user can 60 as shown in the example with the representation 112 in Fig. Continue with 3E.

[0054] Fig. 3D shows an alternative input method to voice input, using an on-screen keyboard. Based on the display... 112 in Fig. 3B the user can 60 , instead of using voice input, use the text input field 112i activate (e.g. via touchscreen or "click") so that a text input mode with a corresponding keyboard is available. 112t is displayed. The one with the text input field 112i associated keyboard 112t allowed in conjunction with an entry in the text input field 112i displayed cursor element 112ic a text input. The keyboard can be used for this. 112t This can be used, or an alternative modality, for example graffiti / gestures via the iDrive controller, where the search term can be entered textually, keystroke by keystroke or character by character. Once the search term has been entered, the user can 60 The function (e.g., search) can be actively triggered (e.g., using the "Enter" key or a corresponding graffiti symbol), or it can be triggered automatically after a predetermined waiting period (e.g., 2 seconds) without any text input. After successful (correct) input, the user is taken to... 60 for display in Fig. 3C.

[0055] In the presentation 112 in Fig. 3C allows the user to select the (already selected) contact 112es select or alternatively in one of the representations 112 in the Fig. 3A, Fig. 3B or Fig. 3D directly one of the controls 112e Activate the selection (e.g., via touchscreen, "click"). This allows the user to 60 Select a desired entry from the list of contacts, preferably after prior filtering or searching, and the display will appear. 112 in Fig. 3E.

[0056] Fig. 3E shows a state of the representation 112 , in which the user 60 the pointer element 114 on the text input field 112i-2 in the right display area 112r-2 has positioned itself. Analogous to what was described above with regard to Fig. 3B and Fig. 3C the user can 60 Now enter an entry into the text input field. 112i-2 Perform via voice input. Pictograms 112ip , 112ip' and help texts 112it correspond to the above description. The message history in the display area 112r-2 is with the selected input element 112es (cf. hook) associated and provides a short message history consisting of received 112mi and outgoing 112mo News again.

[0057] Fig. 3F shows a subsequent state of the representation 112 in which the user 60 the text input field 112i-2 The sample text message "Ok, see you at 8:30 pm!" was used. The text message recognized by the voice input function is then processed accordingly. 112mn (“Ok, see you at 8:30 pm!”) in the text input field 112i-2 displayed as it would be seen when typing, so that the user 60 This in turn allows the system to immediately recognize correct (or potentially incorrect) recording as such. To understand the recognized short message text... 112mn to send, represents the display 112 a corresponding control element 112b (e.g., send button) ready, which is connected to the text input field 112i-2is associated.

[0058] In Fig. 3G is a further subsequent state of representation. 112 illustrated, whereby the user 60 again via eye-tracking (see above). Fig. 2 (a)), via pointing gesture control (cf. Fig. 2 (b)) or via touchpad (cf. Fig. 2 (c)), the pointer element 114 on the control element 112b moved and, by means of a further input (e.g. touchscreen, "click", voice command "Send!"), the short message text 112mn has been sent. This is in the display area. 112r-2 evident, which in Fig. 3G a new outgoing message 112mo' displays the text of the short message. 112mn (“Okay, see you at 8:30 pm!”) Older messages are moved to the top and gradually disappear from the display area. 112r-2 with the receipt or dispatch of subsequent messages.

[0059] Fig. Figure 4 shows a flowchart of a process. 200 for controlling a user interface according to embodiments of the present disclosure. The method 200 begins at step 201 In step 202 A control element will be 112i , 112i-2 within a representation 112 the user interface 110 , 110' as shown (see e.g. Fig. 3A). In step 204 This will be the first input from a user. 60 the user interface 110 , 110' Captured. The first input is configured, the pointer element. 114 (e.g. a mouse pointer) within the display 112 to position (cf. Fig. 3B). In step 206 A relation of the control element will be established. 112i , 112i-2 and the pointer element 114 determined in relation to each other. This could, for example, involve an overlap of the pointer element. 114with the control element 112i , 112i-2 act (e.g., active point, such as the tip of the mouse pointer over the control). The covering is done in step 208 determined. In step 210 will occur if the relation involves at least a partial overlap of the control element. 112i , 112i-2 through the pointer element 114 includes one with a control element 112i , 112i-2 associated second user input 60 recorded. The process ends at step 212 .

[0060] Although the invention has been further illustrated and explained in detail by means of preferred embodiments, the invention is not limited by the disclosed examples, and other variations can be derived from them by a person skilled in the art without departing from the scope of protection of the invention. It is therefore clear that a multitude of possible variations exist. It is also clear that the embodiments mentioned as examples are truly only examples and are not to be understood in any way as limiting, for example, the scope of protection, the possible applications, or the configuration of the invention.Rather, the preceding description and the description of the figures enable the person skilled in the art to implement the exemplary embodiments in concrete terms, whereby the person skilled in the art, with knowledge of the disclosed inventive concept, can make various changes, for example with regard to the function or the arrangement of individual elements mentioned in an exemplary embodiment, without leaving the scope of protection defined by the claims and their legal equivalents, such as further explanations in the description. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 8160883

[0006]

Claims

[1] Method (200) for controlling a user interface (110, 110'), the method comprising: Providing (202) a control element (112i, 112i-2) within a representation (112) of the user interface (110, 110'); Capturing (204) a first input from a user (60) of the user interface (110, 110'), wherein the first input is configured to position a pointer element (114) within the display (112); Determine (206) a relation between the control element (112i, 112i-2) and the pointer element (114); and, if the relation includes at least a partial covering (208) of the control element (112i, 112i-2) by the pointer element (114), capture (210) a second user input (60) associated with the control element (112i, 112i-2). [2] Method (200) according to the preceding claim 1, wherein the control element (112i, 112i-2) includes a text input field and the second input of the user (60) includes a textual input. [3] Method (200) according to one of the preceding claims 1 or 2, wherein capturing the second input of the user (60) comprises capturing a text input based on speech recognition. [4] Method (200) according to any one of the preceding claims 1 to 3, wherein capturing the second input of the user (60) comprises capturing a first text input based on a first interaction of the user (60) with a keyboard (112t) provided on the display (112), preferably wherein: - the keyboard (112t) is displayed at least on a portion of the screen (112); and / or - the user (60) makes the first text input via the keyboard using the pointer element (114) and / or by operating a touchscreen. [5] Method (200) according to any one of the preceding claims 1 to 4, wherein capturing the second input of the user (60) comprises capturing a second text input based on a second interaction of the user (60) with an input device connected to the user interface (110, 110'), preferably wherein the input device is configured to capture the second text input in the form of one or more individual characters based on one or more character gestures of the hand (68) of the user (60) on the input device. [6] Method (200) according to any one of the preceding claims 1 to 5, wherein the at least partial coverage relates to an active part of the pointer element (114), preferably wherein the pointer element (114) includes a mouse pointer, more preferably wherein the active part is defined by the tip of the mouse pointer. [7] Method (200) according to any one of the preceding claims 1 to 6, wherein the first and second input of the user (60) have different modalities. [8] Method (200) according to one of the preceding claim, wherein the different modalities include one or more of the following: - an input based on a user's gaze direction and / or gaze direction (60); - an input based on a pointing gesture and / or hand gesture of the user (60); - a user interaction based on a touchscreen (60); - Voice input or speech recognition; - an interaction based on the operation of a control element of the vehicle (80) by the user (60), preferably wherein the control element includes the input device. [9] System (100) for controlling a user interface (110, 110'), the system comprising: the user interface (110, 110'); and a control unit (120) configured to perform the method (200) according to any one of the preceding claims 1 to 8. [10] Vehicle (80) comprising a system (100) according to the preceding claim 9.

Images