Method for operating an operating system in a vehicle and operating system in a vehicle

Abstract

In the case of a method for operating an operating system in a vehicle, an output with at least one first operating object is generated and output in an output area. Here, the first operating object is assigned to a first element of a first hierarchical level of a hierarchically organized database. An operating gesture is acquired, wherein the acquired operating gesture includes a movement direction of a manipulation object perpendicular to the output area, and the output is changed in such a way as to replace the first operating object with a second operating object depending on the movement direction of the operating gesture. Here, in the case of changing the output, the movement of the first operating object in the movement direction of the operating gesture is outputted perspectively and the second operating object is assigned to a second hierarchical level of the hierarchically organized database. The operating system includes a control unit (8), an acquisition unit (4, 5) and an output unit (3) coupled to the control unit (8) and having an output area.

Description

Technical Field

[0001] The invention relates to a method for operating an operating system in a vehicle and to an operating system in a vehicle. Background Art

[0002] Modern everyday life is influenced by a large number of devices, which can be operated in different ways in different contexts. For this purpose, different operating concepts are proposed, such as control by means of buttons, gestures, voice or indicators, which have specific advantages and disadvantages depending on the field of application. In some cases, in order to achieve the same system response, different operating modes can be provided, which recognize the user and the user has to decide between them.

[0003] The problem often arises in known operating systems in that they are designed only for certain modes of operation. Therefore, the user can only access various operating options with difficulty and use them for operation.

[0004] US 2015 / 0175172 A1 describes a system for gesture-based input with tactile feedback. The gestures are detected and the corresponding feedback is output. In addition, voice control is also possible. Different gestures are described, which can be detected by means of a camera and are associated with different controllable actions of the system.

[0005] US 2004 / 0189720 A1 discloses an architecture for controlling a computer by means of gestures, in which gestures can be used in combination with voice commands.

[0006] DE 10 2009 039 114 A1 proposes an operating device for a vehicle, in which a body part of a user is detected in one of three spatial regions and the associated object is displayed in the case of a graphical object. For example, quantities and scales can be displayed.

[0007] US2018 / 0307347 A1 describes a method for detecting gestures, in which a capacitive sensor is used. The user can define gestures for certain commands in a database and learn preset gestures by viewing a guide.

[0008] A method for operating an interactive operating device is known from DE 10 2006 037 156 A1, in which operating elements are displayed and the user's approach to the operating elements is detected. The operating elements are then optimally displayed, for example, in such a way that they are enlarged. In addition, the matching of the illustrations can be controlled by means of gestures. Summary of the invention

[0009] The invention is based on the object of providing a method for operating an operating system and an operating system which enable particularly simple and rapid operation for a user, in particular if this operation is possible using different operating modes.

[0010] According to the invention, this object is achieved by a method described below and an operating system described below.

[0011] In the case of the method according to the invention for operating an operating system in a vehicle, an output with at least one first operating object is generated and output in an output area. Here, the first operating object is assigned to a first element of a first hierarchical level of a hierarchically organized database. An operating gesture is acquired, wherein the acquired operating gesture includes a direction of movement of the operating object perpendicular to the output area, and the output is changed in such a way that the first operating object is replaced by a second operating object depending on the direction of movement of the operating gesture. Here, in the case of changing the output, the movement of the first operating object in the direction of movement of the operating gesture is output in a perspective manner. Here, the second operating object is assigned to a second hierarchical level of the hierarchically organized database.

[0012] This advantageously enables particularly simple and intuitive operation by means of operating gestures and rapid finding of desired locations within a plurality of hierarchical levels.

[0013] In the case of this method, an operating object is first generated, which is configured in a manner known per se, for example, as a widget, a window or a graphic object (e.g., an icon or a symbol) and can be output on a display surface. The operating object is in particular an operable element. It can be used alternatively or additionally for the output of information. The operating object can, for example, be configured as a graphic element of a display and define such an area of ​​the display surface, i.e., manipulation in this area causes the operation of the operating object. This manipulation can, for example, be achieved by contacting the touch screen in the area of ​​the operating object with the aid of the operating object or by manipulation with the aid of a pointing tool (e.g., a cursor controlled by a mouse or similar device). In addition, in this case, manipulation can be achieved with the aid of gestures, wherein contact with the touch screen or input device is not necessarily necessary, for example, with the aid of pointing and manipulating gestures.

[0014] Here, the operating objects are assigned to the hierarchical levels of a hierarchically organized database. Such a database includes in particular data objects (e.g. data or data sets), which are organized hierarchically, for example, in the form of a directory structure. Media data, data about a radio station, schedule records or address data may belong to this, for example. In addition, the operating menus of the operating surface can be organized hierarchically, wherein the individual menus in the menu structure are understood as objects of the hierarchically organized database. The database is arranged in such a way that its data objects are correspondingly assigned to the hierarchical levels. These hierarchical levels are in turn at the upper and lower levels relative to each other.

[0015] The output of the operating object is realized in an output area in a manner known per se. The output area can, for example, include a display surface of a display device, such as a display or a touch screen. In addition, the output area can be constructed in three-dimensional space, especially in the case of holographic output. Similarly, an output device with an autostereoscopic display can generate an output in such a way that the operating object appears three-dimensionally in space. Here, the device used for the output can be fixedly arranged in the vehicle or realized as an instrument outside the vehicle or that can be used separately from the vehicle.

[0016] In the case of this method, operating gestures are also acquired, which can be implemented in a manner known per se. "Gestures" are particularly understood to be a certain position and orientation (that is, a posture) of an operating object (such as a user's hand) or a certain movement implemented with an operating object. Gestures can be designed according to forms known per se. They particularly include pointing gestures, waving gestures, and gestures used in daily use, such as hand rotations, grabbing gestures, and a combination of multiple such gestures that are implemented directly and immediately in sequence if necessary. Gestures are implemented in an acquisition space. The acquisition space can be in a spatial association with a relatively certain device (such as an output device), and can be adjacent to it, for example. In addition, gestures can include approaching a certain spatial position (such as a device or a position on a display surface). A particularly simple and intuitive operational feasibility solution is provided to the user through gesture operation.

[0017] The direction in space can also be determined by means of a gesture and, depending on the determined direction, in particular the position for which the indication gesture is determined can be determined. For example, the user can point to a point in the display area and thereby perform a selection. Subsequently, the selection of a graphic object can be visualized when the graphic object is located in a predetermined surrounding area around the calculated point. Thus, the graphic object can be selected by the indication. The visualization of the selection of the graphic object has the advantage that the user directly receives feedback on his operating actions. In this case, even when the object has not yet been identified in the surrounding area around the point, the point can also be visualized in the display area. This avoids the situation where the user has to search in the gap between the objects until he encounters the desired graphic object. This can further accelerate the selection process.

[0018] The gestures can be detected, for example, contactlessly by a camera system which collects temporally dispersed video data from a detection space and assigns the detected user movements to specific gestures by means of a connected evaluation unit. Alternatively or also additionally, gestures can be detected by means of resistive and / or capacitive surfaces.

[0019] In the context of the present invention, the operating gesture includes a direction of movement of the operating object perpendicular to the output area. In the context of the present invention, this is understood to mean that the direction of movement does not extend parallel to the output operating object or at a constant distance relative to the output operating object, but that the operating object moves, for example, toward the operating object or away from it. The direction of movement extends in particular with a direction component perpendicular to the display plane in which the display surface is arranged.

[0020] In one embodiment of the method according to the invention, the movement of the first operating object is output in a three-dimensional representation when the output is changed. This advantageously enables a particularly clear and easily accessible representation.

[0021] The movement of the first operating object is outputted in perspective and corresponds to the direction of movement of the operating gesture. The perspective output can be generated in a manner known per se. The first operating object is shown in a mobile manner so that its direction of movement roughly corresponds to the direction of movement of the operating gesture. Here, the first operating object can move in the same direction simultaneously with the operating gesture, wherein the shown movement of the operating object is proportional to or equal to the actual movement of the operating object. The operating object can, for example, move at the same or optional speed or in the case of movement, the distance traveled can be equal to or proportional to the distance traveled by the operating object.

[0022] The three-dimensional illustration can be produced in a known manner, for example by a two-dimensional illustration of a projection of a virtual object in a virtual three-dimensional space. In addition, the output can be realized by means of a holographic or autostereoscopic display illustration, by which an object is displayed in a three-dimensional space.

[0023] In another embodiment of the method, the second operating object is assigned, depending on the direction of movement of the operating gesture, to a hierarchical level one level above or one level below the hierarchical level of the first operating object. This advantageously makes it particularly easy to switch between hierarchical levels.

[0024] In particular, it is determined as follows, namely, whether the direction of movement of the operating gesture includes a directional component toward the output area or away from the output area. For example, in the case of a movement away from the output area, the second operating object of the next hierarchical level is output. For example, such an operating gesture is constructed similarly to "bring" or "pull closer" toward the user. On the contrary, in the case of a movement toward the output area, the second operating object of the upper hierarchical level can be output. For example, such an operating gesture is constructed similarly to "push away" or "reject" away from the user. For example, when the movement component extends downward or upward, to the right or to the left, and the second hierarchical level and / or the second operating object to be output can be determined depending on this directional component, other components of the movement direction can also be evaluated similarly.

[0025] In particular, a change in the output is achieved starting from a certain hierarchical level to a hierarchical level arranged directly above or below it. In addition, depending on the type of operating gesture, a transition to a hierarchical level can also be made, which is further away in the hierarchy than the level of the hierarchical level of the first operating object; for example, a certain operating gesture can be recognized and the operating gesture is assigned a large number of hierarchical levels to be skipped, or in the case of a certain "shortcut" operating gesture, a change can be made to a certain hierarchical level, for example the topmost hierarchical level. In this way, it is possible to navigate through the hierarchy particularly quickly and find a certain operating object.

[0026] In another embodiment, the output includes at least two first operating objects, which are assigned to the same hierarchy level and are arranged in a row. As a result, multiple elements of the hierarchy level can be outputted in an advantageously particularly clear and easily findable manner.

[0027] In particular, all operating objects assigned to the same hierarchy level are arranged in a row. The row can extend horizontally, vertically or in other directions, wherein the operating objects are arranged along a straight line.

[0028] The row can be designed such that it extends in the form of a strip, for example across the display surface. Furthermore, the hierarchical level operating objects can be arranged along a virtual spatial structure, for example along a ring-shaped path, wherein the operating objects of different hierarchical levels are arranged along a structure arranged one behind the other and, for example, concentrically. This arrangement can be designed, for example, along a virtual roller surface or in the form of a carousel.

[0029] For example, in the case of a change in the output from the first to the second operating object, the movement of the arrangement is shown in such a way that the virtual perspective of the arrangement of the operating objects at different hierarchical levels changes. For example, the first and second operating objects are arranged along a ring, which extends concentrically to one another and is assigned to each hierarchical level. In the case of a further example, the arrangement can also be realized along concentric roller surfaces or the surface of a carousel arrangement as an alternative to concentric rings.

[0030] The user can perform an operating gesture such that, for example, he performs a swipe gesture in the direction of the output area and the arrangement of the operating objects shown there is also moved away by the user. In this way, the second operating object of the ring further outside is displayed, while the first operating object shown first moves to the background. Conversely, the user can perform a pull gesture away from the output area and towards the user; the representation is then changed in such a way that the arrangement of the operating elements moves towards the user and the perspective is changed in such a way that the second operating object of the ring further inside is displayed.

[0031] The operation implemented in the context of this method is then in particular such that the user moves the virtual arrangement of the operating objects in space by means of an operating gesture, so that the movement of the operating gesture extends similarly to the displayed movement of the operating object. The user can then move the virtual arrangement by his operating gesture and thus navigate to a specific hierarchical level and / or to a specific operating object.

[0032] In one development, the outputted first operating object is marked as the currently selected element, wherein the marking is carried out in particular automatically. This advantageously speeds up the operation by means of the operating gesture.

[0033] For example, when the first operating object is in a certain area of ​​the output area, for example in the middle of the display surface, the first operating object can always be automatically marked. That is to say, marking can also be carried out by moving the operating object to a certain position, for example by means of a waving gesture.

[0034] The currently selected operating object can be highlighted in a manner known per se, for example by a particular position, size, color or brightness.

[0035] In one embodiment, the operating gesture comprises a push or pull gesture with the hand of the user, wherein the operating gesture is performed in particular with a closed or open hand. The push or pull gesture is performed in particular in the direction of the first operating object or away from it. The user can thereby advantageously distinguish between different operating gestures particularly easily and quickly.

[0036] In particular, it is detected in which hand position the operating gesture is performed and in which direction the hand is pointed. For example, it can be distinguished whether, in the case of an open hand, the palm is facing the operating object or pointing away from it, in particular towards the user. In addition, the opening and closing of the hand can be detected, in particular at the beginning or end of an operating gesture, for example, to perform an operating gesture for grabbing and / or releasing.

[0037] In the case of a further development, the operating system has at least one first and second operating mode. Here, the output is formed depending on the operating mode. A trigger signal is received, wherein a change between the first and second operating mode is performed depending on the trigger signal. Thus, the output is advantageously generated in a manner adapted to the current situation. In particular, a change is made between different operating modes, and different outputs of the first or second operating object are also generated for different operating modes.

[0038] In the case of a change between different operating modes, the output can also be changed, in particular the arrangement of the operating object or multiple operating objects in the output. For example, multiple operating objects can be arranged side by side and on top of each other in a matrix in the case of a first operating mode, while they are arranged side by side or on top of each other in a row in the case of a second operating mode. In addition, the arrangement of the operating objects can be changed in the case of a transition by means of an animation, wherein, for example, the operating objects are moved along the display surface and rearranged.

[0039] Depending on the currently activated operating mode, operation can be achieved, for example, by means of different preferred operating methods. In the first operating mode, operation can be achieved, for example, preferably by means of operating gestures in three-dimensional space, while in the second operating mode, operation by means of operating gestures along a two-dimensional surface is preferred. In addition, other operating methods may be preferred depending on the operating mode, for example by means of voice input, an indicator or control of a physical operating element (such as a switch or a regulator).

[0040] The trigger signal for the change between the operating modes can be generated in different ways, for example automatically in the case of meeting a certain condition or manually by means of a trigger signal by means of an operating action of a user. For example, the state or setting parameters of the vehicle equipment can be acquired and the operating mode is set depending on the acquired state or parameters. In addition, the trigger signal can be generated depending on the setting for a certain user.

[0041] In one embodiment, the trigger signal is detected when the seat position is changed. For example, a distinction is made between an approach mode and a distance mode. The operating mode can thus be advantageously adapted to the user's possibilities for operation.

[0042] The seat position can be determined, for example, relative to the output area, for example by means of the distance to the display. Furthermore, the range of action of the user in the seat can be determined or predefined, for example by means of arm length or by means of a fixed predefined distance value. In this case, for example, a distinction is made between a position of the seat close to the display and a position far from the display, depending on whether the display is in the range of action of the user in the seat or outside the range. A trigger signal is then generated when the seat position exceeds the range value or an adjustment below it is detected.

[0043] In another embodiment, the trigger signal is detected as a function of a change in the vehicle driving mode. The operating mode can thus advantageously be particularly well adapted to the driver's requirements.

[0044] For example, the operating mode can be changed when a certain degree of automation of the vehicle control is exceeded or fallen below. For example, a first operating mode can be set when the vehicle is controlled manually or with a small degree of automation, and a second operating mode can be set when the vehicle is controlled with a higher degree of automation or fully automatically.

[0045] For example, in the case of a transition from manual or partially automatic control of the vehicle to at least largely fully automatic control of the vehicle, not only is the seat position adjusted from a position close to the display to a position far from the display, but a trigger signal for changing the operating mode can also be generated at the same time.

[0046] The operating system according to the present invention in a vehicle comprises a control unit, an acquisition unit and an output unit coupled to the control unit with an output area. Here, the control unit is configured to generate an output with at least one first operating object and output it in the output area of ​​the output unit. Here, the first operating object is assigned to the first element of the first hierarchical level of a hierarchically organized database and the acquisition unit is configured to acquire an operating gesture, wherein the acquired operating gesture includes a direction of motion of the operating object perpendicular to the output area. In addition, the control unit is configured to change the output in such a way that the first operating object is replaced by the second operating object depending on the direction of motion of the operating gesture. Here, in the case of changing the output, the movement of the first operating object in the direction of motion of the operating gesture is outputted perspectively and the second operating object is assigned to the second hierarchical level of the hierarchically organized database.

[0047] The operating system according to the invention is designed in particular to operate with the aid of the method according to the invention described above. Therefore, the operating system has the same advantages as the method according to the invention.

[0048] In one embodiment of the operating system according to the invention, the output unit and the acquisition unit are integrated in the touch screen, thereby advantageously making it possible to acquire inputs particularly easily. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The invention is now explained using exemplary embodiments with reference to the accompanying drawings.

[0050] Figure 1 A vehicle is shown with an embodiment of an operating system according to the invention,

[0051] Figure 2 An embodiment of the method according to the invention is shown,

[0052] Figure 3 shows an exemplary embodiment of an output generated in the context of the method according to the invention with a prompt for the execution of an operating gesture,

[0053] Figure 4 shows an example of an output produced in the context of the method according to the invention with prompts for different directions, Figure 5 An example of an output produced in the context of the method according to the invention is shown with a pop-up window regarding volume adjustment,

[0054] Fig. 6A and 6B shows an example of an output generated in the context of the method according to the invention with a prompt regarding the execution of a relative operating gesture,

[0055] Figure 7 An embodiment of an output produced in the context of the method according to the invention with a pop-up window for controlling a lighting device is shown, and

[0056] Figures 8A to 8D Examples of the outputs produced in the context of this method and its operation in different operating modes are shown. DETAILED DESCRIPTION

[0057] Reference Figure 1 A vehicle with an exemplary embodiment of an operating system according to the present invention is described.

[0058] The vehicle 1 comprises a touch screen 2 which is designed in a manner known per se and comprises a display unit 3 and a touch-sensitive surface 4 arranged thereon. The vehicle 1 also comprises a detection unit 5 which is designed as a camera 5 and by which gestures can be detected in a detection area 7 in the interior of the vehicle 1, and an output unit 6 which is designed as a loudspeaker 6. The touch screen 2, the camera 5 and the loudspeaker 6 are coupled to a control unit 8, to which a multimedia unit 9 is likewise coupled.

[0059] refer to Figure 2 and 3Explanation of an exemplary embodiment of the method according to the present invention and the output produced therewith with instructions for the execution of an operating gesture. Figure 1 The exemplary embodiments described are used as a starting point; the operating system is further explained in detail by the description of the method.

[0060] In step S1, it is determined which function is currently operable. To this end, it is determined which device can currently be controlled by the control unit 8 and is in a state in which the user's input can be processed. In the case of this embodiment, the functions of the multimedia unit 9 are operable, for example, functions for outputting audiovisual content, texts and images, for controlling a navigation instrument, for accessing other devices of the vehicle 1 or for controlling a communication device (for example a telephone). Here, the user can also navigate through hierarchically organized databases, such as music titles or address data or menu structures. In the case of other embodiments, other functions and / or devices of the vehicle 1 may be operable as an alternative or in addition.

[0061] In this step S1, in the case of this exemplary embodiment, the identity of the user is also acquired. In the case of this exemplary embodiment, this is done by means of a mobile device, in particular a vehicle key, which is assigned to the user. In the case of other exemplary embodiments, the identification can be done by means of a mobile phone, whose position is determined in the area of ​​a specific vehicle seat, for example the driver's seat, in the vehicle 1 and which is assigned to a specific user. Furthermore, the identification can be done by means of an input by the user, for example by logging in via a login name by means of an input via the touch screen 2.

[0062] Depending on the operable function, in a further step S2 the possible types of operating actions are determined, that is to say which form of operating action the user can perform in order to make a specific input for operating the operable function. In the case of this exemplary embodiment, firstly different operating modes are determined, for example three-dimensional gestures in the acquisition area 7, gestures on the touch-sensitive surface 4, voice commands and / or actuation of physical operating elements (e.g. keys, pressure and rotary switches, slides and adjustment knobs or buttons).

[0063] In a further step S3, the preferred operating action type is determined, wherein the operating action type of the determined preferred operating action type of the preferred operating mode in the case of this embodiment is determined as preferred. That is to say, it is determined as follows, that is, which operating action type is particularly well suitable in the current situation or for the determined operable function.

[0064] In the case of this exemplary embodiment, the position of the vehicle seat in which the user is located is determined for this purpose. For this purpose, it is firstly possible to determine which vehicle seat the user is actually located in, for example by means of a seat cushion or a detection device, and the determined vehicle seat can also be assigned to an operating system of the vehicle 1 or a certain operable device 9. In the case of this exemplary embodiment, it is assumed that the driver of the vehicle 1 is regarded as the user and is located in the driver's seat. The determined position of the vehicle seat can be preferably assigned to different operating modes.

[0065] If, for example, the driver's seat (e.g. in the case of automated driving) is in a rearwardly moved position, from which operating elements on the dashboard of vehicle 1 are moved away, an operating method that does not require manipulation of operating elements arranged on the dashboard can be determined to be preferred; for example, in this case control by means of voice commands or three-dimensional gestures may be preferred.

[0066] Alternatively or additionally, in another embodiment, it can be determined which type of operating action requires the least physical effort for a certain input or allows the fastest possible operation. For example, scrolling through a list may require multiple actuations of physically or virtually displayed keys, whereas a similar operation by means of a swiping gesture along the touch-sensitive surface 4 or in the acquisition area 7 can be performed significantly faster and with less effort. In another example, a jump to a certain entry in a list can be achieved particularly quickly by means of a voice command, whereas the entry by means of a keyboard requires multiple actuations.

[0067] In the case of further embodiments, other parameters may alternatively or additionally be used to determine the preferred type of operating action or operating method, such as a manual or at least partially automatic driving mode, the type of road traveled on, the geographical surroundings or settings, inputs and preferences of an identified user or driver of the vehicle 1.

[0068] In a further step S4, the operation history for the authenticated user is obtained. In the case of this embodiment, the operation history includes information about the following, that is, by means of which type of operation action the user has made inputs for a determined operable function in the past. In addition, the operation history includes information about the following, that is, in which way the user implements the operation action, such as how to better identify the input with the help of the operation action of the determined operation action type and how the changed execution of the operation action improves recognizability. That is to say, the operation history includes information about the following, that is, whether the user used the operation action of the determined operation action type in the past to facilitate the determined input, and how the user implemented these operation actions.

[0069] For example, in the past, the user did not accurately implement an operating action of a certain operating action type, for example, in the following way, that is, he performed a waving gesture in three-dimensional space at too high a speed, over too small a distance and / or not in a sufficiently straight line. This has led to the following in the past, that is, the recognition of the waving gesture was achieved with a low confidence, wherein, for example, the amplitude of the waving gesture or its direction can only be determined with low accuracy. In the case of another example, the user spoke a voice command unclearly, too softly or too quickly in the past. In the case of another example, the user touched an area at the edge of the operating element when operating a virtual operating element on the touch screen 2, so that the input could not be clearly separated from the operation of adjacent operating elements.

[0070] In the case of this embodiment, the operation history is acquired in such a way that it includes a preset number of inputs of the user. In the case of other embodiments, the determined duration can be preset or the number of operating actions considered in the past can be variably determined during the implementation of the method.

[0071] If it is determined based on the acquired operating history that the user usually uses preferred operating action types for operations and performs corresponding operating actions in a well-recognizable manner, then in a further step S5 the operating actions are acquired, corresponding control signals are generated, and devices of vehicle 1 are actuated.

[0072] If, on the other hand, it is determined that the user rarely or never uses the preferred operating action type or that the user does not optimally perform operating actions of the preferred operating action type, an output with instructions for performing possible operating action types is generated and output in step S6 .

[0073] It should be indicated to the user that he does not perform a possible gesture at all or performs it incorrectly or that he can perform it with less physical effort. The output is then generated individually in the respective case in such a way that the type of operating action (e.g. a certain gesture) that was not performed, performed incorrectly or not performed optimally by the user is indicated.

[0074] In the case of this example, in the case of a specific gesture not being used at a specific location of the operation of the human-machine interface, for example, a schematic hand is displayed at these locations, which shows the implementation of the gesture type that has not been used. In the case of an incorrect or suboptimal implementation of a gesture of a specific gesture type, the intended gesture is recognized and a schematic hand is likewise displayed, which shows the correct or more economical implementation of the gesture in motion.

[0075] In other examples, the correct gesture execution shown in other forms is implemented. Thus, for example, a three-dimensional hand can be displayed in the space in front of the display by means of a holographic or autostereoscopic display, or the correct gesture movement can be displayed in another acquisition area. A possible extension of the method is that, in addition to the visual display, visual suggestions are used to supplement the voice output. Therefore, a prompt can be output via voice output about a gesture that is never implemented in a certain operating situation (but the user correctly implements and recognizes the gesture at other locations); for example, "You can also implement a sliding gesture here." In the case of a measure or non-optimal gesture execution, a voice prompt for implementation and improvement can be output; for example, "You must hold the gesture very briefly at the end" or "You don't need to move your arm so far." This can solve the problem of the implementation of the type of operating action in a more targeted manner and simulate, for example, a natural interaction with a natural counterpart (Gegenüber).

[0076] Furthermore, in the case of a user's identity being acquired by means of a key or by means of biometrics, gesture manipulation and use by different users can be considered individually. For example, in the case of entering a vehicle, the system can already detect using the manipulation history that the user does not perform certain gestures in certain manipulation situations. In this way, it can also be detected that the user has had problems in the past with performing certain gestures correctly or with little physical effort. Individual assistance can then be provided for the user for optimal gesture manipulation.

[0077] Figure 3 An example of an output with a prompt for the implementation of a possible type of operating action is shown. In the case shown here by way of example, a preferred type of operating action is a waving gesture in three-dimensional space, by which different graphic objects 22 to 25 in the operating surface 20 can be moved, in particular for bringing an element to the foreground, selecting and then further operating. The prompt for such a waving gesture is constructed as a diagram of a hand 21 shown schematically in a contour manner. This output is dynamically formed here, that is, the hand 21 shown schematically shows how the waving gesture should be implemented in three-dimensional space in this way. In the case of the operating surface 20 of this embodiment, however, it is also possible to realize the manipulation by other means with the aid of non-preferred operating actions, such as the manipulation by arrow keys, by a waving gesture along a touch-sensitive surface 4, by voice commands or by the manipulation of physical operating elements (such as a rotary push adjuster).

[0078] In the case of this exemplary embodiment, the output is also dependent on the configuration of how the user can improve the execution of the operating action of his preferred operating action type, that is to say, those aspects of the embodiment in which there is a need for improvement are particularly clearly indicated. For example, the hand position during a gesture can be particularly clearly indicated, as can the speed or amplitude of a movement (for example over a particularly long path).

[0079] In the case of another embodiment, the presentation of the prompt for the implementation of the possible operating action type is realized by the output of a text prompt. In the case of a further embodiment, the output can be realized by means of a holographic or autostereoscopic display, in particular for showing three-dimensional gestures. In the case of a further embodiment, the output can alternatively or additionally be acoustically perceptible, for example by a voice prompt for the implementation of a preferred operating action type. In particular, the output can be realized by means of an output method that prompts a preferred operating method, for example by the output of an acoustically perceptible prompt for the operation by means of a voice command or by emphasizing a manipulable physical or virtually displayed operating element (for example by brightness, color or dynamic effects such as flashing).

[0080] After the output, in a further embodiment, in step S5 the user's operating action can be captured as input and a corresponding control signal generated in order to operate the device 9 of the vehicle 1. This operation can also be achieved by means of an operating action of a non-preferred operating action type.

[0081] With the acquired operation actions, the operation history is updated.

[0082] In the case of another embodiment of the method according to the invention, an operating action of the user is captured, for example a gesture or a voice command in the capture area 7, wherein the operating action, however, is not assigned to the currently available operation of the device 9 of the vehicle 1. This can result in that the operating action is not correctly recognized and is assigned to an incorrect operating action type or that the user performs an operating action of an unavailable operating action type, for example because he is not aware of which operating action type is currently available or how these operating action types should be implemented. The intention hypothesis is determined, that is, the system forms an assumption about which operating action types may be intended by the user. The output is then generated with a prompt for the type of operating action to be performed according to the intention hypothesis.

[0083] For example, as an intention hypothesis, an operating action type can be determined, which has the highest similarity to the acquired operating action. In this context, it is also considered which possible operating action types are actually currently available. In addition, the operating history can be used to consider such operating action types that the user prefers to use, or it can be determined whether the acquired operating action has a special similarity to operating actions acquired in the past in which the user did not optimally implement a determined operating action type.

[0084] For example, such a voice instruction can be obtained as an operation action, that is, the voice instruction has a similarity in sound or semantics with respect to a possible operation action type, and the voice instruction is then determined as an intention hypothesis. In another example, such a three-dimensional gesture can be obtained as an operation action, that is, the three-dimensional gesture has a direction that is not set at this time, such as a vertical direction instead of a horizontal direction, wherein the operation action type with a horizontal direction is then determined as an intention hypothesis.

[0085] In the case of a further embodiment, a learning process is also carried out with the aid of the operating history and / or with the aid of the acquired operating actions, wherein the recognition of the types of operating actions is improved by the system. For this purpose, methods known per se, such as machine learning, can be used. In this way, in the case of this method, not only a learning requirement for the user is identified, i.e., for whom optimal application of the preferred types of operating actions is made possible, but also a learning requirement of the system is identified, for which the recognition of the types of operating actions determined is improved for the user.

[0086] In the case of another embodiment, the user's pointing gesture is acquired in the acquisition area 7 and the direction is determined by means of the acquired pointing gesture. It is determined as follows, that is, which device 9 of the vehicle 1 the user points to and the device 9 is selected. The operating action acquired immediately afterwards is processed as the operation of the selected device 9, that is, a control signal for the selected device 9 is generated by means of the operating action. For example, the user points to the push door of the vehicle 1 and the automatic opening or closing of the push door is performed by means of a push or waving gesture in a direction. In the case of another example, the user points to the display of the vehicle 1, on which image data or an operating surface can be output, wherein the subsequent operating action is processed as an operation of the output or operating surface. In this way, the number of available operating action types can be limited to the relevant operating feasibility schemes for the determined device 9.

[0087] refer to Figure 4 Exemplary embodiments of outputs generated in the context of the method according to the invention with instructions for different directions are described. Here, starting from the described exemplary embodiments of the operating system and method according to the invention, they are further explained in detail by further exemplary embodiments.

[0088] The output 40 according to this exemplary embodiment comprises arrows 41 to 46, which respectively indicate spatial directions and are partially designed as double arrows. The arrows 41 to 46 are used to indicate in which spatial directions possible gestures can be implemented as operating actions. Depending on the specific operating situation and depending on which type of possible operating action is set by means of the gesture, the arrow or arrows 41 to 46 are simultaneously displayed in the output 40. In addition, the arrows 41 to 46 can be generated in the output 40 in an overlapping manner in addition to any other graphic representations.

[0089] In the case of further exemplary embodiments, it can be outputted in addition whether and in what manner a gesture with a certain hand position is performed during the execution of the gesture, such as an open or closed hand, a palm pointing upward, downward, forward or backward, extended or bent fingers or an orientation of the hand in space. This can be outputted by means of additional graphical objects or certain shapes of arrows 41 to 46. Furthermore, other representations of graphical objects for output can be used as an alternative to arrows 41 to 46, such as other representations of manipulation objects such as hands, in particular schematically by means of contours. Furthermore, this representation can be generated dynamically.

[0090] In the case of this embodiment, the upward arrow 41 and the downward arrow 42 represent gestures that include a movement from top to bottom. With the help of such gestures, for example, scrolling or moving the screen content upward or downward in the vertical direction can be implemented. In addition, the rightward arrow 43 and the leftward arrow 44 represent gestures with a movement toward the corresponding side. With the help of such gestures, for example, flipping or moving toward a side can be implemented. Gestures that include a movement parallel to the surface of the output 40 are particularly referred to as waving or "slide" gestures.

[0091] Furthermore, a perspectively illustrated backward arrow 45 and a forward arrow 46 are provided, that is to say in particular pointing toward or away from the display surface of the output 40. These arrows 45, 46 are in particular shown in perspective, wherein a graphic representation known per se is utilized in such a way that the spatial impression for an observer of the output 40 is formed in such a way that the backward arrow 45 or the forward arrow 46 is directed into the plane of the output 40 and thus away from the observer or out of the plane of the output 40 and thus toward the observer.

[0092] In the case of other exemplary embodiments, other arrow shapes are conceivable, in particular straight or curved paths, by which, for example, the execution of a gesture with a certain trajectory, such as a circular path or a circle segment, is indicated. In addition, in the case of other exemplary embodiments, the output can be designed so that it is indicated that at the beginning and / or at the end of the execution of a gesture (slide or other movement), the posture of the manipulated object should be maintained for a longer or shorter time. In this way, for example, the end of a movement for an input by means of a gesture can be marked.

[0093] For gesture operation, in the case of this embodiment, a gesture group is used which is limited to a certain type of operating action, that is to say a certain form of gesture. In particular, the gesture group is constructed in such a way that the complete operability of the infotainment system is realized. With the help of such a gesture group, the entire system of data and operating levels, which are divided in a hierarchical manner, can be operated, rather than only parts within a single background. Such a gesture group includes in particular a clear number of gestures, so that the user can notice these gestures. At the same time, the gestures of such a gesture group are constructed in such a way that they are particularly clearly distinguishable from each other and can be easily captured, for example with the help of a camera.

[0094] In the case of this embodiment, the gesture group includes five basic gestures. Figure 4 A graphical representation with graphical objects 41 to 46 is described, by means of which instructions for basic gestures can be output.

[0095] In the case of the "Slide, horizontal" gesture, the hand is moved in a horizontal direction to the right or left, in particular in a vertical position, in particular relative to an operating object output, for example, on a display surface or in an output 40. The maintenance of the hand position at the end of the gesture can, for example, in particular cause a continuous function in the following manner, that is, a permanent operation is acquired. In the case of the "Slide, vertical" gesture, the hand is moved in a flat or horizontal position in a vertical direction upward or downward, in particular relative to the operating object. Here, too, the maintenance of the hand position at the end of the gesture can, for example, in particular cause a continuous function in the following manner, that is, a permanent operation is acquired. In the case of the "OK" gesture, the hand moves with the palm upward, wherein the fingertip points in particular in the direction of an operating object, for example, displayed on the screen. In the case of the "Stop" gesture, the hand moves in an open position and with the palm facing the operating object, in particular in the direction of the operating object. Here, too, the hand position at the end of the gesture can be maintained and acquired as a permanent operation. In the case of the "Volume" gesture, the index finger can finally point toward the operating object and move, for example, in the side direction. To facilitate memorability, most of the basic gestures mentioned can be performed in pairs in opposite directions. With a swipe gesture, for example, you can swipe not only to the right but also to the left or up or down. The selection or manipulation of an operating object, for example, for reaching a certain level of hierarchy or a certain menu or for manipulating a query, is achieved by a movement toward the body, while exiting a level of hierarchy or a menu or rejecting a query is achieved by an opposite movement away from the body.

[0096] These paired gestures are used to enable complete navigation through a large number of hierarchically organized menus or other hierarchical levels and correspond to three spatial directions in three-dimensional space. The feasibility of spatial gestures is thus fully utilized with all three dimensions. These additional dimensions enable better structuring of the content compared to known operations with two-dimensional operation (e.g. via a touch screen) and are more consistent with the user's spatial way of thinking.

[0097] The basic gestures mentioned for navigating through menus or hierarchically organized elements are particularly based on the principle that the element to be operated does not need to be selected on the screen in advance, for example, by a pointing gesture towards a certain object. As an alternative, the element to be output is always already selected. The gesture is then evaluated as an input directly related to the element. For example, for an operating object displayed in the middle of the display surface and automatically selected, the "open" process can be directly performed, for example with the help of an "OK" gesture. In order to select the operating object, it can be moved to the middle of the display surface or to another automatic selection area with the help of a waving gesture. In particular in a vehicle, this has the advantage that the content can be reliably and simply selected, even when, for example, vibrations during driving make accurate navigation difficult, for example relative to known operations with the help of a cursor or a touch screen.

[0098] In the case of the gesture group described above, it is also provided that the volume or other parameters of the system are adjusted by means of a corresponding "volume gesture". For example, the user points with an extended index finger or an operating object in the direction of an operating object or a display surface. The volume adjustment can be displayed by showing a volume display as a pop-up window, wherein the volume level can be adjusted by lateral movement of the finger. In this case, the volume display in particular follows the movement of the finger and the volume is increased or decreased.

[0099] Furthermore, option gestures can be defined which do not serve for menu navigation, but rather serve as shortcuts for calling up certain functions. In particular, three option gestures are provided, namely for jumping directly to the home screen (with a stop gesture forming a fist at the end), for calling up a guide that displays and explains the gesture and its function, and for muting the sound playback (downward movement of the open hand). In the case of this exemplary embodiment, the option gesture always leads to the calling up of the same function, regardless of the respective context in which the gesture is performed.

[0100] refer to Figure 5 An exemplary embodiment of an output with a pop-up window for volume adjustment generated in the context of the method according to the invention is described. Starting from the described exemplary embodiment of the operating system and method according to the invention, it is further explained in detail by further exemplary embodiments.

[0101] The output 50 according to this embodiment includes a graphic object 51, which here represents the currently played media data, and a volume operation object 51, which outputs another element of the output 50 as a pop-up window superimposed. The volume operation object 51 includes a volume symbol 52 and a scale object 54, and the scale object 54 is configured as a horizontal straight line, wherein different positions extending longitudinally are assigned to different values ​​of the volume, in particular the volume rising from left to right. An indicator object 53 (here a filled circle) is arranged at the scale object 51, wherein the position of the indicator object 53 at the scale object 51 shows the currently set value of the volume.

[0102] In the case of this embodiment, when the user points his extended index finger to the output 50 and performs a horizontal movement during the playback of the media data displayed by the graphic object 55, the user reaches the output 50 shown. According to the movement of the index finger (in particular proportionally thereto), the pointer object 53 in the output 50 is moved along the scale object 51 and the set value of the volume is changed by the same degree.

[0103] In the case of this embodiment, the volume operating object 51 can also be operated by contacting the touch screen 2 of the output 50. The user can, for example, contact the indicator object 53 and move along the scale object 54 by means of a swiping gesture along the touch screen 2. In addition, it is possible to set a value of the volume assigned to a position by contacting the scale object 54 at a certain position, wherein the indicator object 53 is moved to the contacted position.

[0104] The above-described pointing gesture with the extended index finger in the direction of the output 50 (followed by a movement in the horizontal direction) is referred to in particular as a "volume gesture" in the case of the exemplary embodiment. In this case, it is in particular an option gesture, which in the case of the exemplary embodiment can be applied independently of the context of the current output. This means that, independently of what is output in the output 50, the currently set volume can be displayed and changed with the help of the volume gesture. This simplifies the operation, since a specific context or content must be called up in order to change the volume.

[0105] In particular, the volume gesture and the further option gestures in this example can be used in any context, that is to say independently of the currently active display or currently selected operating object. For example, the volume gesture for volume adjustment and the gesture for muting affect the volume of the currently active sound context, similar to similar volume regulators known per se, which use, for example, a rotary knob.

[0106] In the case of the exemplary embodiments of the method according to the invention described in this description, different operating modes can be used for operating a function. In this case, a particularly preferred operating mode is determined, which specifies in which manner the operation can be performed particularly advantageously according to the specific situation. The operation can be carried out by means of the preferred operating mode, for example, particularly reliably, quickly, with less physical effort and / or without excessive demands on the user's attention.

[0107] Possible operating methods include, for example, gestures during contact with a surface or operating object, manipulation or execution of gestures in three-dimensional space, manipulation of similar operating elements (such as switches, buttons or controllers), voice commands, use of remote control or gesture operation in combination with voice commands.

[0108] The operation of the infotainment system can be realized, for example, completely via different ways, in order to give the user a personally preferred mode of operation. Thus, some users prefer voice operation in principle, other users prefer gesture operation. In addition, the user can freely select one or another mode, for example gesture as a voice alternative, depending on the situation, so as not to disturb the sleeping front passenger. In addition, in the case of this example, it is suggested to the user which type of operating action and / or which mode of operation is best suitable now and for the currently operable functions. The preferred mode of operation is determined in the case of this embodiment by means of the following logic:

[0109] In particular, gesture operation can be the preferred mode of operation when there are "unclear", unnamed operation targets, when the user is, for example, performing a search or navigating through a row of objects ("browsing"), and when, for example, the precise name of a particular target is cumbersome to enter. In addition, gesture operation can be preferred in the case of shorter interactions or when verbal interruptions in certain situations are disruptive. In addition, it can be preferred for spatial selection or spatial operation.

[0110] Operation by means of voice commands can be particularly advantageous when the control targets are clearly named, for example when the user can easily pronounce the name of the function or device to be operated. The user can also jump through multiple hierarchical levels particularly easily by means of voice commands, without having to first perform a long search. In addition, natural language input can be provided.

[0111] Operation by means of remote control may then be preferred if an operation that is particularly trustworthy to the user is to be ensured.

[0112] Operation by means of a combination of gestures and voice commands may then be preferred when interaction is to be performed with units whose names are unclear or cumbersome, or when special operating actions for activating voice control, such as pressing a button or a specific code word, are to be avoided.

[0113] In the case of this example, gesture operation is determined as the preferred operating mode when a search or browsing is to be performed, for example, for finding a name or address or for performing a search in a list of movie or music titles. In addition, gesture operation is preferred for shorter interactions (e.g., the receipt of a call), for the operation of operating elements or environmental elements that can be selected most easily in space or whose names are unknown (e.g., in order to operate a certain window or adjust a certain lighting device) and when interrupted by voice commands (e.g., during a call or when the front passenger is disturbed).

[0114] In particular, in the case of the example of a system, although a plurality of operating modes for an operation are supported, the system nevertheless indicates the respectively best suitable form of the operation in the output.

[0115] For example, a gesture operation for accepting an incoming call is indicated in such a way that a visual element shows in a similar manner for different operating options how the associated operating gesture should be carried out in a dynamic motion. In particular, the movement of the operating element in the direction in which the corresponding gesture should be carried out is shown. At the same time, the operating element in this example is also operable itself, for example by touching the display surface in the area of ​​the operating element. In the case of other examples, the possibility of voice operation can also be provided; a prompt for the corresponding possibility is then output, for example, by means of a displayed text repeating the corresponding voice command or by means of a symbol for voice control.

[0116] refer to Fig. 6A and 6B Exemplary embodiments of outputs with instructions for carrying out an operating gesture generated in the context of the method according to the invention are described. Starting from the described exemplary embodiments of the operating system and method according to the invention, these are further explained in detail by further exemplary embodiments.

[0117] According to this embodiment, the output 60 is Fig. 6A The situation shown in FIG. 6 includes a pop-up window 61, which outputs additional elements of the output 60 in a superimposed manner. Information about the incoming call, in particular the name and face image of the caller, is output in the area of ​​the pop-up window 61, wherein this information is associated with a database with contacts and associated additional information. In addition, the pop-up window 61 includes an operating element 62 for accepting and a further operating element 63 for rejecting the incoming call.

[0118] In the case of this embodiment, the operating elements 62, 63 can be operated by a tap gesture, wherein the user briefly touches the touch screen 2 at the position of the corresponding operating element 62, 63 in order to implement the corresponding assigned function. In addition, gesture operation is provided in the case of this embodiment. In the case of this example, it is determined that gesture operation is the preferred operating method, that is, the operation for accepting or rejecting a call is preferably implemented by the gestures correspondingly assigned to these functions. This preference is caused in the case of this embodiment because the user's seat position is located far away from the touch screen 2 and it can therefore only be touched with difficulty.

[0119] The gesture for accepting or rejecting a call is thus output, i.e. the operating elements 62, 63 are dynamically displayed. Fig. 6A , this is shown by dynamic objects 64, 65. To accept a call, the user should perform a gesture in which the user moves his hand away from the plane of the output 60 in a direction perpendicular to it, for example in accordance with a "wave", "pull in" or "take" gesture. Conversely, to reject a call, the user should perform a gesture in which the user moves his hand toward the plane of the output 60 in a direction perpendicular to it, for example in accordance with a "push away", "reject" or "reject" gesture. The corresponding operating elements 62, 63 are shown dynamically and perspectively in such a way that they appear to have a movement in a direction that corresponds to the corresponding movement of the gesture.

[0120] In the case of further exemplary embodiments, the output 60 is generated by means of three-dimensional display techniques, for example by means of autostereoscopic display or a holographic display.

[0121] exist Fig. 6A In the situation shown in , an animation output is then generated which includes operating elements for accepting and rejecting calls, wherein the operating elements are shown moving in the direction of the gesture to be performed. The movement of the operating object away from or towards the driver is shown with the aid of a perspective illustration, wherein red (reject) and green (accept) circles appear to move spatially in the display with the aid of the display of a large number of smaller or larger circles.

[0122] exist Figure 6B In the case shown in FIG. 6 , the output 60 includes a background illustration 66, which here corresponds to the playback of the video data. In addition, the output 60 includes operating elements 67, 68, 69a, 69b, which are similar to those in FIG. Fig. 6AThe operating elements 62, 63 in the case of can not only be operated by touching the touch screen 2, but also used to output a prompt for a possible operating gesture. In the case of this embodiment, especially when an operating intention is detected, for example, when the operating object enters the acquisition space 7 or when the touch screen 2 is touched, the operating elements 67, 68, 69a, 69b are then displayed.

[0123] As already mentioned above Fig. 6A As explained, here, operating elements 67, 68, 69a, 69b are also shown dynamically. It shows a movement that matches the movement of possible operating gestures. For example, operating element 67 is provided for stopping playback and is constructed with corresponding symbols. Operating element 67 is also shown in an animated manner so that it seems to be away from the user. The corresponding operating gesture for stopping includes a movement toward output 60, for example, in accordance with a "press" gesture. In addition, in the case of this embodiment, the operating element 68 for calling the associated menu (Kontextmenüs, sometimes referred to as a shortcut menu) is shown to move downward and the corresponding operating gesture includes waving downward. Operating elements 69a, 69b for playback (operating element 69a) or forward (operating element 69b) are shown animatedly with a movement to the left or right, which also matches the movement of the corresponding operating gesture, especially the waving gesture.

[0124] In the case of other exemplary embodiments, the operating gestures may also be configured in other ways or may be combined differently. In particular, the gestures provided are based on the above reference to the basic gestures. Figure 4 The groups described may optionally be supplemented by the selection gestures also described above.

[0125] In the case of this embodiment, not only the menu hierarchy is navigated through by means of gesture operation, but also the background is operated. Different functions are set due to the difference in the background. Fig. 6A and 6B As described, for example, an incoming call can be accepted or rejected. In addition, the currently playing movie can be commanded (gestured) or stopped in both directions. In addition, a context menu can be selected for setting additional options, such as subtitles or voice. The variety of necessary functions can be operated with the help of the gesture group described above, wherein, for example, the rejection of a call is realized by the same stop gesture as the movie is stopped. In order for the user to know which operating options are available to him in which context, it is displayed to the user depending on the context which gestures are possible and which functions they have respectively. In addition, it can be output which is the preferred operating method.

[0126] In the case of the embodiment for an incoming call, two operating objects 62, 63 are displayed as "bubbles", and in the case of the movie example, multiple operating objects 67, 68, 69a, 69b are displayed as bubbles. These bubbles contain a symbol and optionally a color, whereby the function is output. For example, a red receiver symbol represents the end of a call or a stop symbol represents a stop. In addition, the bubble moves in the direction in which the corresponding gesture is to be performed, that is to say to the right or left, up or down, or perspectively toward the user or away from him.

[0127] In the case of a display design, the user is prompted with available operating action types or other preferred operating action types for gesture control in such a way that they are displayed in a specific context, thereby reducing the memory expenditure for the user and the number of unwanted operations is reduced.

[0128] Reference above Fig. 6A and 6B The described illustrations are particularly suitable for informing the user in a simple and intuitively accessible, but relatively unobtrusive and subtle, manner which operating options are available to him in the respective operating situation. Alternatively or in addition, other outputs can be used to request the user to use certain operating gestures. For example, the above reference Figure 4 The described arrows 41 to 46 can be output, wherein such additional objects are often perceived as interfering. In contrast, the spatially displayed movement of the element in the gesture direction can also be output without additional graphical objects, so that the output operating surface is not filled.

[0129] refer to Figure 7 An exemplary embodiment of an output with a pop-up window for controlling a lighting device generated in the context of the method according to the invention is described. Starting from the described exemplary embodiment of the operating system and method according to the invention, it is further explained in detail by further exemplary embodiments.

[0130] The output 70 according to this embodiment comprises a graphic object 75, which here represents the currently played media data, and a pop-up window 71, which superimposes and outputs further elements of the output 70. The pop-up window 71 comprises a brightness symbol 72 and operating elements 73, 74 for switching the lighting device on or off.

[0131] In the case of this embodiment, when the user points his extended index finger at a device of the vehicle 1 during the playback of the media data displayed by the graphic object 75 and keeps the index finger for a preset time, the user reaches the output 70 shown. It is determined according to the direction of the index finger which device of the vehicle 1 the user is pointing at, in the case of this embodiment, the lighting device. A pop-up window 71 is then generated, which includes operating elements 73, 74, which prompt possible operations. In the case of this embodiment, there are also different possible operating methods here, especially contact of the touch screen in the area of ​​the operating elements 73, 74 or voice commands. In the case of this embodiment, additional operation by voice command is preferred, for which the operating elements 73, 74 are constructed as text elements. The user is thus informed by which voice command he can operate the device.

[0132] In addition to this, in the case of this exemplary embodiment, operation can also be performed by means of other pointing gestures, wherein the user points to one of the operating elements 73, 74 in the pop-up window 71 in order to operate this operating element and trigger the function linked thereto. It can also be provided that the operation can be performed by means of other operating gestures, for example swiping down to close or swiping up to switch on.

[0133] exist Figure 7 In the case of the embodiment shown in , the operation can then preferably be carried out by means of gestures in combination with voice commands. This mode of operation is determined to be preferred when a device of the vehicle is to be operated and the wake-up phrase determined for this purpose is to be used to activate a purely voice command. In the case of this example, the user points to the device to be operated, which in this example is a lamp in the vehicle interior. The user then continues the operation by means of voice commands. In order to show the user that voice operation can be carried out after indicating the gesture, a pop-up window with possible voice options is displayed.

[0134] refer to Figures 8A to 8D Exemplary embodiments of outputs generated in the context of the method and its operation in different operating modes are described. Starting from the described exemplary embodiments of the operating system and method according to the invention, these are further explained in detail by further exemplary embodiments.

[0135] exist Fig. 8A In the case of the situation shown in , the operating system is in approach mode, that is to say the user with his seat is arranged in the range of action of the touch screen 2. The output 80 comprises a plurality of widgets 81, which are arranged in a matrix form next to each other and one above the other. The preferred operating method in the case of this exemplary embodiment is operation in approach mode by contacting the touch screen 2 and, if necessary, gestures along the touched surface of the touch screen 2.

[0136] exist Figure 8BIn the case of the situation shown in , the automatic driving mode of the vehicle 1 is activated and the seat of the user (who no longer has to manually intervene in the control of the vehicle) is moved backwards. The touch screen 2 can thus be difficult to reach and the operating mode is therefore switched to the distance mode. The preferred mode of operation is now operation by means of gestures in the acquisition area 7 in three-dimensional space, in particular in the interior of the vehicle 1. In the case of a change of operating mode, the output 80 is also changed so that the display is now optimal for the new preferred mode of operation.

[0137] That is to say, in the case of this embodiment, the operating system can be operated not only by means of a touchpad but also by means of gestures. In the case of at least partially automated driving modes, in particular in the case of Level 4 or Level 5 degrees of automation, the driver's seat is moved back in particular in order to enable a higher seat comfort, which, however, also makes the operation of the touch screen on the instrument panel difficult due to the inaccessibility of the display. In this case, gesture operation should be provided as the preferred operating method. For this purpose, the display design must be adapted, since not all displays are well suited to gesture operation. In addition, the displayed content may otherwise be difficult to see due to the greater seat distance being too small.

[0138] In the case of this example, the display that is optimal for gesture operation for a seat position far from the display is displayed in "distance mode". In this case, in the case of this example, only three "backgrounds" are displayed simultaneously, which are designed as tiles, widgets or windows in the case of this exemplary embodiment. These are shown large enough to be recognizable even from a greater distance.

[0139] Since a simpler structure of the display is useful in the case of operation by means of gestures, the objects 82, 83 of the output 80 are now arranged side by side. Here, the object 82 arranged in the middle is always automatically selected, while the other object 83 is arranged next to it. In the case of this exemplary embodiment, the selected object 82 is also highlighted in the following way, that is, it is output with a larger surface, a special edge and with a stronger brightness than the other object 83.

[0140] That is to say, in the case of the display in the distance mode, the background is arranged in a single row, while it is arranged in multiple rows for touch operation. The middle background or the middle tile window is always in focus, that is to say, it is always automatically selected and visually larger, brighter and prominently displayed. This background can then be operated immediately, for example to start the movie shown therein, without having to select it in advance. In the case of other embodiments, the background is arranged in multiple rows and is selected by means of a gesture, such as a pointing gesture, before opening or activation.

[0141] exist Figure 8C In the case of the situation shown in FIG. 8 , it is shown how the user can move the graphical objects displayed in the output 80 by swiping his hand 84 sideways in the direction indicated by the arrow 85. In this case, the selected object 87 is moved from the center to the side and the other object 86 is moved to the position of the object which was automatically selected at this time. In the case of this exemplary embodiment, it is provided that in addition to the swiping gesture sideways in three-dimensional space, a touch-sensitive swiping gesture along the touch screen 2 and / or a voice command can also be used to move the objects 86, 87.

[0142] exist Fig.8D In the case of the situation shown in , the navigation through a hierarchically organized database or menu structure is shown starting from the situation shown in the middle. In addition, the content of the infotainment system can be structured hierarchically. The user can manipulate the object 82 currently shown in the middle and thus automatically selected in such a way that the object is activated or opened or is changed to the next hierarchical level; the possible types of operating actions that can be used for this purpose are shown by prompts 88, which can also be shown in an analogous manner in output 80. In addition, the user can stop the process for the selected object 82 or change to the next hierarchical level; the possible types of operating actions that can be used for this purpose are shown by prompts 89.

[0143] In order to activate the opening of the background or to change to the next hierarchical level, the prompt 88 shows the possibility of operation by means of a three-dimensional gesture, wherein the palm is pointed towards the user and a movement towards the user is performed. Alternatively or additionally, the voice command "OK" can be used. In contrast, the prompt 89 for stopping or changing to the next hierarchical level shows a prompt for a three-dimensional gesture, wherein the palm is pointed towards the touch screen 2 or towards the output 80, and wherein a movement towards the output 80 is performed. Alternatively or additionally, the voice command "Stop" can be used.

[0144] also, Fig.8D It shows how the transition between the hierarchical levels of the database is represented: starting from the starting situation shown in the middle, in the case of an operation according to prompt 88 (for example, using an "OK" gesture), the selected object 82 is enlarged and further objects 83 are squeezed out of the representation. The representation then corresponds to the approach of the selected object 82 to the user or the movement of the user's perspective towards the selected object 82. Subsequently, further objects of the next hierarchical level are enlarged in the same manner as in the representation. Fig.8D82 is displayed in the same manner as shown in the middle in FIG. 8 , that is, the object in the middle is automatically selected and the other objects are arranged next to it. In contrast, in the case of an operation according to prompt 89 (for example, using a “stop” gesture), the selected object 82 is reduced, wherein the other object 83 can also be reduced. This corresponds to the situation explained above, in which the selected object 82 is moving away from the user or the user seems to be moving away from the selected object 82.

[0145] The output 80 is generated in particular such that the selected object 82 and the further object 83 are shown next to one another, such that objects of different hierarchical levels or menu levels appear arranged one behind the other. The output 80 is generated in particular in perspective such that the arrangement is shown along a three-dimensional structure, for example along concentric rings or rollers. When changing the hierarchical level or menu level, the perspective of the displayed view is shifted to objects of another hierarchical level.

[0146] In the case of this embodiment, the backgrounds or objects 82,83 on the same level are shown side by side and slightly perspectively offset backwards as on a carousel. The external tile windows or objects 82,83 are shown cut to illustrate that there are other contents beside them. In the case of opening the background or object 82, a conversion to a deeper menu level is performed and the zoom to the corresponding tile window is shown, wherein the selected middle tile window is briefly enlarged. Then the updated display of the three backgrounds on the deeper menu level at that time is shown again after zooming out.

[0147] The closing of the background or object 82 and the conversion to a higher menu level are output as a perspective away from the tiled window. In the display, the automatically selected background or object 82 in the middle is reduced and then enlarged again until the updated illustration of, for example, three backgrounds on the higher menu level at that time.

[0148] The described visual animation outputs to the user as a visual aid the operating direction in which the user moves through the menu hierarchy, so that the user is better oriented with respect to the menu structure and his current position within the hierarchy.

[0149] In another embodiment, the list contents, for example the entries of a phone book, are also output vertically instead of horizontally. Here, too, the entries are shown in a perspective arrangement along the roller or carousel structure. The upper and lower elements are shown accordingly cut away in order to illustrate that there are further contents next to them. The plane transformation can be as described above with reference to Fig.8D Execute as described.

[0150] In addition to the visual aids to the plane transformation, an auditory or tactile output may also be generated.

[0151] Reference numerals list

[0152] 1 Vehicle

[0153] 2 Touch screen

[0154] 3 output unit; display unit

[0155] 4 Acquisition unit; touch-sensitive surface

[0156] 5 acquisition unit; camera

[0157] 6 output units; speakers

[0158] 7 Get Region

[0159] 8Control Unit

[0160] 9 Equipment; Multimedia Unit

[0161] 20 operating surfaces

[0162] Schematic diagram of 21 moves

[0163] 22 to 25 graphic objects

[0164] Steps S1 to S6

[0165] 40 output

[0166] 41,42 Arrows ("slide, vertical")

[0167] 43,44 Arrows ("slide, horizontal")

[0168] 45,46 arrows (perspective)

[0169] 50 output

[0170] 51 Volume operation object

[0171] 52 Volume Symbols

[0172] 53 Pointer Object

[0173] 54 scale objects

[0174] 55 Graphics Objects

[0175] 60 output

[0176] 61 Pop-up window

[0177] 62 Operation elements (acceptance)

[0178] 63 Operational Elements (Rejection)

[0179] 64 dynamic objects (forward)

[0180] 65 Dynamic Objects (Backward)

[0181] 66 background image

[0182] 67 operating elements (stop)

[0183] 68 operating elements (context menu)

[0184] 69a, 69b Operation element (release)

[0185] 70 output

[0186] 71 Pop-up windows

[0187] 72 brightness symbol

[0188] 73 Operating elements ("Open (An)")

[0189] 74 Operating elements ("Close (Aus)")

[0190] 75 Graphics Objects

[0191] 80 output

[0192] 81 widgets (proximity mode)

[0193] 82Selected objects

[0194] 83 Additional objects

[0195] 84 lots

[0196] 85 Arrow

[0197] 86 Additional objects

[0198] 87Selected objects

[0199] 88 instructions for the "OK" hand gesture

[0200] 89 instructions for the "stop" gesture.

Claims

1. A method for operating an operating system in a vehicle, wherein: An output with at least one first operating object is generated and output in an output area; wherein The first operation object is assigned to a first element of a first hierarchical level of a hierarchically organized database; An operation gesture is acquired, wherein the acquired operation gesture includes a movement direction of the manipulation object perpendicular to the output area; and The output is changed depending on the movement direction of the operating gesture in such a way that the first operating object is replaced by a second operating object; wherein, In case of changing the output, the movement of the first operating object in the movement direction of the operating gesture is outputted perspectively; and wherein The second operation object is assigned to a second hierarchy level of the hierarchically organized database.

2. The method according to claim 1, characterized in that The movement of the first operating object is output in a three-dimensional graphical form in a case where the output is changed.

3. The method according to claim 1 or 2, characterized in that: Depending on the direction of movement of the operating gesture, the second operating object is assigned to a hierarchical level that is one level above or one level below the hierarchical level of the first operating object.

4. The method according to claim 1 or 2, characterized in that: The output includes at least two first operation objects which are assigned to the same hierarchy level and are arranged in a row.

5. The method according to claim 1 or 2, characterized in that: The outputted exactly one first operand is marked as the currently selected element.

6. The method according to claim 1 or 2, characterized in that: The operation gesture includes a push gesture or a pull gesture using a hand of a user.

7. The method according to claim 1 or 2, characterized in that: The operating system has at least a first and a second operating mode; wherein The output is configured depending on the operating mode; and A trigger signal is acquired, wherein a changeover between the first and the second operating mode is performed depending on the trigger signal.

8. The method according to claim 7, characterized in that The trigger signal is detected in the event of a change in seat position.

9. The method according to claim 7, characterized in that: The trigger signal is acquired depending on a change in a driving mode of the vehicle.

10. The method according to claim 6, characterized in that The operating gestures are performed with a closed or open hand.

11. Operating systems in vehicles, including Control unit (8); Get cell (4,5); and An output unit (3) having an output area coupled to the control unit (8); wherein: The control unit (8) is configured to generate an output having at least one first operating object and output the output in an output area of ​​the output unit (3); wherein The first operation object is assigned to a first element of a first hierarchical level of a hierarchically organized database; wherein The acquisition unit (4, 5) is configured to acquire an operation gesture, wherein the acquired operation gesture includes a movement direction of the manipulation object perpendicular to the output area; and The control unit (8) is further configured to change the output in dependence on the direction of movement of the operating gesture in such a way that the first operating object is replaced by a second operating object; wherein In case of changing the output, the movement of the first operating object in the movement direction of the operating gesture is outputted perspectively; and wherein The second operation object is assigned to a second hierarchy level of the hierarchically organized database.

Images