Method for operating a display device, and display device

Abstract

The invention relates to a method for operating a display device (10), which has an operating surface (16) with at least two display elements (38, 40), wherein the operating surface (16) comprises two partially overlapping areas (18, 20, 32, 34), each having a display element (38, 40), wherein the display element (38, 40) is actuated, in the area (18, 20, 32; 34) at which an actuation action of an operating element is detected. For this purpose, first an approach of the operating element to the operating surface (16) is detected. Furthermore, the actuation action is detected at an overlap area (22, 36) of the at least two areas (18, 20, 32, 34). Then, a movement of the operating element relative to the operating surface (16) is detected in the overlapping area (22, 36). In addition, the actuation action is assigned to an area (18, 20, 32, 34) as a function of a direction of movement of the detected movement of the operating element. Finally, a display element (38, 40) of the area (18, 20, 32, 34) associated with the detected movement direction is actuated.

Description

[0001] AUDI AG P24641WO.O

[0002] Method for operating a display device and display device

[0003] DESCRIPTION:

[0004] The invention relates to a method for operating a display device, wherein the display device has a control surface, in particular for detecting an actuation action of a control element, wherein at least two display elements are shown on the control surface. A further aspect of the invention relates to a display device.

[0005] Modern vehicles are equipped with display devices, such as touchscreens, which show various display elements. Each display element has a specific function. A specific area or surface on the display device is defined for each display element. This area can also be called the hit area. This area must have a certain size to increase accuracy for the driver and / or passenger while driving, as activating the desired display element can be difficult due to vehicle movement or because the driver cannot constantly look at the display while driving. This area can have a rectangular shape, for example, 15 mm by 15 mm.

[0006] Furthermore, operating a display device in a motor vehicle is particularly difficult because, unlike a mobile phone or tablet, the display device or control surface is not held by the user but is permanently installed in the vehicle. To simplify operation of a display device in a motor vehicle, DE 11 2022 002 985 T5 proposes a method for touch stabilization. The method involves filling the surface of a touchscreen display with a multitude of elements. Each element comprises an active area for registering a touch interaction by the user. The method also includes determining the user's focus area on the surface, which is determined, for example, by the user's gaze. Depending on the focus area, an active area can be enlarged.For example, the active area can be extended to a neighboring element of the multitude of elements.

[0007] DE 10 2012 005 084 A1 proposes a further method for adapting the display of an input device. The input device, particularly for controlling devices in a motor vehicle, comprises a sensor surface sensitive to contact by a foreign object, a control unit configured to define at least one sensitive area of ​​the sensor surface, and to deliver a predetermined detection signal when the sensitive area is touched by the solid. Furthermore, the input device includes means for estimating at least the direction of a change in acceleration acting parallel to the sensor surface. The control unit is configured to temporarily shift the sensitive area in the direction of the change in acceleration under the influence of such a change.

[0008] The disadvantage of these methods is that they are particularly complex to operate and the design of the display area is completely determined by these areas.

[0009] The invention is based on the objective of providing a method for operating a display device that is particularly simple, safe, and reliable. This objective is achieved by the subject matter of the independent claims. Advantageous embodiments of the invention are described in the dependent claims, the description, and the figures.

[0010] One aspect of the invention relates to a method for operating a display device. The display device has a user interface, in particular for detecting an actuation or an operating action of a control element. For example, the display device or the user interface is designed as a touch display or a touchscreen. Preferably, the user interface or the display device is designed as a user interface, in particular as a GUI (Graphical User Interface). For example, the display device has a display or a screen, in particular with a sensor film or sensor surface, which reacts to approach and / or touch and / or movement, in particular by a control element.

[0011] At least two display elements are shown on the user interface. In other words, at least two display elements can be arranged or shown on the user interface. In other words, the user interface can be populated with at least two display elements. For example, the display elements can be designed as symbols and / or icons and / or menus and / or windows and / or buttons and / or list entries. The display elements can represent or depict functions, such as device functions, and / or programs, or be linked to them. The display device and / or the user interface constitutes, in particular, an interface or interface device. Specifically, it is intended that the actuation action on the user interface, which can be designed as a display unit, is performed on the interface device.The display device or interface device may have a human-machine interface on which an actuation action can be performed.

[0012] Furthermore, the user interface comprises at least two areas, with a display element arranged in each area. In other words, the user interface can be divided into at least two areas, with a display element assigned to each area. The user interface can also comprise more than two areas. Preferably, the areas are not visible. For example, the user interface can have a first area and a second area, with the first display element arranged in the first area and the second display element in the second area. Preferably, the areas are rectangular or square. Particularly preferably, the areas are identical. For example, the areas can have a length of 15 mm and a width of 15 mm.

[0013] Furthermore, the two areas in which the display elements are arranged overlap, particularly partially. In other words, the at least two areas can overlap or intersect. Specifically, the user interface can also have more areas, with at least two areas overlapping. If the areas are preferably rectangular, each area can have an overlap area on each side, i.e., up to four overlap areas. If the user interface has, for example, three areas, the first and second areas can partially overlap, and the second and third areas can partially overlap. The second area then preferably has two overlap areas, and the first and third areas each have only one overlap area. Alternatively, each of the three areas can have at least two different overlap areas.Preferably, the user interface has at least one column and / or at least one row. The at least two areas can be arranged in a row and / or in a column and may overlap, particularly partially. One or more areas and / or display elements can be arranged side by side in each row. One or more areas and / or display elements can be arranged side by side or one above the other in each column.

[0014] The overlapping areas are particularly preferred as remaining static. In other words, the overlapping areas preferably do not move.

[0015] Furthermore, the indicator element is activated in the area where an action of a control element is detected. In other words, the indicator element can be activated in the area where the action is detected and / or to which the action is assigned. This area can also be referred to as the hit area. For example, the first indicator element is activated or triggered when the action is detected in the first area. Alternatively, the second indicator element can be activated or triggered when the action is detected in the second area. The action can include approaching the control surface, in particular one of the at least two areas, and / or touching the control surface, in particular one of the at least two areas, and / or moving the control element relative to the control surface.The control element could be, for example, a user's finger, a user's hand, or a pen.

[0016] In this method, a detection device begins by recording the approach of the control element to the operating surface. Specifically, the detection device can detect whether the control element is approaching the operating surface of the display device. For example, the detection device can comprise a camera and / or a sensor, particularly a capacitive sensor and / or a resistive sensor and / or a light barrier, particularly an infrared light barrier, and / or a sensor layer that is arranged on or incorporated into the operating surface. In a further process step, the detection device records the actuation of the control element on the operating surface.The actuation action can be detected by the detection device, in particular the display device, at the first area and / or at the second area and / or at an overlapping area of ​​the at least two areas.

[0017] If the detection device captures an actuation action at an overlap of at least two areas, it further detects, in particular by a control device, a movement of the control element relative to the operating surface at the overlap, and assigns the actuation action to an area of ​​the at least two regions depending on the direction of movement of the detected movement of the control element within the overlap, in particular by a control device. In other words, the detection device can detect that an actuation action is being performed or executed at the area where the two regions overlap or intersect. "Overlap" refers in particular to an intersection of the at least two regions. In a further process step, a movement of the control element relative to the operating surface at the overlap can then be detected.In particular, the detection device detects that the control element moves within the overlap area on the control surface or at a predetermined distance from the control surface. "Movement relative to the control surface" specifically means that the control element moves while the control surface remains stationary. The actuation action can then be assigned to one of at least two areas, depending on the direction of movement of the detected control element within the overlap area. The direction of movement can be detected and / or determined by the detection device. In particular, the...

[0018] The detection device can detect or determine the direction in which the control element moves within the overlap area. The control element can move towards one of the at least two areas. For example, the detection device can determine whether the control element is moving towards the first area or the second area. Preferably, the direction of movement can be detected vectorially or using vectors. Particularly preferably, a user intention can be determined upon a hit, especially of the control element, within the overlap area. In particular, the user intention can be determined by evaluating a movement vector of the control element.

[0019] In a further process step, one of at least two display elements in the area associated with the detected direction of movement or the actuation action is activated. For example, if the control element moves in the direction of the first area, the actuation action is assigned to the first area, and the first display element assigned to the first area is activated. If the control element moves in the direction of the second area, for example, the actuation action is assigned to the second area, and the second display element assigned to the second area is activated. The assignment and / or activation of the corresponding display element can be performed by a control unit of the display device. Accordingly, the process can also be described as a method for actuating a display element.

[0020] If the detection device registers an action in one of the two areas, specifically the first or the second area, and not within the overlap area, then the indicator element corresponding to the area where the action was registered can be activated directly. In other words, if the action is registered outside the overlap area of ​​one of the two areas, the indicator element corresponding to the area where the action was registered can be activated. For example, if the action is registered in the first area outside the overlap area, the first indicator element is activated. Similarly, if the action is registered in the second area outside the overlap area, the second indicator element is activated.Preferably, no evaluation of the movement of the control element or the direction of movement of the control element is required.

[0021] An advantage of the method according to the invention is that it reduces operating errors of the display device. Furthermore, the design of the user interface can be more flexible, and more display elements can be shown on the interface.

[0022] The invention also includes embodiments that offer additional advantages.

[0023] One embodiment provides that the direction of movement is determined by detecting at least two successive points of contact of the control element on or within the control surface. For example, if a user wishes to perform an actuation action using their finger as the control element and touches the control surface at the overlapping area of ​​the first and second areas, the detection device can record, at predetermined intervals, where, at which points, locations, or positions the user touches or taps the control surface within the overlapping area with their finger. The detection device can then determine the sequence of contact points, specifically whether the contact points approach a particular area, and assign this sequence to that area as the actuation action.In particular, finger movement direction can be determined or detected by imprints on the control surface. This allows for a particularly simple and reliable definition of an area and highly reliable operation of the display device.

[0024] According to an advantageous embodiment, the direction of movement is determined by detecting a changing, particularly over time, extent of the contact area of ​​the control element on the operating surface. For example, if the user wishes to perform the actuation using their finger as the control element and touches the operating surface at the overlap of the first and second areas, the detection device can, at a predetermined time interval, detect how the user's fingerprint, as the extent of the contact area on the operating surface within the overlap area, changes or behaves. The detection device can then determine a change in the contact area of ​​the finger, specifically whether the contact area is approaching a particular area, and assign this change as an actuation to that area.During operation, the finger can leave a deformation of the fingertip or a changing fingerprint on the control surface, which can be detected as a changing, especially over time, extent of the contact area. This allows for a particularly simple and reliable way to determine an area or assign it to a specific area of ​​operation.

[0025] Particularly preferably, the direction of movement is determined by detecting at least two successive positions of the control element within the overlap area, wherein the control element moves at a predetermined, in particular constant, distance to the control surface relative to the control surface.

[0026] In particular, the detection device can record how the control element, which moves at a predetermined, especially constant, distance from the control surface, moves relative to the control surface. Specifically, the control element moves parallel to the control surface at a predetermined distance. For example, if the user wants to perform the action with their finger as the control element and moves their finger over the control surface, but without touching it, at the overlap area, this can be recorded by the detection device. The detection device can then, for example, record whether the user's finger, as it moves above the control surface, moves towards the first area or the second area.This type of movement of the control element at a predetermined distance from the control surface can be detected by the sensing device in such a way that the device determines, at predetermined intervals, where or at which points or positions the user's finger is located above the control surface within the overlap area. The sensing device can then determine the trajectory of the finger's position, specifically whether the finger positions are approaching a particular area, and assign this trajectory as an action to that area. In particular, the direction of finger movement can be determined or detected by the finger's movement across the control surface. This allows for a particularly simple and reliable method of determining an area or assigning an action to that area.

[0027] Another embodiment provides that, prior to assigning a control action, particularly an actuation action, to a specific area depending on the direction of movement, a vehicle movement is detected by a sensor device of a motor vehicle. Any movement of the control element associated with the vehicle movement is disregarded when determining or detecting the direction of movement of the control element. In other words, movement transmitted to the control element by the movement of a motor vehicle can be neglected when determining the direction of movement of the control element. Specifically, a separate or additional sensor device, distinct from the sensor device, particularly the display device, detects the vehicle movement.For example, vehicle movement can include vibration of the vehicle, movement when driving over a pothole, cornering, movement of the steering wheel, and / or movement of a shock absorber. The additional detection device can include, for example, a G-force sensor (also known as an accelerometer), a gyroscope, a steering angle sensor, a shock absorber sensor, and / or a camera directed at the interior of the vehicle and / or its surroundings.For example, if the user wants to perform an action using their finger as a control element, and the vehicle is turning right while performing the action, the user's finger may move to the right due to the vehicle's movement. This finger movement, initiated or transmitted to the finger by the vehicle's movement, can be disregarded. For instance, the detection device might register a rightward movement of the finger while the vehicle is turning right. This rightward movement of the finger is then disregarded when determining or assigning the area based on the direction of movement. In particular, when assigning the area based on the direction of movement, the direction of movement can be corrected, especially by the control unit, based on or in conjunction with the vehicle's movement.This prevents incorrect operation, especially while the vehicle is in motion, in a particularly reliable manner.

[0028] Another embodiment provides that a user performing the actuation action is detected, whereby a plausibility criterion of the detected user is taken into account when assigning the area, depending on the direction of movement of the detected control element being moved by the user. In particular, a control unit of the display device can check for plausible user interaction. The plausibility criterion can encompass user interaction or user behavior. This allows an area to be determined or an area to be assigned to the actuation action in a particularly simple and reliable manner.

[0029] Another embodiment provides that the plausibility criterion includes a display element assigned to the user at a predetermined time. For example, if it is 9 a.m. and the user of the display device is the driver of the motor vehicle, it is probable or plausible that the driver is driving "to work" at that time. Therefore, if the driver performs an action on the control panel at 9 a.m. that displays, for example, the navigation destination "home" as the first display element and the navigation destination "to work" as the second, then the user behavior at that time will include, as a plausibility criterion, the activation of the "to work" display element.If the driver moves their finger at 9 o'clock in the overlapping area between the navigation destinations "home" and "work," the action is automatically assigned to the area corresponding to the "work" destination based on a plausibility criterion. Specifically, the direction of movement can be corrected based on the plausibility criterion when assigning the area. This allows for a particularly simple and reliable method of determining and assigning an area to a specific action.

[0030] Another embodiment provides that the user is detected by monitoring whether a driver's seat and / or passenger's seat is occupied, whereby an action is assigned to a driver or passenger as the user. In particular, it can be detected whether a driver's seat and / or a passenger's seat is occupied and whether a driver from the driver's seat and / or a passenger from the passenger's seat is performing the action. The seat occupancy can be detected by a separate or additional detection device, distinct from the primary detection device, particularly the display device. For this purpose, the additional detection device can, for example, include a seat occupancy sensor and / or a camera directed at the interior of the vehicle, particularly at the vehicle's cockpit.

[0031] Another embodiment provides that, prior to assigning an action, particularly an actuation action, to an area depending on the direction of movement, a user characteristic associated with a user is detected. This user characteristic includes a tremor in the user's hand while performing the actuation action and / or the size of a user's finger. This user characteristic is disregarded when determining or detecting the direction of movement of the control element. In particular, the additional detection device can detect which characteristics a user possesses. A control unit can then assign these characteristics to the corresponding user. For example, the detection device can detect that a user's hand trembles while performing the actuation action with their finger as the control element.This hand movement, i.e., the tremor, can then be disregarded or ignored when assigning the direction of movement. In other words, when assigning the area based on the direction of movement, the direction of movement can be corrected using the recorded user characteristic. This allows for a particularly simple and reliable way to determine an area or assign an area to the action being performed.

[0032] Advantageously, a detected actuation action and / or a user attribute assigned to a user and / or a plausibility criterion and / or a detected movement of the control element can be provided or transmitted to a machine unit, in particular the display device, depending on or in response to a vehicle movement of the motor vehicle. This allows the display device to be machine-trained, in particular automatically, to actuate a display element or to operate the display device by means of the machine unit. Thus, future actuation of the display element or operation of the display device can be more precise and efficient.

[0033] For use cases or application situations that may arise during the procedure and are not explicitly described here, it may be provided that, according to the procedure, an error message and / or a request for user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0034] Another aspect of the invention relates to a display device for a motor vehicle. The display device comprises a control surface, in particular a touch-sensitive or capacitive one, wherein the control surface is configured to display at least two display elements. In other words, at least two display elements can be displayed on the control surface. Furthermore, the control surface comprises at least two areas, with one display element arranged in each area, wherein the areas in which the display elements are arranged overlap, in particular partially. The display device also comprises a detection device configured to detect the approach of a control element to the control surface. Furthermore, the detection device is configured to detect an actuation action of the control element on the control surface.Furthermore, the detection device is configured to detect the actuation action at an overlap area of ​​the at least two areas. When an actuation action is detected at the overlap area, the detection device is configured to detect a movement of the control element relative to the operating surface at the overlap area. In particular, when the actuation action is detected at the overlap area, the detection device additionally detects a movement of the control element performing the actuation action relative to the operating surface. Furthermore, the display device includes a control unit, the control unit being configured to assign an area of ​​the at least two regions to the actuation action depending on the direction of movement of the detected movement of the control element within the overlap area.The direction of movement can be detected by the sensing device, particularly before it is assigned by the control unit. Finally, the control unit can be configured to activate one of the at least two display elements in the area assigned to the detected direction of movement.

[0035] Another aspect of the invention relates to a motor vehicle with the display device according to the invention. The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0036] The invention also includes the control unit for the display device or for the motor vehicle. The control unit can comprise a data processing device or a processor unit (processor circuit) configured to carry out an embodiment of the method according to the invention. For this purpose, the processor unit can comprise at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor unit can comprise program code configured to carry out the embodiment of the method according to the invention when executed by the processor unit.The program code can be stored in a data memory of the processor device. The processor device can be based, for example, on at least one circuit board and / or on at least one SoC (System on Chip).

[0037] The invention also includes further developments of the display device and the motor vehicle according to the invention, which have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments of the display device and the motor vehicle according to the invention are not described again here.

[0038] As a further solution, the invention also includes a computer-readable storage medium comprising program code which, when executed by a computer or a computer network, causes it to execute an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data storage medium (e.g., as flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data storage medium (e.g., as RAM - random access memory). The storage medium can be located within the computer or computer network. However, the storage medium can also be operated, for example, as an app store server and / or cloud server on the internet. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code can be provided as binary code, assembly code, source code in a programming language (e.g., C), or a program script (e.g., Python). Alternatively, the computer-readable storage medium can be implemented as a signal containing computer-readable data, such as a time-varying voltage signal or a radio signal.

[0039] The invention also includes combinations of the features of the described embodiments. The invention therefore also includes realizations that each exhibit a combination of the features of several of the described embodiments, provided that the embodiments have not been described as mutually exclusive.

[0040] The following are exemplary embodiments of the invention described. This is illustrated by:

[0041] Fig. 1 shows a schematic representation of a user performing an actuation action on a display device according to an embodiment of the inventive method for operating the display device;

[0042] Fig. 2 shows a schematic representation of the user performing a further actuation action on the display device according to an embodiment of the inventive method for operating the display device; and

[0043] Fig. 3 shows a schematic representation of a motor vehicle with a driver performing an operation on the display device while the motor vehicle is driving.

[0044] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described. In the figures, identical reference numerals denote functionally equivalent elements.

[0045] In connection with Fig. 1 and Fig. 2, the display device 10 and the method for operating the display device 10 or the method for actuating a display element of the display device 10 are discussed in more detail.

[0046] Figures 1 and 2 each show a display device 10. The display device 10 is, for example, designed as a touchscreen. The display device 10 is, for example, arranged in the interior of a motor vehicle. Preferably, the display device 10 is arranged in the cockpit of the motor vehicle. The display device 10 is operated by a control element. The control element in Figures 1 and 2 is a finger 12 of a user's hand 14. However, the control element can also be a stylus, several fingers, or a hand, with which an action is to be performed.

[0047] The display device 10 comprises a control surface 16. The control surface 16 is designed, in particular, as a touch-sensitive or capacitive control surface. The user performs an actuation action on the control surface. This actuation action includes, accordingly, bringing the control element close to the control surface 16 and / or moving the control element relative to the control surface 16 and / or touching the control surface 16 with the control element.

[0048] One or more display elements can be shown or displayed on the control surface 16. The control surface 16 is preferably designed as a touch-sensitive display. The display elements are, for example, designed as icons and / or symbols and / or windows and / or menus and / or buttons. Each display element is associated with a function. Thus, several functions can be displayed or shown on the control surface 16. The display device 10 comprises a detection device which is configured to detect an actuation action of the control element on the control surface 16 or at a predetermined distance from the control surface 16. For example, the detection device can detect an actuation action of the control element approaching the control surface 16 and / or a movement of the control element relative to the control surface 16 and / or a touching of the control surface 16 by the control element.Depending on the detected actuation action, a control unit of the display device 10 can then be configured to actuate or select a display element. Depending on which display element is actuated by the control element, i.e., touched or pressed, a different function is triggered.

[0049] Especially while driving a motor vehicle, the vehicle's movement makes it difficult to operate the display device 10 or the control surface 16. To facilitate operation or to increase the accuracy of selecting one of the displayed elements, the control surface 16 is divided into predetermined areas. In Figures 1 and 2, the control surface 16 is divided into two areas. In other words, the control surface 16 has at least two areas – a first area 18 and a second area 20. Both areas 18 and 20 are identical. In particular, both areas 18 and 20 are preferably not visible to the user. Both areas 18 and 20 have a rectangular shape. For example, the length and width of each area 18 and 20 can be 15 mm. Furthermore, both areas are static and do not move.

[0050] Each area 18, 20 is assigned a display element. In other words, a display element is located in each area. For example, the first area 18 has a first display element, and the second area 20 has a second display element. The detection device of the display unit 10 is configured to detect an action being performed on the respective area 18, 20. The control unit is then configured to assign the action to an area and / or actuate a display element, depending on the detected action. If the user's finger 12 touches or approaches the first area 18, the detection device transmits a detection signal to the control unit indicating that an action being performed on the first area 18 has been detected. The control unit then actuates the first display element or the function associated with the first display element.If the user's finger 12 touches or approaches the second area 20, the detection device transmits a detection signal to the control unit indicating that an action has been detected in the second area 20. The control unit then activates the second display element or the function associated with the second display element.

[0051] As shown in Figures 1 and 2, the first area 18 and the second area 20 overlap. In other words, the first area 18 and the second area 20 have an overlap area 22, or a common intersection area. If the detection device detects an actuation action within the overlap area 22, it also detects a movement of the operating element, in this case the finger 12, relative to the operating surface 16. Furthermore, the detection device detects the direction of movement of the operating element. The control unit of the display device 10 is then configured to assign the actuation action to one of the at least two areas 18, 20, depending on the direction of movement of the detected movement of the operating element.

[0052] As shown in Figures 1 and 2, the detection device captures a touch point on the control surface 16 at predetermined time intervals. Thus, several touch points are captured sequentially. For example, three consecutive touch points are captured. In Figure 1, the user's finger 12 moves on the control surface 16 within the overlap area 22 in the direction of the first area 18. The detection device captures the first touch point on the control surface 16 at time T1, then the second touch point at time T2, and the third touch point at time T3. The control unit or the detection device can determine the direction of movement of the finger 12 based on the path of the three touch points within the overlap area 22. The direction of movement in Figure 1 is represented by arrow 24.Within the overlap area 22, the finger 12 moves towards the first area 18. The control unit assigns the direction of movement to the first area 18. Accordingly, the control unit activates the first display element through the detected actuation action. In Fig. 2, the user's finger 12 moves on the operating surface 16 within the overlap area 22 in the direction of the second area 20. The detection unit detects the first point of contact on the operating surface 16 at time T1', then the second point of contact on the operating surface 16 at time T2', and the third point of contact on the operating surface 16 at time T3'. The control unit or the detection unit can determine the direction of movement of the finger 12 based on the path of the three points of contact within the overlap area 22. The direction of movement in Fig. 2 is represented by arrow 26.Within the overlap area 22, the finger 12 moves towards the second area 20. The control unit then assigns the direction of movement to the second area 20. Accordingly, the control unit activates the second display element through the detected actuation action. The timing is shown in Fig. 1 and Fig. 2 below the operating surface 16, serves only for clarification and is not actually depicted or forms part or element of the operating device. Fig. 3 shows a driver 28 in a motor vehicle 30. The driver 28 operates the display device 10 while the motor vehicle 30 is in motion. In the embodiment of Fig. 3, the display device 10 has four areas on the operating surface 16: the first area 18, the second area 20, a third area 32, and a fourth area 34. As already described in the explanations for Fig. 1 and Fig. 2, the display device 10 has four areas on the operating surface 16: the first area 18, the second area 20, a third area 32, and a fourth area 34.The first area 18 and the second area 20 overlap. The first area 18 and the second area 20 have the overlap area 22. Furthermore, the third area 32 and the fourth area 34 overlap. The third area 32 and the fourth area 34 have a further overlap area 36. Additionally or alternatively, the first area 18 and the third area 32 can also partially overlap, and the second area 20 and the fourth area 34 can also partially overlap. All four areas can then each have two different overlap areas. Each area can, for example, have up to four different overlap areas. The operating surface 16 can therefore have several rows and columns. As shown in the embodiment of Fig. 3, the operating surface 16 can have two rows and two columns or be divided into two rows and two columns, with two areas arranged in each row and in each column.In the first row, at least part of the first area (18) and the third area (32) are arranged, and in the second row, at least part of the second area (20) and the fourth area (34) are arranged. In the first column, the first area (18) and the second area (20) are arranged, and in the second column, the third area (32) and the fourth area (34) are arranged.

[0053] The driver 28 wants to operate one of the displayed elements with their finger 12. The first display element 38 is located in the first area 18, and the second display element 40 is located in the second area 20. For example, the driver wants to activate a navigation function by pressing one of the display elements 38 or 40. In this case, the driver 28 wants to activate the "home" navigation function, which is associated with the first display element 38. However, the vehicle's movement makes it difficult for the driver 28 to operate the function. To detect the driver's action, the detection device senses the approach of the driver's finger 12 to the operating surface 16. The detection device can, for example, comprise a capacitive sensor film or capacitive layer that detects the approach and / or touch of a finger 12.Finger 12 moves within the overlap area 22 of the first area 18 and the second area 20. The detection device detects the movement of finger 12 within the overlap area 22. In a further process step, the detection device detects several successive points of contact on the operating surface 16 within the overlap area. "Points of contact" refers in particular to points or locations on the operating surface 16 that are touched by finger 12 within the overlap area 22 or that are detected within the overlap area 22, even if finger 12 is still at a predetermined distance from the operating surface 16.The detection device or control unit 42 detects the direction of movement towards or within the first area 18 or towards the first display element 38 and assigns the actuation action to the first area 18 and / or the first display element 38. The control unit 42 can then actuate or trigger the first display element 38 or the function associated with the first display element 38. For example, the "home" navigation function can then be triggered, and the vehicle 30 will be navigated "home" from its current location.

[0054] To increase accuracy or to make operation more precise, further factors can be considered during the operation of the display device 10 or during the execution and / or detection of the actuating action. For example, it may be necessary to first identify a user of the display device 10 or a user performing the actuating action on the control surface 16. For this purpose, the motor vehicle 30 may have a further detection device 44, separate from the detection device. The further detection device 44 may be configured to detect which user is performing the actuating action. For example, the detection device 44 may include a camera. The camera may be located in the interior of the motor vehicle 30 and / or directed towards the interior of the motor vehicle 30.The camera can detect whether the driver 28 or operator of the motor vehicle 30, in particular in a driver's seat 46, or a passenger in a passenger seat, is performing the operation. Specifically, the additional detection device 44 checks whether a driver's seat and / or a passenger seat is occupied. Additionally or alternatively, the detection device can detect whether a driver or a passenger is performing the operation at the control surface 16.

[0055] Additionally or alternatively, the control unit 42 may consider a plausibility criterion when assigning the area to the operation. For example, the display elements 38 and 40 may be linked to a plausibility criterion, particularly regarding the user. For instance, the first display element 38, which represents the "home" navigation function, may also be linked to a time as a plausibility criterion. For example, the driver 28 of the vehicle 30 may drive "home" from work at a predetermined time, such as 6 p.m. That is, if the driver 28 operates the display device 10 or performs an operation on the control surface 16 at 6 p.m. and wishes to activate a navigation function, the control unit can consider the plausibility criterion, in this case the time and the display element linked to that time.The plausibility criterion can be provided to a machine control unit of the display device 10. For example, if the driver 28 always activates the same display element 38, 40 at a predetermined time, this can be transmitted or transferred to the machine control unit as a plausibility criterion. The machine control unit is then automatically trained. The control unit 42 can then take the learned plausibility criteria into account when assigning the area depending on the activation action.

[0056] To increase accuracy or to make operation more precise, further factors can be taken into account during the operation of the display device 10 or during the execution and / or detection of the actuation action. For example, the additional detection device 44 can be configured to detect vehicle movement of the motor vehicle 30. Furthermore, the detection device can be configured to detect a movement of the control element associated with the vehicle movement of the motor vehicle 30. The additional detection device can, for example, include a steering angle sensor to detect cornering and / or a shock absorber sensor to detect movement of the shock absorbers of the motor vehicle 30, for example, when the motor vehicle drives over a stone 48 and / or over a pothole.

[0057] In the embodiment shown in Figure 3, the motor vehicle 30 drives over a stone 48 and successively navigates a left turn and a right turn. If the driver 28 performs the actuation action on the control surface 16 during one of these vehicle movements, particularly at the overlap area 22, the control device 42 can be configured to disregard or neglect the movement of the driver's finger 12 associated with the vehicle movement. That is, if the motor vehicle 30, for example, navigates a right turn, the driver's finger 12 would also move to the right.If the additional detection device 44 detects a vehicle movement through a right turn, and simultaneously the detection device of the display device detects a movement of the driver's finger 12 also to the right or in the same direction, this movement of the finger 12 is not taken into account when assigning the direction of movement to a corresponding area, but rather disregarded. Additionally or alternatively, a user characteristic can also be disregarded or neglected during the execution of the operating action. For example, the user characteristic might include a tremor in the hand of the driver 28 or user. This movement cannot be taken into account when assigning the direction of movement to a corresponding area, and is therefore disregarded.

[0058] A user's reactions to the vehicle's movement, i.e., the movement of the control element caused by the vehicle's movement, and / or the user's characteristics can also be provided to a machine learning unit of the display device 10. The machine learning unit is then trained automatically. The control unit 42 can then take the learned reactions and / or user characteristics into account when assigning the areas 18, 20, 32, 34 depending on the actuation action.

[0059] Overall, the examples show how a strategy for minimizing user error on a touchscreen can be provided.

[0060] The hit areas, i.e., the areas between lines or buttons on the touchscreen, are deliberately defined with an overlap. A user's touch between two buttons or areas, i.e., within the overlap, is detected using touch sensors and / or other sensors, such as cameras and / or a user signature based on an electrical signal, and / or based on usage data and / or behavioral data, which, in particular, contextually substantiates the user's intention.

[0061] First, the system, in particular a detection device of the display, can verify that only one user, or a user's finger or hand, is approaching the touchscreen. This can be done using a process of elimination with parameters such as the following: First, it can be checked whether a passenger seat is occupied. Additionally or alternatively, it can be detected or checked whether a hand belonging to the driver or passenger is approaching the touchscreen. Additionally or alternatively, the driver and / or passenger can be exposed to different electrical signals and detected separately by the detection device, in particular a touch controller.

[0062] The detection device, especially the touch controller, can still evaluate, based on vectors for example, which area was intended, especially how the finger was approached and / or positioned, even if the user hits exactly between 2 hit areas or regions.

[0063] Furthermore, the system, and in particular the display device, can learn the following parameters for each user. This learning can be performed by a machine learning unit. For example, a finger delta position, specifically a position relative to the touchscreen or a change in finger position, can be determined depending on vehicle movement, such as during a vibration, a pothole, or a curve. The vehicle movement can be detected or determined by G-force sensors, a gyroscope, steering wheel movement, or shock absorber movement.

[0064] Additionally or alternatively, a correction can be made based on plausible user interaction, particularly learned through user behavior. For example, the user always wants to select the destination "home" at a specific time or a predetermined time, and not the "to work" option below.

Claims

28 PATENT CLAIMS:

1. A method for operating a display device (10), wherein the display device (10) has a control surface (16), wherein at least two display elements (38, 40) are displayed on the control surface (16), wherein the control surface (16) comprises at least two areas (18, 20, 32, 34), wherein a display element (38, 40) is arranged in each area (18, 20, 32, 34), wherein the areas (18, 20, 32, 34) in which the display elements (38, 40) are arranged overlap, in particular partially, wherein the display element (38, 40) is actuated, and the area (18, 20, 32, 34) in which an actuation action of a control element is detected is determined, comprising the steps: - Detection of the approach of the control element to the operating surface (16) by a detection device; - Detection of the actuation action of the control element on the operating surface (16) by the detection device, wherein - if the detection device detects the actuation action at an overlap area (22, 36) of the at least two areas (18, 20, 32, 34), a movement of the control element relative to the operating surface (16) at the overlap area (22, 36) is also detected, and an area (18, 20, 32, 34) of the at least two areas (18, 20, 32, 34) is assigned to the actuation action depending on a direction of movement of the detected movement of the control element within the overlap area (22, 36); and - Activating one of the display elements (38, 40) of the at least two display elements (38, 40) of the area (18, 20, 32, 34) assigned to the detected direction of movement.

2. Method according to claim 1, characterized in that the direction of movement is determined by detecting at least two successive points of contact of the control element on the operating surface (16).

3. Method according to claim 1 or 2, characterized in that the direction of movement is determined by detecting a changing, in particular temporal, extent of a contact surface of the control element on the control surface (16).

4. Method according to one of the preceding claims, characterized in that the direction of movement is determined by detecting at least two successive positions of the control element within the overlap area, wherein the control element moves at a predetermined, in particular constant, distance to the control surface (16) relative to the control surface (16).

5. Method according to one of the preceding claims, characterized in that prior to assignment a vehicle movement is detected by a detection device (44) of a motor vehicle (30), wherein a movement of the control element associated with the vehicle movement is disregarded when determining the direction of movement of the control element.

6. Method according to one of the preceding claims, characterized in that a user performing the actuation action is detected, wherein, when assigning the area depending on the direction of movement of the detected movement of the control element which is moved by the user, a plausibility criterion of the detected user is taken into account.

7. Method according to claim 6, characterized in that the user is detected by detecting the occupancy of a driver's seat and / or passenger's seat, wherein an actuation action is assigned to a driver (28) or a passenger as user.

8. Method according to claim 6 or 7, characterized in that the plausibility criterion comprises a display element (38, 40) assigned to the user at a predetermined time.

9. Method according to one of the preceding claims, characterized in that a user characteristic assigned to a user is recorded before the assignment, wherein the user characteristic comprises a trembling of a hand when performing the actuating action and / or a finger size of a finger (12) of the user, wherein the user characteristic is disregarded when determining the direction of movement of the control element.

10. Display device (10) for a motor vehicle (30) comprising: - a control surface (16) wherein the control surface (16) is configured to display at least two display elements (38, 40), wherein the control surface (16) comprises at least two areas (18, 20, 32, 34), wherein one display element (38, 40) is arranged in each area (18, 20, 32, 34), wherein the areas (18, 20, 32, 34) in which the display elements (38, 40) are arranged overlap; - a detection device which is designed to detect an approach of a control element to the operating surface (16), wherein - the detection device is further configured to detect an actuation action of the control element on the operating surface (16), wherein - the detection device is further configured to detect the actuation at an overlap area (22, 36) of the at least two areas (18, 20, 32, 34), wherein, in the event of a detected actuation at the overlap area (22, 36), the detection device is configured to to detect the movement of the control element relative to the control surface (16) at the overlap area (22, 36); and - a control device (42), wherein the control device (42) is configured to assign a range of at least two ranges to the actuation depending on a to assign the direction of movement of the detected movement of the control element within the overlap area (22, 36).

Images