Method for operating a display device for a motor vehicle

Abstract

The invention relates to a method for operating a display device (2) for a motor vehicle (1), wherein the display device (2) extends at least partially in a longitudinal direction behind a steering wheel (10) of the motor vehicle (1) and at least partially laterally to the steering wheel (10) in a transverse direction, the method comprising: determining element information (20) that describes an operating intention and / or an interest in an element (3, 4, 5, 6, 7) on the display device (2) by a user of the motor vehicle (1); determining situation information (21) that describes a current situation of the motor vehicle (1); determining user information (24) that describes the user;Determining a control command (27) that specifies an arrangement and / or design of at least one element (3, 4, 5, 6, 7) on the display device (2) by applying a control command determination criterion (28) to the determined element information (20) on the one hand and the determined situation information (21) and / or the determined user information (24) on the other hand; and operating the display device (2) according to the determined control command (27).

Description

[0001] The invention relates to a method for operating a display device for a motor vehicle and to a motor vehicle for carrying out such a method.

[0002] A motor vehicle may have a display device that extends over a relatively large area in a front region of the motor vehicle. Such a display device may, for example, extend from a first A-pillar of the motor vehicle to a second A-pillar of the motor vehicle. Various information can, for example, be displayed simultaneously on the display device. Furthermore, the display device may, at least in some regions, have a touch-sensitive surface so that it can function as an operating device. By operating an operating element provided by the display device, a navigation system, a multimedia system, an air conditioning system, and / or a settings menu of the motor vehicle can be operated, for example.

[0003] Due to the potential profusion of display elements and / or controls that can be displayed on the surface of the display device, a motor vehicle driver may be distracted from a driving task. The display device should therefore be operated in such a way that driver distraction is minimized, as the display device is particularly easy and intuitive to operate.

[0004] DE 10 2008 004 970 A1 discloses an operating system comprising an operating unit with at least one operating element for detecting user inputs, as well as a position and / or detection unit for contactlessly detecting the approach of an actuating element, such as a body part, to the operating element. Upon detection of an approach, the operating unit is automatically adjusted by mechanically moving it between a first position and a second position.

[0005] DE 10 2009 032 069 A1 discloses a method for providing a user interface in a vehicle, in which operating objects and / or display objects can be displayed on a display surface arranged in the vehicle. A gesture is detected in a detection space arranged in front of the display surface, and the detected gesture is assigned to an operating object and / or display object and a control command.

[0006] It is the object of the invention to provide a solution by means of which a display device for a motor vehicle is provided which hardly distracts a driver from a driving task.

[0007] This object is achieved by the subject matter of the independent patent claims. Possible embodiments of the invention are specified in the dependent claims, the following description, and the figures.

[0008] A first aspect of the invention relates to a method for operating a display device for a motor vehicle. The display device extends at least partially behind a steering wheel of the motor vehicle, viewed in a longitudinal direction of the motor vehicle. Furthermore, it extends at least partially laterally to the steering wheel, viewed in a transverse direction of the motor vehicle. The display device is preferably arranged between two A-pillars of the motor vehicle and extends from one of the A-pillars to the other. The display device can therefore be arranged over a large area in a front region of the motor vehicle. The display device can comprise a plurality of individual display devices, which can each be directly adjacent to one another and / or spatially spaced from one another. Alternatively, the display device can have a single, continuous surface.Preferably, the display device extends at least partially over a region of a center console of the motor vehicle. The display device can alternatively be referred to as a screen device, wherein the screen device comprises at least one screen. The display device can comprise a head-up display and / or a combination display.

[0009] The method comprises determining element information. The element information describes an operating intention and / or an interest in at least one element provided by the display device. The element can be an operating element in the case of a touch-sensitive surface of the display device. Alternatively or additionally, the element is a display element that displays at least one piece of information on the surface of the display device. The operating intention or the interest in the element relates to a user of the motor vehicle, i.e., the element information describes whether the user of the motor vehicle indicates or has indicated an operating intention and / or an interest in the at least one element. The user here is preferably a driver of the motor vehicle. Alternatively or additionally, it is possible for the user to be a passenger or another occupant of the motor vehicle.

[0010] The determination of the element information can be supported, for example, by an interior sensor device of the motor vehicle, such as a driver observation camera or another interior camera of the motor vehicle. To detect the operating intention and / or interest in at least one element, camera data from the driver observation camera or the interior camera, for example, can be evaluated. For this purpose, an element information determination criterion can be provided, which is applied to the data from the interior sensor device in order to calculate the element information. The element information determination criterion comprises at least one algorithm and / or a rule, the execution of which determines the element information.

[0011] The method comprises determining situation information that describes a current situation of the motor vehicle. The situation information is determined by applying a situation determination criterion to motor vehicle data that describe a journey and / or the surroundings of the motor vehicle. The motor vehicle data can be acquired, for example, using an environmental sensor device of the motor vehicle, such as a camera, a radar device and / or a LiDAR device of the motor vehicle. In this case, they describe the surroundings of the motor vehicle. If the motor vehicle data describe the journey, they can be acquired, for example, using a vehicle sensor in the motor vehicle. The situation information describes, for example, whether or not the motor vehicle is currently in a situation with high traffic density, for example in a busy city center.

[0012] Alternatively or in addition to determining the situation information, the method comprises determining user information that describes the user of the motor vehicle. The user information is determined by applying a user determination criterion to user data describing the user. The user data describing the user can, for example, correspond to the data captured by the driver observation camera or the interior camera, which was used to determine the user's intention to operate and / or interest in at least one element.

[0013] The method comprises determining a control command for the control device. The control command specifies an arrangement and / or configuration of the at least one element on the display device. The control command can, for example, specify a position, a size, and / or a scope of information of the element arranged on the display device. The control command can therefore, for example, provide for repositioning and / or enlarging the element. The control command is determined by applying a control command determination criterion to the acquired element information, on the one hand, and the determined situation information, and / or the determined user information, on the other.To determine the control command, for example, the element information can be taken into account together with the situation information or the element information together with the user information or the element information together with the situation information and the user information.

[0014] The method further comprises operating the display device according to the determined control command. The element information can, for example, describe that the driver is looking at a control element associated with a navigation system of the motor vehicle. It can also, for example, describe that the driver is extending their hand toward the control element to perform an operating action. Furthermore, the situation information can describe that the motor vehicle is located in an inner-city area with high traffic density, for example, during rush hour. Alternatively or additionally, the user information can describe that the driver is stressed.Subsequently, for example, a control command for the display device is determined that provides for the control element to be displayed larger than before and / or that the control element is moved into a focus area of ​​the driver so that the driver can easily recognize and operate the control element in the current traffic situation and / or in a stressed state. Thus, the arrangement and / or design of the element for which the operating intention and / or interest was determined is specifically improved, depending on the situational information and / or user information.

[0015] The described method is carried out, for example, by means of a control device of the motor vehicle, which is designed to determine the element information, the situation information and / or the user information, determine the control command, and control the display device. Alternatively, it is possible for the control command to be determined by an external device to which corresponding data is provided by the motor vehicle. In this case, the determined control command can be provided to the motor vehicle, i.e., transmitted to it, for example, via vehicle-to-vehicle communication or vehicle-to-infrastructure information.

[0016] It can be provided that the situation information and / or the user information are only determined if this information is actually required to determine the control command. The method can then be reduced to determining the element information and the situation information in order to determine the control command based on these two pieces of data. Alternatively, it can be reduced to determining the element information and the user information if the control command is determined based on these two pieces of information.

[0017] The described criteria, i.e., the element information determination criterion, the situation determination criterion, the user determination criterion, and / or the control command determination criterion, are each embodied, for example, as a computer program or part of a computer program. The respective criterion comprises an algorithm and / or at least one rule, the execution of which determines the respective information or control command. For this purpose, the respective data, i.e., for example, the vehicle data and / or the user data, or the respective information, i.e., the element information, situation information, and / or user information, are provided to the respective criterion, if applicable.

[0018] For example, information within the meaning of the invention comprises at least one data element. The respective information, i.e., the element information, the situation information, and / or the user information, can be described by and / or comprise corresponding data.

[0019] In other words, what has been described so far is how, for example, a user's direction of view and thus their interest on the one hand and / or their operating intention on the other hand can be used, depending on a situation in the motor vehicle and / or the user themselves, to implement an optimal strategy to support the user when viewing and using the display device. This can, for example, always ensure that a driver as user is not distracted from their driving task or is distracted as little as possible. For example, the determined control command, which specifies the arrangement and / or design of the element on the display device, can be used to specify that the element is to be moved to an area that is less distracting for the user and / or that an area of ​​the element is to be expanded. Furthermore, the element can be moved into a reachable area of ​​the user if necessary.However, depending on the situational information and / or user information, it may also be appropriate to select the arrangement and / or design of the element according to the control command such that it is no longer displayed to the user, for example, to avoid distracting them from their current driving task. This ultimately results in minimal distraction, efficient operation, and an intelligent and empathetic display device. Thus, a display device for a motor vehicle is provided that hardly distracts a driver from their driving task.

[0020] The invention further includes embodiments which provide additional advantages.

[0021] One embodiment provides that the situation information describes at least one of the following situations: a driving situation of the motor vehicle, a degree of automation of at least one vehicle function activated in the motor vehicle and / or a driving task of a driver of the motor vehicle.

[0022] The driving situation of the motor vehicle can depend in particular on the environment through which the motor vehicle is traveling. Alternatively or additionally, the driving situation can depend on the traffic situation in the environment, such as the current traffic density. Alternatively or additionally, the driving situation can depend on the weather in the environment. For example, a distinction can be made between rain, snow, heat, cold, fog and / or ice. For example, a driving situation can be recognized that the motor vehicle is driving in a city center in the rain and / or there is heavy traffic, which therefore requires greater attention from the driver compared to other driving situations, for example in sunshine and with less heavy traffic.In this example, taking the situational information into account, the control command can be selected such that the element is moved quickly, for example within 4 seconds instead of the previous 6 seconds, from a position to the side of the steering wheel to a central position behind the steering wheel, which is in the driver's line of sight. Furthermore, switching off the content to protect the driver can be carried out more quickly, for example after just 2 seconds of looking away from the road instead of a default setting where the element is no longer displayed after 3 seconds of looking away from the road. The default setting can be specified, for example, when driving on a highway with low traffic density compared to a city center.

[0023] The degree of automation is, for example, the autonomy level of a driver assistance system and / or another vehicle function activated in the motor vehicle. During automated or at least partially automated driving, for example, greater distraction of the driver as a user may be permitted than during manual driving. The displayed element can then, for example, be assigned to a convenience function, whereas during manual driving it is only assigned to driving-relevant information and / or functions. The higher the degree of automation, the more the element can distract the driver, especially from their driving task.

[0024] The driving task specifies an attention requirement for the driver. Alternatively or additionally, the relevance of the element to the driving task can be taken into account. In other words, the relevance of the element to the driving task as well as a required cognitive workload, such as the driving task, can be taken into account. It can be provided that, for example, only those elements relating to content relevant to the driving task are displayed in an area of ​​the display device visible to the driver. Other applications, such as the vehicle's comfort functions, games and / or videos, should only be displayed when the motor vehicle is parked or driving with a high degree of automation, such as a fully automated driving mode. This is justified because otherwise the driver's driving task would be disrupted by displaying or making the element available.Therefore, depending on the degree of automation and / or the driving situation, the attention required for the driving task may be too demanding to display certain content at specific positions and / or in a specific size. Driving-related information, such as information on the activated driver assistance system, a driving route, and / or a warning, is preferably displayed within the user's field of vision, for example, behind the steering wheel and thus within the driver's field of vision. In contrast, information that is not urgently relevant to the driving situation, such as information on comfort functions (air conditioning, multimedia system, video output, games), is only displayed when the attention requirement is correspondingly low. Ultimately, the current situation of the vehicle can be determined particularly reliably, and this can be taken into account when generating the control command.

[0025] A further embodiment provides that the user information describes at least one of the following user characteristics of the user: a cognitive state, an emotional state, physical mobility, and / or body size. The cognitive state can alternatively be described as the user's cognitive position. It can depend, for example, on how quickly the user reacts to a change in the element with their gaze and / or an operating action. For example, a searching gaze and / or a finger hovering immediately in front of the display device before an operation is carried out can be detected and evaluated to assess the cognitive state. If, for example, it is determined that the user reacts only comparatively slowly, the complexity and thus the amount of content of the element can be reduced to take this cognitive position into account.

[0026] The emotional state can alternatively be described as the user's emotional state. Depending, for example, on the degree of automation and / or the driving situation, as well as whether or not a passenger is present, the element on the display device can lead to confusion or irritation. Such emotions of the user can be captured and recognized, for example, using the driver observation camera, i.e., the interior camera. Alternatively or additionally, at least one microphone in the motor vehicle can be used to capture and evaluate a user's utterances in order to assess their emotional state. However, a time-of-flight camera, a 3D camera, and / or the interior camera are preferably used for gesture analysis, and their data is evaluated and taken into account. The user's emotional state, for example, distinguishes at least between happy, angry, excited, bored, tired, restless, and / or asleep.Depending on, for example, how excited the user is at the moment, reducing the complexity of the element and / or placing it in the user's direct field of vision can help to improve the user's emotional state or to choose the arrangement and / or design of at least one element on the display device in accordance with the user's emotional state.

[0027] The physical mobility of the user can, for example, describe how quickly and / or how far the user can stretch in order to move a hand to the element, for example. For example, it is analyzed which parts of the display device are touched little or not at all by the user, by evaluating data from one of the aforementioned cameras in the interior. If the user has comparatively low mobility, for example, the element can always be arranged behind the steering wheel in the case of the driver as the user, and this can be predetermined by the determined control command. If, however, the user's mobility is comparatively high, the element can be arranged to the side of the user's field of vision.

[0028] Body height can alternatively be referred to as user height. Body height can be determined, in particular estimated, for example, based on seat adjustment and / or steering wheel adjustment and / or by evaluating data from the interior camera and / or the driver observation camera. The area of ​​the display device in which, for example, the elements for which an operating intention was detected are arranged can then be delimited. Body height can be estimated, for example, based on the height of the user's eyes viewed in a vertical direction of the motor vehicle. Alternatively or additionally, an arm length of the user's arm can be determined and an arm reach quantified from this. Various pieces of information are therefore available regarding the user's current state, and the determined state can be taken into account when determining the control command.

[0029] According to an additional embodiment, the element information is detected based on at least one of the following signs: a hand approaching and / or finger approaching the at least one element, a voice command relating to the at least one element, focusing on the at least one element with a gaze and / or an operating gesture. Various options are thus provided for determining that the user is interested in operating the element and / or in information displayed by the element. Upon detecting the hand approaching and / or finger approaching, a proximity sensor in the area of ​​the display device can, for example, detect corresponding data, which is then evaluated. For example, a resistive and / or capacitive sensor can be positioned inside and / or outside the display device.The approach of the hand and / or finger is particularly indicative of the intention to operate the device. This is particularly useful when using a touchscreen as a display device.

[0030] For the voice command, the user's spoken language is recorded and evaluated using at least one microphone. For example, the user can use the voice command to access a specific function of the vehicle, for which more precise commands and / or information are then provided via the element. After the voice command has been recorded, this element should be displayed as close to the user as possible, thus within their reach.

[0031] If the gaze is focused on the element, for example, a turn of the user's head and / or neck can be detected and / or the user's pupil can be observed using the interior camera. Depending on how tightly the pupil is closed, it can be inferred, for example, that an object is being viewed in the distance or nearby. If the gaze is directed at a nearby object and the object is the element, the intention to operate and / or interest in the element can be detected. The camera used for this can be a mono or stereo camera. If, in addition, focusing on the element lasts a predetermined minimum time, for example 1 second, interest in the element can be inferred particularly reliably.

[0032] Furthermore, an operating gesture can be recognized, for example, one that points to a specific element and / or is linked to it. Thus, an operating intention can be detected, for example, by capacitive sensors in the display device and / or time-of-flight cameras and / or 3D cameras and / or a radar device in the interior. Furthermore, the seat occupancy sensor and / or the interior camera and / or capacitive steering wheel sensors can be used to determine the element information. Finally, comprehensive recording of the user, who is in particular the driver, can be performed in order to reliably and accurately determine the element information.

[0033] One embodiment particularly preferably provides that when the element information is detected, it is checked whether a passenger is present in the motor vehicle. In particular, if this is the case, i.e. if a passenger is sitting in the motor vehicle, when determining the control command it is taken into account whether the at least one sign is detected for the passenger or for the driver. For example, by means of the interior sensor device, i.e. for example the interior camera, and / or the seat occupancy sensor and / or a belt buckle sensor of a seat belt buckle of a driver's seat and a passenger seat, it can be determined whether only the driver or also a passenger is located within the reach of the display device. If both the driver and the passenger are located within the reach of the display device, this can lead to potential conflicts between the operating intentions and / or the interest in the element between the driver and the passenger.The focus is preferably on the driver, who should always be shown the information relevant to him in the form of the element or made available for operation. However, if, for example, the approach of a passenger's hand is detected, there is no need to move the element closer to the driver and / or display it larger to the driver, as the driver is not at all interested in operating this element. In such a case, for example, no control command is determined that makes the element easier for the driver to reach, but rather a control command is determined that moves the element closer to the passenger and / or displays it larger to the passenger. It is therefore sensible to first identify potential conflicts between operation of and / or interest in the element by the driver and the passenger and to take these into account accordingly.If necessary, the presence of the passenger and / or the interest detected for the passenger and / or their operating intention can be described by passenger information. When determining the control command, the control command determination criterion can be applied to the passenger information, for example.

[0034] An additional embodiment provides that the control command is only determined when the element information is detected for a period of time greater than a predefined time limit and / or with a repetition frequency greater than a repetition frequency limit. A minimum time can therefore be specified for which one of the above-mentioned signs, based on which the element information is determined, must be detected before the corresponding element information is accepted as such. Alternatively or additionally, it can be specified that the corresponding sign must be detected repeatedly, i.e., at least twice, in particular within a predefined time window. Only when the user has, for example, repeatedly looked at the element is this considered to be interest in the element. This reduces possible incorrect activations of the method due to inaccurate element information.

[0035] In a further embodiment, it is provided that the control command comprises at least one of the following specifications: moving the at least one element from its current position on the display device to a different target position, wherein the target position is arranged in particular centrally behind the steering wheel; and / or enlarging a representation of the at least one element at the current position and / or at the target position; and / or adding at least one sub-element to the at least one element, for example by popping up or displaying an additional selection list; and / or removing the at least one element from the display device. Thus, various ways of influencing the arrangement and design, and thus the representation of the element, are possible. These are always advantageous because they help ensure that the element is displayed close and clearly visible to the user.

[0036] When moving an element, for example, this can be done slowly at first and then gradually faster, so that the element is moved on the display device in the direction of the user's field of vision, for example, towards the steering wheel, within a few seconds, for example, 1 to 3 seconds. The driving situation and / or the degree of automation can be taken into account to select the target position of the element. In general, the driver can interrupt the movement at any time, for example, by no longer looking at the element and instead looking back at the road, which automatically stops and / or reverses the movement.

[0037] If, for example, the driver repeatedly focuses on the same element within a period of one minute, its movement will be continued or forced (started). However, the user can select a different element at any time if necessary, for example using controls on the steering wheel, which will then be positioned within their field of vision, in particular behind the steering wheel. Taking the driving situation into account and in combination with at least one of the aforementioned indicators used to determine the element information, the element can, for example, be moved into the user's reach and dynamically transformed from a pure display element into a control element, in particular within a few milliseconds. For example, the control element can be larger than the display element to make operation easier.

[0038] Depending on, for example, the current driving task and / or the function of the element, it can be optimized for easy haptic operation. In this context, list elements can be displayed larger, for example, to achieve better readability and / or accuracy in the case of touch-sensitive displays. Should the user nevertheless take their eyes off the road for too long or too often within a time window, the element can first be moved towards the steering wheel and then an escalation strategy can be implemented through warnings and finally a blocking and thus at least temporary disappearance of the element to signal that, for example, given the current driving situation and / or driving task and thus the current attention requirement, this element can no longer be displayed.

[0039] It can also be detected whether the user, depending on the driving situation and / or the degree of automation, operates the element for too long or too often within a short period of time, for example several times per minute, each time for more than two seconds. In this case, the element's design, in particular its size, can be changed and / or moved and / or removed, for example using a similar escalation strategy as explained above. The user's intention to operate can be detected unconsciously, for example by intuitively stretching their arm towards the display device, or intentionally, for example by pointing at the element with their index finger. It is particularly advantageous for the user to have to stretch their arms or shoulders towards the display device if they want to operate and / or view other elements or elements located further away.

[0040] An additional embodiment provides that when determining the control command, it is taken into account which part of the display device lies in a focus area, a field of vision and / or a grasping range of the user. The focus area here refers to an area in which the user can see clearly. This is ±5°, starting from the user's eyes. The field of vision is an area in which information is at least partially perceived. The field of vision is arranged, for example, in a range of ±15° around the user's eyes. Outside the field of vision lies a peripheral vision area. Elements displayed there are, for example, not perceived by the user. In order to actually read information in the field of vision, it is necessary to turn and / or move the head, so that it is assumed that sharp vision is only actually achieved in the focus area.Within the field of vision, however, information is relatively easy to see without constantly looking away from the road, for example, with only a slight turn of the head while still keeping the road in view. The field of vision is therefore suitable for displaying information in motor vehicles. Preferably, the element of interest and / or intended to be operated is displayed in the focus area. For the driver, for example, the part of the display device located behind the steering wheel is in the focus area.

[0041] The reach range is limited by a reach limit. This can depend, for example, on the user's physical mobility. For example, the reach range extends as far as the user can reach without having to lean out of their seat and / or bend over. If the user's intention to operate the device is determined, the corresponding element should preferably be positioned within the reach range.

[0042] By taking these areas into account when determining the control command, it can be ensured that at least the target position lies within the focus area, field of view, or grasping area. Preferably, it lies within both the focus area and the grasping area, although these two areas typically overlap. Furthermore, it can be specified that only target positions within the focus area and not within the field of view are desired. The control command therefore always aims to move or better display the element within the focus area, and if this is not possible, at least within the field of view. This makes the element easier to see, potentially making it easier to operate, and reduces distractions.

[0043] A further embodiment provides that the control command determination criterion is determined using machine learning methods. Thus, the described method is preferably supported by artificial intelligence. For example, an artificial neural network can be trained that can determine the control command depending on the information described above, i.e., the element information, the situation information, and / or the user information. It can further be provided that user data, behavior data, and / or context data are collected in order to improve the control command determination criterion and, for example, to optimize it specifically for the respective user based on their preferences and / or habits. The control command determination criterion can therefore be at least partially trained in a user-specific manner.For example, if a certain user with a 90 percent probability wants to see a navigation map behind the steering wheel in the driver's axis (driver's focus area) after focusing on a corresponding element to the side of the steering wheel for longer than 0.82 seconds, the system can understand this in such a way that, for example, it takes 2 seconds to move behind the steering wheel instead of the average 3.5 seconds. Preferably, for example, a general pre-training of the control command determination criterion is provided, which is then improved and personalized through use in the vehicle and subsequent relearning. This allows the method to be adapted to a specific user particularly efficiently and empathetically. This increases user satisfaction.

[0044] For use cases or application situations that may arise during the method and which are not explicitly described here, it may be provided that, in accordance with the method, an error message and / or a request to enter user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0045] A further aspect of the invention relates to a motor vehicle with a display device. The display device extends at least partially behind the steering wheel, viewed in a longitudinal direction of the motor vehicle, and at least partially laterally to the steering wheel, viewed in a transverse direction of the motor vehicle. It is assumed that the display device can be viewed at least partially through the steering wheel, for example, through a hole in the steering wheel. The motor vehicle is designed to carry out the method described above.

[0046] The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0047] The invention also includes the control device for the motor vehicle. The control device can have a data processing device or a processor device that is configured to carry out an embodiment of the method according to the invention. For this purpose, the processor device can have at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). In particular, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), or an NPU (Neural Processing Unit) can be used as the microprocessor. Furthermore, the processor device can have program code that is configured to carry out the embodiment of the method according to the invention when executed by the processor device. The program code can be stored in a data memory of the processor device.The processor device can be based, for example, on at least one circuit board and / or on at least one SoC (System on Chip).

[0048] The invention also includes further developments of the motor vehicle according to the invention that have features already described in connection with the further developments of the method according to the invention. For this reason, the corresponding further developments are not described again here.

[0049] As a further solution, the invention comprises a computer-readable storage medium comprising program code which, when executed by a computer or computer network, causes the computer to carry out an embodiment of the method according to the invention. The storage medium can be provided at least partially as a non-volatile data memory (e.g. as a flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data memory (e.g. as a RAM - random access memory). The storage medium can be arranged in the computer or computer network. However, the storage medium can also be operated on the Internet, for example, as a so-called app store server and / or cloud server. The computer or computer network can provide a processor circuit with, for example, at least one microprocessor.The program code may be provided as binary code and / or as assembly code and / or as source code of a programming language (e.g. C) and / or as a program script (e.g. Python).

[0050] The invention encompasses combinations of the features of the described embodiments. The invention thus encompasses implementations that each comprise a combination of the features of several of the described embodiments, unless the embodiments are described as mutually exclusive.

[0051] Exemplary embodiments of the invention are described below. Shown are: Fig. 1 is a schematic representation of an interior of a motor vehicle with a display device; and Fig. 2 shows a schematic representation of a signal flow graph of a method for operating a display device in a motor vehicle.

[0052] The exemplary embodiments explained below are preferred embodiments of the invention. In the exemplary embodiments, the described components of the embodiments each represent individual features of the invention that can be considered independently of one another, each of which also develops the invention independently of one another. Therefore, the disclosure is intended to encompass combinations of the features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0053] In the figures, the same reference symbols designate elements with the same function.

[0054] Fig. 1 shows the interior of a motor vehicle 1. A display device 2 is arranged in the interior. The display device 2 has a plurality of elements 3, 4, 5, 6, 7. These are displayed at various positions on the display device 2. The respective element 3, 4, 5, 6, 7 can be an operating element and / or a pure display element. The display device 2 is preferably designed as a touch-sensitive screen. It can extend completely between two A-pillars 17 of the motor vehicle 1, as sketched here as an example.

[0055] The display device 2 extends, as viewed in a longitudinal direction of the motor vehicle 1, at least partially behind a steering wheel 10 of the motor vehicle 1. Element 4 is arranged there. At least one driver 8 of the motor vehicle 1 can see element 4. Element 4 contains, for example, driving information, such as information on a speed and / or a next required steering action. Furthermore, the display device 2 extends, as viewed in a transverse direction (y-direction) of the motor vehicle 1, at least partially laterally to the steering wheel 10. It extends, for example, over at least an area of ​​a center console of the motor vehicle 1. Here, elements 3, 5, 6, 7 are located to the side of the steering wheel 10. Element 3 here is a map and thus a navigation instruction. Element 5 is provided, for example, for setting a multimedia system, such as a radio station.Element 6, for example, relates to settings for motor vehicle 1, such as an activated driver assistance system. Element 7 here is multimedia content, such as a game and / or a film.

[0056] The motor vehicle 1 can have an interior sensor device 11, such as a driver observation camera and / or an interior camera. Furthermore, the motor vehicle 1 can have an environmental sensor device 12, which is designed, for example, as a camera for detecting the surroundings of the motor vehicle 1 and / or as another external sensor. Furthermore, a passenger 9 is present in the motor vehicle 1 as a possible user in addition to the driver 8. The motor vehicle 1 here has a control device 13 designed to carry out a method according to the invention.

[0057] A focus area 14 is shown for the driver 8, in which he can see clearly and, for example, read, a field of vision 15 in which he can perceive things at least to a limited extent, and a boundary of a grasping area 16 up to which he can grasp or reach.

[0058] Fig. 2 shows method steps of a method for operating the display device 2. In a method step S1, element information 20 is determined. The element information 20 describes an operating intention and / or an interest in the at least one element 3, 4, 5, 6, 7 provided by the display device 2. The operating intention and / or the interest relates to the user of the motor vehicle 1, such as the driver 8 and / or the passenger 9 here, for example. The element information 20 can be detected based on at least one of the following signs: a hand approach and / or finger approach to the at least one element 3, 4, 5, 6, 7; a voice command relating to the at least one element 3, 4, 5, 6, 7; focusing on the at least one element 3, 4, 5, 6, 7 with a glance; and / or an operating gesture. When determining the element information 20, it can be checked whether the passenger 9 is present in the motor vehicle 1.In particular, if this is the case, it is checked whether at least one sign is detected for passenger 9 or driver 8. This can be taken into account in further procedural steps, i.e., it can be considered whether passenger 9 or driver 8 has shown the intention to operate or interest in element 3, 4, 5, 6, 7.

[0059] A method step S2 comprises determining situation information 21 that describes a current situation of the motor vehicle 1. This is done by applying a situation determination criterion 22 to motor vehicle data describing a journey and / or an environment of the motor vehicle 1. The motor vehicle data can be acquired, for example, using the environment sensor device 12. The situation information 21 can describe at least one of the following situations: a driving situation of the motor vehicle 1; a degree of automation of at least one vehicle function activated on the motor vehicle 1; and / or a driving task of the driver 8 of the motor vehicle 1. The driving situation depends in particular on the environment through which the motor vehicle 1 is traveling, a traffic situation in the environment, and / or the weather in the environment.The driving task of the driver 8 includes in particular an attention requirement for the driving task and / or a relevance of the at least one element 3, 4, 5, 6, 7 for the driving task.

[0060] In a method step S3, user information 24 is determined that describes the user of the motor vehicle 1. The user information 24 is determined by applying a user determination criterion 25 to user data 26 describing the user. The user data 26 is detected and provided, for example, by means of the interior sensor device 11. The user information 24 can describe at least one of the following user characteristics of the user: a cognitive state; an emotional state; physical mobility; and / or a body size.

[0061] In a method step S4, a control command 27 for the display device 2 is determined. This command specifies an arrangement and / or design of the at least one element 3, 4, 5, 6, 7 of the display device 2. The control command 27 is determined by applying a control command determination criterion 28 to the element information 20 on the one hand and the determined situation information 21 and / or the determined user information 24 on the other hand. In a method step S5, the display device 2 is operated according to the determined control command 27.

[0062] The control command 27 can, for example, only be determined when the element information 20 is captured for a period of time greater than a predefined duration limit and / or with a repetition frequency greater than a repetition frequency limit. The duration limit is, for example, 2 seconds or 3 seconds, and the repetition frequency is, for example, at least three glances per minute.

[0063] The control command 27 can comprise at least one of the following specifications: a displacement of the at least one element 3, 4, 5, 6, 7 from its current position on the display device 2 to a different target position, wherein the target position is arranged in particular centrally behind the steering wheel 10. For example, in the case of a frequent or prolonged look at the element 3, this can be displayed centrally behind the steering wheel 10 instead of the previous element 4. For this purpose, it is preferably displaced there, i.e., it moves at a certain speed from the location of the element 3 according to Fig. 1 to the location of element 4 in Fig.1. Alternatively or additionally, a representation of the at least one element 3, 4, 5, 6, 7 can be enlarged at the current position and / or at the target position. Alternatively or additionally, at least one sub-element can be added to the at least one element 3, 4, 5, 6, 7, such as a list of possible options that can be selected. Alternatively or additionally, the at least one element 3, 4, 5, 6, 7 can be removed, for example, if this would only distract the driver 8 and thus should not be displayed. When determining the control command 27, it can also be taken into account which sub-area of ​​the display device 2 lies in the focus area 14, the field of vision 15, or the grasping area 16 of the user, so that at least the target position lies in this sub-area.

[0064] It is preferably provided that the control command determination criterion 28 is determined using machine learning methods. It can be at least partially user-specifically trained.

[0065] Overall, the examples show gaze and gesture detection to reduce distraction by situational adaptation of an element 3, 4, 5, 6, 7 on a display device 2. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2008 004 970 A1

[0004] DE 10 2009 032 069 A1

[0005]

Claims

[1] Method for operating a display device (2) for a motor vehicle (1), wherein the display device (2) extends at least partially behind a steering wheel (10) of the motor vehicle (1) when viewed in a longitudinal direction of the motor vehicle (1) and at least partially laterally to the steering wheel (10) when viewed in a transverse direction of the motor vehicle (1), the method comprising: - determining (S1) an item of element information (20) which describes an operating intention and / or an interest in at least one element (3, 4, 5, 6, 7) provided by means of the display device (2) by a user; - determining (S2) a situation information item (21) describing a current situation of the motor vehicle (1) by applying a situation determination criterion (22) to motor vehicle data describing a journey and / or surroundings of the motor vehicle (1); and / or - determining (S3) user information (24) describing the user of the motor vehicle (1) by applying a user determination criterion (25) to user data (26) describing the user; - determining (S4) a control command (27) for the display device (2), which prescribes an arrangement and / or design of the at least one element (3, 4, 5, 6, 7) on the display device (2), by applying a control command determination criterion (28) to the determined element information (20) on the one hand and the determined situation information (21) and / or the determined user information (24) on the other hand; and - Operating (S5) the display device (2) according to the determined control command (27). [2] Method according to claim 1, wherein the situation information (21) describes at least one of the following situations: - a driving situation of the motor vehicle (1), which depends in particular on the environment through which the motor vehicle (1) is driving, a traffic situation in the environment and / or a weather in the environment; - a degree of automation of at least one vehicle function activated in the motor vehicle (1); and / or - a driving task of a driver (8) of the motor vehicle (1), in particular an attention requirement for the driving task and / or a relevance of the at least one element (3, 4, 5, 6, 7) for the driving task. [3] Method according to one of the preceding claims, wherein the user information (24) describes at least one of the following user properties of the user: - a cognitive state; - an emotional state; - physical mobility; and / or - a body size. [4] Method according to one of the preceding claims, wherein the element information (20) is determined based on at least one of the following indications: - a hand approach and / or finger approach to the at least one element (3, 4, 5, 6, 7); - a voice command relating to at least one element (3, 4, 5, 6, 7); - focusing the at least one element (3, 4, 5, 6, 7) with a glance; and / or - an operating gesture. [5] Method according to claim 4, wherein when determining the element information (20) it is checked whether a passenger (9) is present in the motor vehicle (1), and in particular if this is the case, whether the at least one sign is determined for the passenger (9) or for a driver (8), wherein when determining the control command (27) it is taken into account whether the sign was determined for the passenger (9) or for the driver (8). [6] Method according to one of the preceding claims, wherein the control command (27) is only determined when the element information (20) has been determined for a time period which is greater than a predetermined time period limit value and / or with a repetition frequency which is greater than a repetition frequency limit value. [7] Method according to one of the preceding claims, wherein the control command (27) comprises at least one of the following specifications: - moving the at least one element (3, 4, 5, 6, 7) from its current position on the display device (2) to a different target position, which is arranged in particular centrally behind the steering wheel (10); - Enlarging a representation of the at least one element (3, 4, 5, 6, 7) at the current position and / or at the target position; - adding at least one sub-element (3, 4, 5, 6, 7) to the at least one element (3, 4, 5, 6, 7); and / or - removing the at least one element (3, 4, 5, 6, 7) from the display device (2). [8] Method according to claim 7, wherein when determining the control command (27) it is taken into account which partial area of ​​the display device (2) lies in a focus area (14), a field of vision (15) and / or a gripping area (16) of the user, so that at least the target position lies in this partial area. [9] Method according to one of the preceding claims, wherein the control command determination criterion (28) is determined by means of machine learning methods, wherein it is in particular at least partially trained in a user-specific manner. [10] Motor vehicle (1) with a display device (2) which, viewed in a longitudinal direction of the motor vehicle (1), extends at least partially behind a steering wheel (10) of the motor vehicle (1) and, viewed in a transverse direction of the motor vehicle (1), at least partially laterally to the steering wheel (10), wherein the motor vehicle (1) is designed to carry out a method according to one of the preceding claims.

Images