Method for operating a front seat passenger display device in a motor vehicle, computer program product and data processing device

Abstract

Examples relate to a method (10) for operating a front seat passenger display device (24) in a motor vehicle. The method (10) comprises the steps of: determining (11) a front seat passenger activity (26a, 26b), the front seat passenger activity (26a, 26b) being related to an interaction of a front seat passenger (22) with a driver (21) of the vehicle; and controlling (12) a display on the front seat passenger display device (24) depending on the detected (11) passenger activity (26a, 26b).

Description

[0001] METHOD FOR OPERATING A PASSENGER DISPLAY IN A MOTOR VEHICLE, COMPUTER PROGRAM PRODUCT AND DATA PROCESSING DEVICE

[0002] Technical area

[0003] Embodiments relate to a method for operating a passenger display in a motor vehicle. Furthermore, a computer program product for implementing the proposed method and a data processing device for a motor vehicle are provided. The method can advantageously be used to reduce or completely prevent potential distraction of a motor vehicle driver by content of the passenger display, particularly at an early stage.

[0004] background

[0005] Modern vehicles are increasingly featuring large displays for entertainment systems. A key trend for vehicle interiors and user interfaces is passenger displays, which are already offered by many vehicle manufacturers, especially in the premium segment. These displays are positioned in front of the passenger and are designed to allow the passenger to view information directly in front of them, operate vehicle infotainment functions, or watch videos, for example.

[0006] However, using a passenger display can also cause some distraction for the driver, as the proximity between the driver and passenger may also allow the driver to view the contents of the passenger display. Methods are known to reduce such driver distraction.

[0007] US 2023 226 972 A discloses a shading device for a passenger display in a motor vehicle, which comprises a switchable privacy screen. In addition, the device includes a light source directed toward the display and preventing residual fragments of display content from being visible to the driver despite the active privacy screen. The brightness of the light source is controlled by a control unit depending on the brightness of the display. The device may also include an ambient light sensor, a diaphragm in the light source, an operating module in the vehicle headliner, and a prism film in the display. A method for controlling the shading device is also described. The motor vehicle may include such a shading device.

[0008] DE 10 2012 206 030 A1 discloses a method for controlling at least one function of at least one display device for an interior of a vehicle, wherein the vehicle has at least one vehicle camera. The method comprises a step of determining whether a viewing direction of at least one vehicle occupant, detected by means of the at least one vehicle camera, is directed toward a device region of the interior of the vehicle in which the at least one display device is installed. The method also comprises a step of controlling the at least one display device in a first display mode if the detected viewing direction of the at least one vehicle occupant is directed toward the device region, in order to control the at least one function of the at least one display device.

[0009] DE 10 2013 013 227 A1 discloses a motor vehicle with a display unit arranged in the passenger compartment, which is designed to display a display content only for such a spatial area that excludes a driver sitting on a driver's seat of the motor vehicle with a predetermined standard body posture, and with a monitoring device which is designed to monitor a current body posture of the driver and to interrupt the display of the display content in the event that the driver attempts to view the display unit from the spatial area.The monitoring device comprises a 3D camera unit and a calculation unit, wherein at least one optical sensor unit of the 3D camera unit is arranged above the passenger compartment and is designed to generate 3D image data from above at least of the driver's head, and wherein the calculation unit is designed to determine a position of the head and / or a movement of the head based on the 3D bit data. Accordingly, a control unit interrupts the presentation of a display content on the screen whenever the driver attempts to lean over to the passenger seat in order to be able to view the display content. DE 10 2004 033 275 A1 discloses an enabling device for enabling individual predetermined functions of a device for a motor vehicle while the motor vehicle is traveling, in particular for the driver and / or the front passenger. The device can be operated by both the front passenger and the driver via an operating element.To ensure that the driver cannot operate the device via the control element while driving, the invention provides a monitoring device for monitoring and distinguishing whether the driver or the passenger at least intends to operate the device via the control element. Furthermore, a decision device is provided for deciding whether to enable or disable individual predetermined functions of the device for the driver or the passenger, depending on which of these persons was previously identified by the monitoring device as the operator of the control element.

[0010] A disadvantage of other methods according to the known prior art may be that specific structural precautions must be implemented in the vehicle for certain methods, for example, to prevent the driver from viewing the passenger display in a standard position. Another disadvantage may be that the content of the passenger display is only deactivated when the driver could already be distracted from the driving task, for example, by a movement (e.g., leaning over to the passenger display) intended to allow viewing of the content.

[0011] Summary

[0012] It is an object of the present disclosure to provide improved concepts for controlling passenger displays in motor vehicles in order to reduce or avoid distraction of the driver by contents of the passenger display, in particular to enable particularly early control of the passenger display so that distraction of the driver can be reduced or avoided.

[0013] This object is achieved according to the subject matter of the independent patent claims. Further advantageous embodiments are described in the dependent patent claims, the following description, and in conjunction with the figures.

[0014] Accordingly, a method for operating a passenger display in a motor vehicle is proposed. The method comprises determining passenger activity, wherein the passenger activity relates to an interaction between the passenger and the driver of the vehicle, and controlling a display of the passenger display depending on the determined passenger activity.

[0015] The method can be carried out, in particular, while the motor vehicle is moving (e.g., the method cannot be carried out when the motor vehicle is parked). In this case, a driver of the vehicle may have to perform a driving task, and the driver should be prevented from being distracted by the contents of the passenger display. For example, the passenger activity determined according to the method may indicate that the driver, with a predetermined probability, may develop an interest in viewing the display content of the passenger display. In this case, controlling the display of the passenger display may include partially or completely removing the display content (e.g., until the driver's potential interest in the display content has fallen below a safe limit). For example, the display content may be covered with a deactivation symbol (e.g.,Brand emblem of the vehicle manufacturer) to indicate that the interruption of the display content is not due to a technical defect in the passenger display, but rather a safety feature. For example, information can be issued that there is a risk of the driver being distracted by the current passenger activity and that the passenger should therefore stop. Accordingly, the procedure can provide for the content of the passenger display to be displayed again (e.g. to reactivate the passenger display) when the passenger activity indicates that the driver is no longer potentially distracted by the content of the passenger display. Since the passenger activity involves an interaction between the passenger and the driver, it can also be referred to as passenger-driver activity.

[0016] According to one aspect of the disclosure, the brightness of the display is reduced if the determination of the passenger activity shows that the passenger is not currently using the passenger display. In this case, the passenger activity may indicate that the passenger is talking to the driver about something completely different from the contents of the passenger display, so that they are not paying any attention to this content. For example, the brightness may be reduced to 0%, which ultimately turns off the passenger display entirely. This may enable more energy-efficient operation of the passenger display. For example, the brightness may also be reduced to a low residual value (e.g., less than 30% (or less than 20%) and more than 10% of full brightness) so that the driver can no longer see the content from the viewing angle of the driver's seat.

[0017] According to one aspect of the disclosure, determining the passenger activity includes checking whether the passenger informs the driver of the motor vehicle about the contents of the passenger display (e.g., actively informing; e.g., specific content information; e.g., requesting the driver to pay attention to the contents of the passenger display, to look at the passenger display, and / or to respond to a description from the passenger regarding the contents of the passenger display). The interaction between driver and passenger can, for example, be directed at the displayed content of the passenger display. For example, the passenger can show the driver something on the passenger display or tell the driver about the content. For determining the passenger activity, vehicle-internal sensors can be used, e.g., acoustic or optical sensors.

[0018] For example, blocking of the contents of the passenger display may be provided if the determination of passenger activity shows that the passenger actively informs the driver about the display contents and / or requests active feedback on this (e.g. in the case of specific content information). However, if the passenger only provides general information to the driver regarding the use of the passenger display (e.g. general usage information; e.g. brief information that the passenger will now use the passenger display, but no specific information about the content shown on the passenger display), blocking of the contents of the passenger display may be omitted, since in this case the risk that the driver could be distracted by the contents of the passenger display can be considered sufficiently insignificant.

[0019] According to one aspect of the disclosure, for checking whether the passenger is informing the driver of the motor vehicle regarding the contents of the passenger display, the check is carried out based on a semantic evaluation of a speech signal from the passenger. For example, the passenger can report something to the driver about the contents of the passenger display or tell the driver something corresponding to it. The passenger's statements can be recorded by a microphone and evaluated by a computing unit (e.g., an on-board computer or a computing unit coupled to the vehicle, such as a server or mobile device). If it is determined that the passenger activity corresponds to the display contents, the passenger display can be controlled accordingly (e.g., immediately deactivated or its brightness greatly reduced according to previously described examples).According to one aspect of the disclosure, the checking is performed based on an optical evaluation of an image signal from the passenger. For example, the passenger can point to the passenger display (and optionally look at the driver while doing so), which can be captured by a camera. The image signal can be evaluated by a computing unit (e.g., an on-board computer or a computing unit coupled to the vehicle, such as a server or mobile device). If it is determined that the passenger activity is related to the passenger display, the passenger display can be controlled accordingly (e.g., immediately deactivated or its brightness greatly reduced according to previously described examples).

[0020] According to one aspect of the disclosure, deactivation of the passenger display occurs immediately as soon as the detected passenger activity indicates that the passenger is informing the driver of the motor vehicle about the contents of the passenger display. As a particular advantage of the present disclosure, the display can thus be switched off or dimmed even before the driver leans over to view the display or even before turning their head or gaze toward the passenger display.

[0021] For example, controlling the display of the passenger display may include reducing the brightness to such an extent (e.g., by more than 60% or more than 80% and / or less than 90%) that the display content is only visible from a position directly in front of the passenger display (e.g., from the passenger seat; e.g., at a viewing angle of 90° + / - 10°) and not from any other viewing angle (e.g., not from the driver's seat).

[0022] One advantage of the disclosure may be that the passenger display is only deactivated or dimmed when the driver would potentially be distracted by looking at the passenger display. For example, in contrast to other methods, the method can enable the passenger to view the display content unhindered, even if the driver, for example, accidentally glances briefly in the direction of the passenger display (e.g., when moving their head towards the right-hand side of the vehicle). In this case, it may be that the driver is not at all interested in the display content and is not distracted by it during the brief viewing. The method makes it possible to better assess whether or not the driver would be distracted by the display content of the passenger display when viewing the passenger display. According to one aspect of the disclosure, it is provided that an internal display of the motor vehicle is operated as the passenger display.For example, the passenger display can be designed separately from or in conjunction with a display of a central display of the vehicle.

[0023] According to one aspect of the disclosure, it is alternatively provided that a display of a mobile device carried by the passenger in the motor vehicle is operated as the passenger display. It is common for passengers, for example, to view media on mobile devices they have brought into the vehicle themselves, such as cell phones, tablets, or notebooks, which can also distract the driver. For example, the mobile device can be a device coupled to the vehicle. To control the display of the mobile device, the method can be executed on the mobile device itself. For this purpose, the method can optionally comprise: detecting that the mobile device is currently being used in a moving motor vehicle (this is possible, for example, by means of GPS signal analysis or camera analysis). Alternatively, it can be provided that the vehicle sends a signal to the mobile device if it has been determined according to the method that the passenger activity could distract the driver.Accordingly, the method may comprise: sending a display control signal from the vehicle (e.g. deactivation signal upon detection of a risk of driver distraction by the passenger display due to passenger activity) to the passenger's mobile device to control the display of the mobile device and receiving the display control signal at the mobile device.

[0024] A further aspect relates to a computer program product or non-volatile storage medium with a program code for carrying out the method described above or below when the computer program is executed on a processor, a computer, or programmable hardware. Such a program can advantageously be executed on an on-board computer or alternatively a backend (e.g. a server or cloud service coupled to the vehicle) or alternatively on a mobile device of the passenger in order to carry out the proposed method and control the passenger display. By using the computer program product, for example, a control signal can be prospectively sent to a control unit of the passenger display in order to deactivate it, for example, if there is a risk that the driver would view the content of the passenger display.

[0025] A further aspect relates to a data processing device for a motor vehicle, wherein the data processing device is configured to carry out a method described above or below. A further aspect relates to a motor vehicle comprising such a data processing device or configured to carry out a method described above or below, in particular by using vehicle-internal sensors and computing devices.

[0026] Short character description

[0027] Exemplary embodiments are explained in more detail below with reference to the accompanying figures. They show:

[0028] Fig. 1 is a flowchart of a method for operating a passenger display; and

[0029] Fig. 2a, 2b a schematic representation of an example in which a passenger display of a vehicle is deactivated early due to an interaction of the passenger with the driver in order to reduce potential distraction of the driver.

[0030] Description

[0031] Various embodiments will now be described in more detail with reference to the accompanying drawings, in which some embodiments are illustrated. In the figures, the thickness dimensions of lines, layers, and / or regions may be exaggerated for clarity. In the following description of the accompanying figures, which only show some exemplary embodiments, like reference numerals may designate like or comparable components.

[0032] An element described as being "connected" or "coupled" to another element may be directly connected or coupled to the other element, or may have intervening elements present. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as one of ordinary skill in the art to which the embodiments belong.

[0033] Figure 1 shows a flowchart of a method 10 for operating a passenger display in a motor vehicle. The method 10 includes determining 11 a passenger activity, wherein the passenger activity relates to an interaction between the passenger and the driver of the vehicle. The method 10 further includes controlling 12 a display of the passenger display depending on the determined passenger activity.

[0034] For example, it is proposed to link the displays or content displayed on the passenger display to whether it is being used or not. According to other concepts, solutions are already known in which a passenger display is switched off when the driver looks at it, which can be detected by an interior camera. However, the present disclosure proposes, for example, detecting the passenger's gaze relative to this display (e.g. passenger activity) using an interior camera and deactivating it or at least darkening it when not in use. In particular, it is provided that the display can be prospectively blocked depending on the behavior of the passenger (e.g. passenger activity) if it is assumed that this behavior distracts the driver. This occurs, for example, based on use of the interior camera (e.g. optical analysis; e.g. the passenger pointing at the screen when the content is active and, if applicable,glance towards the driver) and / or by analyzing the interaction (e.g. between front passenger and driver) using language in the interior (e.g. semantic analysis). If, for example, the front passenger actively requests the driver to look at the front passenger display (e.g. by saying, "Hey, look at this"), then this display is locked immediately, even without the driver having glanced towards it being detected (e.g. before the driver has glanced there). This means that the display content can be locked even before the driver makes a distracting movement in the direction of his gaze towards the front passenger display.

[0035] The proposed method can make it possible to deactivate the passenger display even before driver distraction is detected. Rather, deactivation can occur prospectively when a risk of driver distraction is detected due to current passenger activity. Furthermore, the passenger display can be deactivated generally if non-use of the passenger display is detected (e.g., if the passenger activity includes communication with the driver that is not directed at the passenger display content). Deactivating or dimming the passenger display can, for example, achieve energy-efficient operation of the passenger display.

[0036] Fig. 2a and 2b schematically show an exemplary process in which a passenger display is deactivated due to detected passenger activity in order to prevent a possible distraction of a driver of a vehicle by viewing the contents of the passenger display. Fig. 2a schematically shows a vehicle interior in a first situation 20a. A driver

[0037] 21 sits in a driver's seat in front of a steering wheel 23 and performs a driving activity. A passenger 22 sits in a passenger seat and views a passenger display 24, on which a display content ABC is shown. A first unit 25a and a second unit 25b for detecting passenger activity are provided in the vehicle, e.g., an optical sensor 25a (e.g., an interior camera) and an acoustic sensor 25b (e.g., an interior microphone).

[0038] For example, the method can be used to determine if the front passenger 22 falls asleep and is no longer looking at the front passenger display 24. In this case, the front passenger display 24 can be deactivated, for example, so that the display content ABC is no longer shown. This can prevent the driver from being accidentally distracted by the content of the front passenger display 24, for example, if the driver were to accidentally glance at the front passenger display 24.

[0039] Fig. 2b shows schematically the vehicle interior in a second situation 20b. The passenger

[0040] For example, 22 is excited about the display content ABC and wants to talk to the driver 21 about it. For example, the passenger 22 may want to draw the driver 21's attention to the display content ABC with a schematically depicted finger point 26a at the passenger display 24 and / or a statement 26b (e.g., "hey, look at this").

[0041] According to the method, in this situation 20b, using the two units 25a, 25b, an immediate determination 11 is made that the current passenger activity 26a, 26b is aimed at actively informing the driver 21 about the display content ABC of the passenger display 24. For example, the camera 25a can be used to determine 11 the finger pointing 26a and the microphone 25b can be used to determine 11 the verbal statement 26b directed at the driver 21 and then evaluate it. The passenger display 24 can then be immediately deactivated so that the content ABC is no longer visible. Accordingly, the driver 21 no longer has any reason to look in the direction of the passenger display 24, since no content is displayed there anymore. A potential distraction of the driver 21 from his driving activity can thus be avoided particularly early on. To determine 11 the passenger activity, (e.g.When semantically evaluating the statement 26b), artificial intelligence can be used, which can correlate the statement content of the statement of the passenger 22 with the display content ABC.

[0042] Further details and aspects are mentioned in connection with the embodiments described above or below. The embodiment shown in Fig. 2a and 2b may have one or more optional additional features corresponding to one or more aspects mentioned in connection with the proposed concept or with one or more embodiments described above (e.g. Fig. 1) or below. Examples relate to a method for controlling the passenger display (e.g. with the aid of an interior camera) such that it can be deactivated or at least darkened when not in use and / or can be prospectively blocked depending on the behavior of the passenger with regard to interaction with the driver, e.g. in order not to distract the driver.

Claims

Patent claims 1. A method (10) for operating a passenger display (24) in a motor vehicle, the method (10) comprising: - determining (11) a passenger activity (26a, 26b), wherein the passenger activity (26a, 26b) relates to an interaction of a passenger (22) with a driver (21) of the vehicle; and Controlling (12) a display of the passenger display (24) depending on the determined (11) passenger activity (26a, 26b).

2. Method (10) according to claim 1, wherein a brightness of the passenger display (24) is reduced if the determination (11) of the passenger activity (26a, 26b) shows that the passenger (22) is not currently using the passenger display (24).

3. Method (10) according to claim 1 or 2, wherein determining (11) the passenger activity (26a, 26b) comprises checking whether the passenger (22) informs the driver (21) of the motor vehicle regarding contents (ABC) of the passenger display (24).

4. The method (10) according to claim 3, wherein the checking is carried out based on a semantic evaluation of a speech signal of the passenger (22).

5. The method (10) according to claim 3 or 4, wherein the checking is carried out based on an optical evaluation of an image signal of the passenger (22).

6. Method (10) according to one of claims 3 to 5, wherein deactivation of the passenger display (24) takes place immediately as soon as the determined passenger activity (26a, 26b) shows that the passenger (22) informs the driver (21) of the motor vehicle regarding the contents (ABC) of the passenger display (24).

7. Method (10) according to one of claims 1 to 6, wherein an internal display of the motor vehicle is operated as the passenger display (24) 8. The method (10) according to one of claims 1 to 6, wherein a display of a mobile device of the passenger carried in the motor vehicle is operated as the passenger display (24).

9. A computer program product or non-volatile storage medium comprising a program code for carrying out the method (10) according to any one of the preceding claims when the computer program is executed on a processor, a computer, or programmable hardware.

10. Data processing device for a motor vehicle, wherein the data processing device is configured to carry out the method (10) according to one of claims 1 to 8.

Images