Method for operating a co-driver display in a motor vehicle, computer program product and data processing device

Abstract

Each example relates to a method (10) for operating a passenger display (24) in a motor vehicle. The method (10) includes the steps of: determining (11) passenger activities (26a, 26b), wherein the passenger activities (26a, 26b) involve interaction between a passenger (22) and the driver (21) of the vehicle; and controlling (12) the display of the passenger display (24) based on the determined (11) passenger activities (26a, 26b).

Description

Technical Field

[0001] Various embodiments relate to a method for operating a passenger-side display in a motor vehicle. Furthermore, a computer program product for performing the proposed method and a data processing device for a motor vehicle are provided. The method can be advantageously used to reduce or completely avoid potential distractions to the driver of the motor vehicle from the content of the passenger-side display, particularly at an early stage. Background Technology

[0002] In modern motor vehicles, large displays are increasingly being provided for entertainment systems. A significant trend in vehicle interior design and user interfaces is the development of passenger-side displays, which are already supplied by many vehicle manufacturers, particularly in the high-end segment. These displays are positioned spatially in front of the passenger and are intended to allow the passenger to directly access information, operate vehicle infotainment functions, or, for example, watch videos.

[0003] However, using a passenger-side display can also cause some distraction for the driver, because the close proximity between the driver and passenger makes it possible for the driver to view the display's contents. Methods for reducing such driver distraction are known.

[0004] US 2023 226 972 A discloses a shading device for a passenger-side display in a motor vehicle, the shading device including an on / off view protection device. Additionally, the device includes a light-emitting element that points towards the display and blocks the remainder of the displayed content, which remains visible to the driver despite the view protection device being activated. The brightness of the light-emitting element is controlled by a control unit based on the brightness of the display. The device may also include an ambient light sensor, a light shield in the light-emitting element, an operating module in the vehicle headliner, and a prism film in the display. A method for controlling the shading device is also described. The motor vehicle may include such a shading device.

[0005] DE 10 2012 206 030 A1 discloses a method for controlling at least one function of at least one display device for the interior space of a vehicle, wherein the vehicle has at least one vehicle camera. The method includes the step of determining whether a line of sight identified by the at least one vehicle camera from at least one vehicle occupant points to a device area within the interior space of the vehicle, in which the at least one display device is mounted. The method further includes the step of operating the at least one display device in a first display mode when the identified line of sight from the at least one vehicle occupant points to the device area, in order to control at least one function of the at least one display device.

[0006] DE 10 2013 013 227 A1 discloses a motor vehicle having a display unit arranged in a passenger space, the display unit being designed to display content only in a spatial area excluding the driver, who is seated in the driver's seat in a pre-defined standard body posture. The motor vehicle has a monitoring device for monitoring the driver's current body posture and is designed to interrupt the display of content when the driver attempts to view the display unit from the spatial area. The monitoring device includes a three-dimensional camera unit and a computing unit, wherein at least one optical sensor unit of the three-dimensional camera unit is arranged above the passenger space and designed to generate three-dimensional image data of at least the driver's head from above, and wherein the computing unit is designed to determine the position and / or movement of the head based on the three-dimensional bit data. Accordingly, whenever the driver attempts to lean towards the passenger seat to see the display content, the control unit always interrupts the display of content on the screen.

[0007] DE 10 2004 033 275 A1 discloses an activation device for activating various predetermined functions of a motor vehicle during driving, particularly for the driver and / or co-driver. The device can be operated by both the co-driver and the driver via an operating element. To ensure that the driver cannot operate the device via the operating element during driving, the invention includes a monitoring device for monitoring and distinguishing whether the driver or the co-driver at least intends to operate the device via the operating element. Furthermore, a decision-making device is provided to decide whether to activate or deactivate the various predetermined functions of the device for the driver or the co-driver based on which of the persons has been pre-identified by the monitoring device as the operator of the operating element.

[0008] Other methods based on known prior art may be disadvantageous because some methods require specific structural pre-measures in the vehicle to prevent, for example, the driver in a standard position from viewing the passenger-side display. Furthermore, it may be disadvantageous that the content of the passenger-side display is only deactivated when the driver's attention has already been diverted from driving tasks, for example by movements that should enable viewing (e.g., leaning towards the passenger-side display). Summary of the Invention

[0009] The objective of this disclosure is to provide an improved scheme for controlling a passenger-side display in a motor vehicle, so as to reduce or avoid driver distraction by the content of the passenger-side display, and in particular to enable particularly early control of the passenger-side display, thereby reducing or avoiding driver distraction.

[0010] The task described herein is solved by the technical solution of the independent claim. Other advantageous embodiments are described in the dependent claims, in the following description, and in conjunction with the accompanying drawings.

[0011] Accordingly, a method for operating a passenger-side display in a motor vehicle is proposed. The method includes determining passenger-side activities, wherein the passenger-side activities relate to interactions between the passenger-side driver and the vehicle's driver; and controlling the display of the passenger-side display based on the determined passenger-side activities.

[0012] The method can be performed, in particular, while the vehicle is in motion (e.g., not while the vehicle is parked). In this case, the driver may have to complete driving tasks and should avoid being distracted by the content of the passenger-side display. For example, passenger-side activity determined by the method may indicate that the driver may have a predetermined probability of being interested in viewing the content displayed on the passenger-side display. In this case, control of the display on the passenger-side display may include partially or completely removing the display content (e.g., until the driver's potential interest in the display content drops below a safe threshold). For example, the display content may be covered with a stop sign (e.g., the vehicle manufacturer's brand logo) so that the interruption of the display content does not involve a technical malfunction of the passenger-side display but rather a safety feature. For example, information may be output indicating that there is a risk of the driver being distracted by the current passenger-side activity and that the passenger should therefore stop the passenger-side activity. Accordingly, the method may specify that when the passenger-side activity identifies that the driver is no longer expected to be potentially distracted by the content of the passenger-side display, the content of the passenger-side display is redisplayed (e.g., the passenger-side display is reactivated). Since the co-pilot's activities involve interaction between the co-pilot and the driver, these activities can also be referred to as co-pilot-driver activities.

[0013] According to one aspect of this disclosure, when the co-pilot's activity determines that the co-pilot is not currently using the co-pilot display, the brightness of the display is reduced. In this case, the co-pilot's activity may indicate that the co-pilot is discussing something completely different from the content on the co-pilot display, thus the co-pilot is completely unaware of the content on the co-pilot display. For example, the brightness can be reduced to 0%, resulting in the co-pilot display being completely turned off. This allows for more energy-efficient operation of the co-pilot display. For example, the brightness can also be reduced to a low residual value (e.g., less than 30% (or less than 20%) and greater than 10% of full brightness), so that the driver can no longer see the content on the co-pilot display from the driver's seat perspective.

[0014] According to one aspect of this disclosure, determining the co-driver's activity includes checking whether the co-driver notifies the driver of the vehicle about the content of the co-driver's display (e.g., proactive notification; e.g., specific content information; e.g., requesting the driver to pay attention to the content of the co-driver's display, look at the co-driver's display, and / or respond to the co-driver's description of the content of the co-driver's display). Interactions between the driver and co-driver may, for example, refer to the content displayed on the co-driver's display. For example, the co-driver may show the driver something on the co-driver's display or narrate content. To determine co-driver activity, sensors within the vehicle, such as acoustic or optical sensors, may be applied.

[0015] For example, it could be stipulated that the content of the passenger-side display be locked when the determination of the passenger's activity leads to the passenger actively informing the driver about the displayed content and / or providing proactive feedback on this request (e.g., in the case of specific content information). However, locking the content of the passenger-side display should not be delayed, for example, if the passenger merely provides general information about the use of the passenger-side display (e.g., general usage information; e.g., a brief notification that the passenger will now use the passenger-side display, but without specific information about the content displayed on it), because in this case, the risk of the driver being distracted by the content of the passenger-side display can be considered sufficiently negligible.

[0016] According to one aspect of this disclosure, a check is provided regarding whether the co-driver has informed the driver of the vehicle about the content of the co-driver's display. This check is based on semantic analysis of the co-driver's voice signals. For example, the co-driver may report something about the content of the co-driver's display to the driver or tell the driver something corresponding to that. The co-driver's speech can be detected using a microphone and analyzed using a computing unit (e.g., an onboard computer or a vehicle-coupled computing unit, such as a server or mobile device). When it is determined that the co-driver's activity corresponds to the displayed content, the co-driver's display can be controlled accordingly (e.g., immediately deactivated or, as in the previously described example, significantly reduced in brightness).

[0017] According to one aspect of this disclosure, the inspection is performed based on optical analysis of image signals from the co-driver. For example, the co-driver may point at the co-driver display (and optionally look at the driver at the same time), which can be detected by means of a camera. The image signals can be analyzed by means of a computing unit (e.g., an onboard computer or a computing unit coupled to the vehicle, such as a server or mobile device). When it is determined that the co-driver's activity is associated with the co-driver display, the co-driver display can be controlled accordingly (e.g., immediately deactivated or, according to the previously described example, significantly reduced in brightness).

[0018] According to one aspect of this disclosure, once the determined co-driver activity results in the co-driver informing the driver of the content of the co-driver's display, the co-driver's display is immediately deactivated. A particular advantage of this disclosure is that the display can therefore be turned off or dimmed before the driver leans over to view the display or before the driver turns their head or gaze toward the co-driver's display.

[0019] For example, controlling the display of the passenger-side display may include significantly reducing the brightness (e.g., reducing it by more than 60% or more than 80% and / or less than 90%) so that the displayed content is visible only from a position directly in front of the passenger-side display (i.e., from the direction of the passenger seat; for example, at a viewing angle of 90° + / - 10°) and not from another viewing angle (i.e., not from the direction of the driver's seat).

[0020] One advantage of this disclosure is that the passenger-side display is only deactivated or dimmed when the driver's attention is potentially distracted by looking at it. For example, compared to other methods, this method allows the passenger to view the display content unimpeded, even if the driver briefly glances in the direction of the passenger-side display (e.g., while making a head movement towards the right-hand side of the vehicle's surroundings). In this situation, the driver may not be interested in the display content at all and will not be distracted even during brief viewing. This method allows for a better assessment of whether the driver's attention is likely to be distracted by the content displayed on the passenger-side display while viewing it.

[0021] According to one aspect of this disclosure, the interior display of the motor vehicle operates as a passenger-side display. For example, the passenger-side display may be implemented separately from or in conjunction with the vehicle's center display.

[0022] According to one aspect of this disclosure, an alternative provision is made to operate the display of a mobile device carried by the front passenger in the motor vehicle as a front passenger display. Typically, passengers may watch media on mobile devices they bring into the vehicle, such as mobile phones, tablets, or laptops, which can also lead to driver distraction. For example, the mobile device may be a device coupled to the vehicle. To control the display of the mobile device, the method can be performed on the mobile device itself. In this regard, the method may optionally include: identifying that the mobile device is currently in use in a moving motor vehicle (this is possible, for example, by means of GPS signal analysis or camera analysis). Alternatively, it may be provided that when the method determines that the front passenger's activity may lead to driver distraction, the vehicle sends a signal to the mobile device. Accordingly, the method may include: sending a display control signal (e.g., a deactivation signal in the event of an identification of a risk of driver distraction via the front passenger display due to the front passenger's activity) from the vehicle to the front passenger's mobile device for controlling the display of the mobile device and receiving the display control signal at the mobile device.

[0023] Another aspect relates to a computer program product or non-volatile storage medium having program code to perform the methods described above or below when the computer program is executed on a processor, computer, or programmable hardware. Such a program can advantageously be executed on an in-vehicle computer, or alternatively on a backend (e.g., a server or cloud service coupled to the vehicle), or alternatively on the passenger's mobile device, to perform the proposed methods and control the passenger display. By applying the computer program product, control signals can be pre-sent to the control unit of the passenger display to deactivate the passenger display, for example, when there is a risk that the driver will view the content of the passenger display.

[0024] Another aspect relates to a data processing device for a motor vehicle, wherein the data processing device is configured to implement the methods described above or below. Another aspect relates to a motor vehicle that includes such a data processing device or is configured to perform the methods described above or below, particularly by means of sensors and computing devices within the vehicle. Attached Figure Description

[0025] The various embodiments are further described below with reference to the accompanying drawings. Wherein are shown: Figure 1 A flowchart illustrating a method for operating a co-pilot display is shown; and Figure 2a , 2bThe diagram illustrates an example in which the vehicle's passenger display is deactivated early due to interaction between the passenger and the driver in order to reduce potential distraction of the driver. Detailed Implementation

[0026] Different embodiments will now be described in more detail with reference to the accompanying drawings, in which some embodiments are illustrated. In the drawings, the thickness of lines, layers, and / or regions may be exaggerated for clarity. In the following description of the accompanying drawings, which illustrate only some exemplary embodiments, the same reference numerals may denote the same or comparable components.

[0027] An element described as being "connected" or "coupled" to another element may be directly connected or coupled to the other element, or there may be elements in between. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as those given to them by one of ordinary skill in the art to which the embodiments pertain.

[0028] Figure 1 A flowchart is shown for a method 10 for operating a passenger-side display in a motor vehicle. Method 10 includes determining 11 passenger-side activity, wherein the passenger-side activity relates to interaction between the passenger-side driver and the driver of the vehicle. Method 10 further includes controlling 12 the display of the passenger-side display based on the determined passenger-side activity.

[0029] For example, it is proposed to couple the display or content on the passenger-side display to whether the passenger-side display is in use. According to other solutions, a solution is known in which the passenger-side display is turned off when the driver looks at it, which can be identified by an in-vehicle camera. However, this disclosure proposes, for example, detecting the passenger's gaze relative to the display (e.g., passenger activity) via an in-vehicle camera, and deactivating or at least dimming it when it is not in use. Specifically, it is specified that the display can be pre-locked based on the passenger's behavior (e.g., passenger activity) if it is assumed that the behavior is distracting to the driver. This is done, for example, based on the application of the in-vehicle camera (e.g., optical analysis; e.g., the passenger pointing at the screen when content is active and possibly looking at the driver) and / or by analyzing interactions (e.g., between the passenger and the driver) using voice in the interior space (e.g., semantic analysis). When, for example, the passenger actively asks the driver to look at the passenger-side display (e.g., through the passenger's words: "Hey, look at this"), the display is immediately locked, even if the driver's gaze in that direction is not detected (e.g., even before the driver's gaze is detected). Therefore, the displayed content can be locked before the driver, for example, makes a distracting movement by looking at the passenger-side display.

[0030] The proposed method enables the deactivation of the passenger-side display before driver inattention is detected. More specifically, it allows for preemptive deactivation when a risk of driver inattention is detected due to current passenger-side activity. Furthermore, the passenger-side display can generally be deactivated when it is identified that it will not be used (e.g., when passenger-side activity involves communication with the driver that is not directed to the display's content). Deactivating or dimming the passenger-side display, for example, enables energy-efficient operation.

[0031] Figure 2a and Figure 2b An exemplary process is illustrated schematically in which the passenger-side display is deactivated due to detected passenger-side activity in order to prevent potential distraction of the vehicle's driver by viewing the contents of the passenger-side display.

[0032] Figure 2aThe interior space of a vehicle in a first scenario 20a is schematically shown. The driver 21 sits in the driver's seat located in front of the steering wheel 23 and performs driving activities. The co-driver 22 sits in the co-driver's seat and views a co-driver display 24, which displays content ABC. In the vehicle, a first unit 25a and a second unit 25b are provided for determining the co-driver's activities; the first unit and the second unit are, for example, an optical sensor 25a (e.g., an in-vehicle camera) and an acoustic sensor 25b (e.g., an in-vehicle microphone).

[0033] For example, it can be determined by the method that the co-pilot 22 has fallen asleep and is no longer looking at the co-pilot display 24. In this case, for example, the co-pilot display 24 can be deactivated, so that the display content ABC is no longer shown. For example, when the driver happens to look at the co-pilot display 24, this can prevent the driver from being accidentally distracted by the content of the co-pilot display 24.

[0034] Figure 2b The interior space of the vehicle in the second scenario 20b is illustrated schematically. The front passenger 22 is interested in the displayed content ABC and wants to discuss it with the driver 21. For example, the front passenger 22 may use a finger gesture 26a (illustratively shown) pointing to the front passenger display 24 and / or words 26b (e.g., “Hey, take a look at this”) to try to draw the driver 21’s attention to the displayed content ABC.

[0035] According to the method, in case 20b, determination 11 is immediately performed when the two units 25a and 25b are applied, such that the current co-pilot activities 26a and 26b are set to actively notify the driver 21 about the displayed content ABC of the co-pilot display 24. For example, camera 25a can be used to determine finger pointing 26a, and microphone 25b can be used to determine and analyze the utterance 26b directed at the driver 21. Accordingly, the co-pilot display 24 can be immediately deactivated, making the content ABC no longer visible. Accordingly, the driver 21 no longer has any reason to look in the direction of the co-pilot display 24, since the content is no longer displayed on the co-pilot display. Thus, it is possible to prevent the driver 21 from being potentially distracted from their driving activities, especially at an early stage. To determine the co-pilot activities 11, artificial intelligence can be used (e.g., in the semantic analysis of utterance 26b), which can associate the utterance content of the co-pilot 22's utterance with the displayed content ABC.

[0036] Additional details and aspects are mentioned in connection with the embodiments described in the context. Figure 2a and Figure 2bThe embodiments shown may have one or more optional additional features, which correspond to one or more aspects, which are related to the proposed scheme or one or more of the above (e.g., Figure 1 (or the embodiments described below) are mentioned in connection with them.

[0037] The examples relate to a method for controlling a passenger-side display (e.g., using an in-vehicle camera) in such a way that the display can be deactivated or at least dimmed when not in use, and / or pre-locked based on the passenger's behavior regarding interactions with the driver, so as not to distract the driver, for example.

Claims

1. A method (10) for operating a passenger-side display (24) in a motor vehicle, the method (10) comprising: - Determine (11) the co-driver activities (26a, 26b), wherein the co-driver activities (26a, 26b) involve the interaction between the co-driver (22) and the driver (21) of the vehicle; and - Control the display of the co-pilot display (24) of (12) according to the co-pilot activities (26a, 26b) determined (11).

2. The method (10) according to claim 1, wherein, When the determination (11) of the co-pilot's activity (26a, 26b) indicates that the co-pilot (22) is not currently using the co-pilot display (24), the brightness of the co-pilot display (24) is reduced.

3. The method (10) according to claim 1 or 2, wherein, The determination (11) of the co-driver's activities (26a, 26b) includes checking whether the co-driver (22) notifies the driver (21) of the vehicle about the contents (ABC) of the co-driver's display (24).

4. The method (10) according to claim 3, wherein, The inspection is based on semantic analysis of the voice signal of the co-driver (22).

5. The method (10) according to claim 3 or 4, wherein, The inspection is performed based on optical analysis of the image signals from the co-driver (22).

6. The method (10) according to any one of claims 3 to 5, wherein, Once the determined co-driver activity (26a, 26b) results in the co-driver (22) informing the driver (21) of the contents (ABC) of the co-driver display (24), the co-driver display (24) is immediately deactivated.

7. The method (10) according to any one of claims 1 to 6, wherein, The vehicle’s interior display is operated as the passenger driver’s display (24).

8. The method (10) according to any one of claims 1 to 6, wherein, The display of the mobile device carried by the co-driver in the motor vehicle is used as the co-driver display (24).

9. A computer program product or non-volatile storage medium having program code for performing the method (10) according to any one of the preceding claims when the computer program is executed on a processor, computer or programmable hardware.

10. Data processing equipment for motor vehicles, wherein, The data processing device is configured to implement the method (10) according to any one of claims 1 to 8.

Images