Display device, method and computer program for operating a display device

Abstract

The present invention relates to a display device (1) comprising a display panel (2), a recording device (3) arranged behind the display panel (2) in the viewing direction, and a control unit (4). The control unit (4) is configured to switch pixels (20) of the display panel (2) into a recording state by means of a bypass circuit (6), independently of a graphics rendering path (7) of the display device (1). The invention further relates to a method and a computer program for operating such a display device (1).

Description

[0001] The present invention relates to a display device with an integrated recording device, as well as a method and a computer program for operating such a display device.

[0002] Monitoring the operator and other users of a means of transport, as well as the interior of the means of transport, using cameras is a common approach to improve safety and user comfort in the means of transport.

[0003] For example, DE 10 2019 218 936 A1 describes a vehicle which has at least two image capture units which are arranged in the interior of the vehicle and which are designed to capture a first and a second single image or a first and a second single video of the interior.

[0004] The camera is often visible to the occupants, who may feel they are being watched. Particularly with cameras designed to capture visible light, it is difficult to position the camera discreetly, as these cameras typically require a clear cover glass.

[0005] One way to discreetly place a camera is to position it behind an existing display device.

[0006] For example, WO 2009 / 051 673 A1 describes a device for displaying and recording images. The device contains a liquid crystal panel that can be switched between a display state and a recording state. In the recording state, at least part of the display and a switchable diffuser become transparent. Behind the display are image recording devices which, in the recording state, capture images of the scene in front of the device through holes or windows in the backlight.

[0007] However, placing a visible light camera behind the active display area is problematic, as the image information displayed by the screen can lead to artifacts in the camera data, for example, due to stray light or crosstalk. For this reason, the camera's recording must be synchronized with the image playback so that the active phases of recording and playback alternate or differ in time.

[0008] Against this background, US 2023 / 0298534A1 describes a method for operating a display device. The display device comprises an LCD screen held by a housing, a camera located in the housing, and a backlight for the LCD screen. In the method, pixels of the LCD screen are switched to allow the camera to capture an image between the display of two images. Simultaneously, the backlight is switched off.

[0009] US 2017 / 0084231A1 describes a method for operating a display device with a camera and a transparent display. The method determines whether an image sensor is in an image acquisition mode. If so, pixels of the display in an area surrounding the image sensor are switched to a transparent mode until the image sensor exits image acquisition mode.

[0010] Switching the pixels of a liquid crystal display (LCD) to a transparent state or deactivating the pixels of a micro-LED (LED) or organic light-emitting diode (OLED) panel involves a significant latency. This is problematic because it can lead to noticeable impairments in image reproduction.

[0011] US 10,062,322 B2 describes a device that combines a light sensor and a display. The light sensor is located beneath the display and can detect incident light while the display shows an image. For this purpose, an area of ​​the display located above the light sensor can be made transparent.

[0012] It is an object of the invention to provide a display device with an integrated recording device as well as solutions for operating such a recording device that enable reduced latency times.

[0013] This problem is solved by a recording device having the features of claim 1, by a method according to claim 10 for operating such a display device, and by a computer program with instructions according to claim 11. Preferred embodiments of the invention are the subject of the dependent claims.

[0014] According to a first aspect of the invention, a display device comprises: - a display panel; - a recording device arranged behind the display panel in the direction of view; and - a control unit configured to switch pixels of the display panel into a recording state by means of a bypass circuit independently of a graphics rendering path of the display device, wherein the bypass circuit is integrated in the area of ​​the pixels to be switched on the display panel and can be activated via control lines.

[0015] Preferably, the control unit is configured to switch pixels of the display panel into a playback state after recording, independently of the graphics rendering path, using the bypass circuit.

[0016] According to a further aspect of the invention, a method for operating a display device with a display panel and a receiving device arranged behind the display panel in the viewing direction comprises the steps: - Switching pixels of the display panel into a recording state, wherein the switching is carried out by means of a bypass circuit which is integrated in the area of ​​the pixels to be switched on the display panel and can be activated via control lines, independently of a graphics rendering path of the display device; - Triggering a recording by the recording device; and - Switching pixels of the display panel to a playback state after recording, whereby the switching is carried out independently of the graphics rendering path using the bypass circuit.

[0017] According to a further aspect of the invention, a computer program comprises instructions which, when executed by a computer, cause the computer to perform the following steps for operating a display device with a display panel and a receiving device arranged behind the display panel in the viewing direction: - Switching pixels of the display panel into a recording state, wherein the switching is carried out by means of a bypass circuit which is integrated in the area of ​​the pixels to be switched on the display panel and can be activated via control lines, independently of a graphics rendering path of the display device; - Triggering a recording by the recording device; and - Switching pixels of the display panel to a playback state after recording, whereby the switching is carried out independently of the graphics rendering path using the bypass circuit.

[0018] The term "computer" is to be understood broadly. In particular, it also includes embedded systems, control units, and other processor-based data processing devices. Furthermore, the individual steps are not necessarily executed directly by the computer. It is equally possible that the computer controls or relies on external components to perform individual steps.

[0019] The computer program can, for example, be made available for electronic retrieval or be stored on a computer-readable storage medium.

[0020] To eliminate the influence of the display panel's image content during recording by the recording device, the pixels above the recording device area are switched to a state suitable for recording in the solution according to the invention. This switching must occur very quickly. It has been found that control via the graphics rendering path, i.e., via the image content of the graphics controller, is generally too slow due to the latencies in the graphics rendering path. For this reason, at least the pixels above the recording device area are controlled by means of a special bypass circuit. This circuit is integrated into the display panel in the corresponding area and can be activated via control lines to briefly override the application-specific image display. The bypass circuit is activated via a control line supplied by a higher-level control system, e.g., a graphics controller.A microcontroller, an ASIC (Application-Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array) selects specific pixels during recording. In the solution according to the invention, recording and image playback are synchronized, so that recording only occurs when the pixels above the recording device are in the recording state.

[0021] According to one aspect of the invention, the recording device is a camera for recordings in the visible wavelength range. The solution according to the invention is particularly advantageous for such a camera because it operates in a wavelength range in which the light emission from the display panel also occurs. The use of a camera for recordings in the visible wavelength range is particularly useful when the camera is used for applications where recordings in natural colors are required for a human viewer, such as in video telephony.

[0022] According to one aspect of the invention, the display panel is a non-self-illuminating display panel. The control unit is configured to switch pixels of the display panel into a transparent state by means of a bypass circuit for an upcoming image acquisition by the recording device. An example of a non-self-illuminating display panel is a liquid crystal panel. In a non-self-illuminating display panel, the pixels must be switched to a transmissive state above the recording device in order to enable image acquisition. For this purpose, pixel voltages can be applied such that the transmission of the pixels is maximized. Since the switching times of the pixels of a liquid crystal panel contribute to the latency, the use of a bypass circuit is particularly advantageous here.

[0023] According to one aspect of the invention, the control unit is configured to switch off the backlight of the display device before an image is captured by the recording device. A non-self-illuminating display panel requires backlighting to generate images for a viewer. For example, a surface backlight or a matrix backlight can be used. Since the light emitted by the backlight can interfere with image capture, the backlight is preferably switched off for the duration of the image capture.

[0024] According to one aspect of the invention, the control unit is configured to locally switch off the backlight of the display device in the area of ​​the receiving device for an upcoming image capture. With a matrix backlight, the receiving device is located behind a single cavity or a few cavities, depending on its size. In this case, it is sufficient to switch off only the light sources located in these cavities.

[0025] According to one aspect of the invention, the display panel is a self-illuminating display panel. The control unit is configured to switch off pixels of the display panel by means of a bypass circuit for an upcoming image acquisition by the recording device. Examples of self-illuminating display panels are a microLED panel or an OLED panel. In a self-illuminating display panel, the pixels above the recording device are adjusted by the control unit such that these pixels no longer emit light for the time slice of the image measurement.

[0026] According to one aspect of the invention, pixels of the display panel are provided on the reverse side, at least in the area of ​​the recording device, with an opaque masking. Such masking, for example a black print, can further reduce the scattered light incident on the recording device.

[0027] According to one aspect of the invention, the control unit is configured to trigger a recording by the recording device. This has the advantage that the recording can be easily synchronized with the switching of the pixels. The recording can be triggered directly by the control unit sending a corresponding signal to the recording device. Alternatively, the recording can be triggered indirectly by the control unit signaling to a higher-level system that the switching of the pixels is complete and a recording can be made.

[0028] Preferably, a display device according to the invention is used in a means of transport. The means of transport can be, for example, a motor vehicle, but alternatively also an aircraft, a rail vehicle, or a watercraft. The display device can be provided for different installation locations, for example, as a driver information display, as a central display, as a passenger display, as a display for the rear seats, or as a display installed in the headliner.

[0029] Further features of the present invention will become apparent from the following description and the attached claims in conjunction with the figures. Figure overview Fig. Figure 1 schematically shows a first embodiment of a display device according to the invention; Fig. Figure 2 schematically shows a second embodiment of a display device according to the invention; Fig. Figure 3 schematically shows a third embodiment of a display device according to the invention; Fig. Figure 4 schematically shows a fourth embodiment of a display device according to the invention; Fig. Figure 5 schematically shows a method for operating a display device according to the invention; Fig. Figure 6 shows an embodiment of a device for operating a display device according to the invention; and Fig. Figure 7 schematically shows a means of transport that uses a display device according to the invention. Character description

[0030] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. The same reference numerals are used in the figures for identical or equivalently acting elements and are not necessarily described again for each figure. It is understood that the invention is not limited to the embodiments shown and that the described features can also be combined or modified without departing from the scope of protection of the invention as defined in the appended claims.

[0031] Fig. Figure 1 schematically shows a first embodiment of a display device 1 according to the invention. The display device 1 has a display panel 2 and a recording device 3 arranged behind the display panel 2 in the viewing direction. The recording device 3 can, for example, be a camera for recordings in the visible wavelength range. In this embodiment, the display panel 2 is a non-self-illuminating display panel, for example, a liquid crystal panel. A backlight 5 is therefore arranged below the display panel 2, which here comprises a planar light guide 50 with laterally adjacent light sources 51. The recording device 3 is arranged behind the light guide 50 in the viewing direction. The image is generated by means of a graphics rendering path 7, which is only schematically indicated here.Additionally, the display device 1 has a control unit 4 configured to switch pixels 20 of the display panel 2 into a recording state independently of the graphics rendering path 7 by means of a bypass circuit 6. For this purpose, at least the pixels 20 above the recording device 3 are transmissively switched. The bypass circuit 6 is integrated on the display panel 2 in the corresponding area and can be activated via control lines 22 to briefly override the application-specific image display. The control unit 4 is also configured to switch off the backlight 5 for an upcoming recording by the recording device 3. For this purpose, the control unit 4 preferably switches off the laterally adjacent light sources 51. Optionally, the control unit 4 can also trigger the recording, e.g., by sending a corresponding signal to the recording device 3.After a successful recording, the control unit 4 can switch the pixels 20 back to a playback state using the bypass circuit 6 and turn the backlight back on.

[0032] Fig. Figure 2 schematically shows a second embodiment of a display device 1 according to the invention. The display device 1 largely corresponds to the one described in Figure 2. Fig. The display device 1 shown in Figure 1. However, in this embodiment, the backlight 5 is designed as a matrix backlight. This comprises a plurality of reflectively designed cavities 52, each containing a light source 51. The light sources 51 are arranged on a circuit board 53. The receiving device 3 is located behind a single cavity 52. ​​Receipt is achieved through an opening 54 in the circuit board 53 and a space 55 encompassing the light source 51. In this case, it is sufficient to switch off only the light source 51 in the cavity 52 through which the receipt is received.

[0033] Fig. Figure 3 schematically shows a third embodiment of a display device 1 according to the invention. In this embodiment, the display panel 2 is a self-illuminating display panel 2, e.g., a microLED panel or an OLED panel. The control unit 4 is therefore configured to switch off pixels 20 of the display panel 2 by means of the bypass circuit 6 for an upcoming image acquisition by the recording device 3. The pixels 20 above the recording device 3 are adjusted by the control unit 4 such that these pixels 20 no longer emit light for the time slice of the image measurement. The bypass circuit 6 is also integrated on the display panel 2 in the corresponding area and can be activated via control lines 22. Optionally, the control unit 4 can also trigger the image acquisition in this embodiment, e.g., by sending a corresponding signal to the recording device 3.After a successful recording, the control unit 4 can switch the pixels 20 back to a playback state using the bypass circuit 6.

[0034] Fig. Figure 4 schematically shows a fourth embodiment of a display device 1 according to the invention. The display device 1 largely corresponds to the one described in Fig. 3 Display device 1 shown. However, the pixels 20, at least in the area of ​​the recording device 3, are provided on the back with an opaque masking 21, e.g., a black print. Such a masking 21 can further reduce the stray light striking the recording device 3. In Fig. 4. The masking 21 is not located directly on the pixels 20, but on the back of the display panel 2. The specific placement and design of the masking 21 is at the discretion of the expert.

[0035] Fig. Figure 5 schematically shows a method for operating a display device according to the invention with a recording device, e.g., a camera for recordings in the visible wavelength range. In a first step, pixels of the display panel are switched to a recording state S1. This switching S1 is carried out by means of a bypass circuit, independent of a graphics rendering path of the display device. The display panel can be a self-illuminating or a non-self-illuminating display panel. In the case of a self-illuminating display panel, pixels of the display panel are switched off by means of the bypass circuit for an upcoming recording by the recording device. In the case of a non-self-illuminating display panel, the corresponding pixels are switched to a transparent state.Additionally, the control unit can be configured to completely or locally switch off the backlight of the display device in the area of ​​the recording device for an upcoming recording (S2). After the switching of the pixels (S1) and, if applicable, the switching off of the backlight (S2) are complete, a recording is triggered (S3). After completion of the recording, the pixels of the display panel are switched back to a playback state by means of the bypass circuit (S4). If present, the backlight can also be switched back on (S5).

[0036] Fig. Figure 6 shows a simplified schematic representation of an embodiment of a device 10 for operating a display device according to the invention. The device 10 has a memory 11 and a processor 12. For example, the device 10 is a computer, an embedded system, or a control unit. Instructions are stored in the memory 11 which, when executed by the processor 12, cause the device 10 to perform the steps according to one of the described methods. The instructions stored in the memory 11 thus embody a program executable by the processor 12, which implements the method according to the invention. The device 10 has an input 13 for receiving information. Data generated by the processor 12 is provided via an output 14. Furthermore, it can be stored in the memory 11. The input 13 and the output 14 can be combined into a bidirectional interface.

[0037] The processor 12 can comprise one or more processor units, such as microprocessors, digital signal processors, or combinations thereof.

[0038] Memory 11 can contain both volatile and non-volatile memory areas and can include a wide variety of storage devices and media, such as hard disks, optical storage media, or semiconductor memory.

[0039] Fig.Figure 7 schematically shows a means of transport 30 that uses a display device 1 according to the invention. In this example, the means of transport 30 is a motor vehicle. The motor vehicle has a display device 1 according to the invention, which in the illustrated example is arranged in an instrument panel 31. The display device 1 can be used, for example, to display instructions from a navigation system 32 or information from an assistance system 33. A connection to a backend can be established by means of a data transmission unit 34, for example, to obtain updated software for components of the motor vehicle. A memory 35 is provided for storing data. Data exchange between the various components of the motor vehicle takes place via a network 36. Reference symbol list 1 Display device 2 Display panels 20 pixels 21 Masking 22 Control line 3 Recording device 4 Control unit 5 Backlight 50 fiber optic cables 51 Light source 52 cavities 53 Circuit board 54 Opening 55 space 6 Bypass circuit 7. Graphics rendering path 10 Device 11 storage 12 processor 13 Entrance 14 Exit 30 means of transport 31 Instrument panel 32 Navigation system 33 Assistance systems 34 Data transmission unit 35 storage 36 Network S1 Switching pixels into a recording state S2 Switching off a backlight S3 Triggering a recording S4 Switching pixels to a playback state S5 Turn on the backlight

Claims

[1] Display device (1) comprising: - a display panel (2); - a recording device (3) arranged behind the display panel (2) in the direction of view; and - a control unit (4) which is configured to switch pixels (20) of the display panel (2) into a recording state (S1) by means of a bypass circuit (6) independently of a graphics rendering path (7) of the display device (1), wherein the bypass circuit (6) is integrated in the area of ​​the pixels (20) to be switched on the display panel (2) and can be activated via control lines (22) to temporarily override the application-specific image display. [2] Display device (1) according to claim 1, wherein the recording device (3) is a camera for recordings in the visible wavelength range. [3] Display device (1) according to claim 1 or 2, wherein the display panel (2) is a non-self-illuminating display panel, and wherein the control unit (4) is configured to switch pixels (20) of the display panel (2) into a transparent state by means of the bypass circuit (6) for an upcoming recording by the recording device (3). [4] Display device (1) according to claim 3, wherein the control unit (4) is configured to switch off a backlight (5) of the display device (1) for an upcoming recording by the recording device (3) (S2). [5] Display device (1) according to claim 3, wherein the control unit (4) is configured to switch off a backlight (5) of the display device (1) locally in the area of ​​the receiving device (3) for an upcoming recording by the receiving device (3) (S2). [6] Display device (1) according to claim 1 or 2, wherein the display panel (2) is a self-illuminating display panel, and wherein the control unit (4) is configured to switch off pixels (20) of the display panel (2) by means of the bypass circuit (6) for an upcoming recording by the recording device (3). [7] Display device (1) according to claim 6, wherein pixels (20) of the display panel (2) are provided on the reverse side with an opaque masking (21) at least in the area of ​​the receiving device (3). [8] Display device (1) according to one of the preceding claims, wherein the control unit (4) is configured to trigger a recording by the recording device (3) (S3). [9] Display device (1) according to one of the preceding claims, wherein the control unit (4) is configured to switch pixels (20) of the display panel (2) into a playback state (S4) after recording, independently of the graphics rendering path (7), by means of the bypass circuit (6). [10] Method for operating a display device (1) with a display panel (2) and a receiving device (3) arranged behind the display panel (2) in the viewing direction, comprising the steps: - Switching (S1) pixels (20) of the display panel (2) into a recording state, wherein the switching (S1) is carried out by means of a bypass circuit (6) which is integrated in the area of ​​the pixels (20) to be switched on the display panel (2) and can be activated via control lines (22), independently of a graphics rendering path (7) of the display device (1), thereby briefly overriding the application-specific image display; - Triggering (S3) of a recording by the recording device (3); and - Switching (S4) pixels (20) of the display panel (2) after recording into a playback state, wherein the switching (S4) is carried out by means of the bypass circuit (6) independently of the graphics rendering path (7). [11] Computer program with instructions which, when executed by a computer, cause the computer to perform the steps of a method according to claim 10 for operating a display device (1).

Images