Suppression of reflections on reflective display surfaces of mobile devices in a vehicle

Abstract

The invention relates to a method for suppressing reflections on display surfaces of mobile devices when capturing their display contents in a vehicle, comprising the steps: - capturing a display content of a mobile device by at least one interior camera of the vehicle using at least one predetermined polarization filter; - analyzing the captured display content for corresponding polarization properties in the case of display light from this device on the one hand and in the case of stray light that is mixed with the display light via reflection from its display surface on the other hand; and - Using polarization properties of the display light and / or the stray light extracted thereby for digital post-processing of this or subsequently captured display content, in which at least a portion of the stray light is subtracted in individual image pixels in order to increase the proportion of display light therein and to reduce or almost completely remove the proportion of stray light.

Description

[0001] The invention relates to a method for suppressing disruptive reflections on reflective display surfaces of mobile devices in a vehicle, in particular a motor vehicle. The invention also relates to a correspondingly configured control unit and a vehicle equipped therewith.

[0002] The interior of a motor vehicle is comparatively dark compared to external light sources such as the sun, streetlights, signal lights, or headlights. If light entering the vehicle from outside falls on the reflective display surface of a mobile device used in the vehicle, such as a smartphone, the reflection of this light from the display surface can cause glare, rendering the displayed content, such as text or images, invisible or unreadable.

[0003] The use of polarizing filters is known in various fields, including to suppress disturbing light reflections. In photography, for example, polarizing filters are used to suppress sky reflections on a water surface. Recently, industrial cameras have also become available with multiple polarizing filters directly on the sensor, arranged in a checkerboard pattern, allowing them to have three different polarization angles and one unpolarized channel.

[0004] It is an object of the present invention to provide a concept for suppressing disturbing reflections on reflective display surfaces of mobile devices in a vehicle, in particular a motor vehicle.

[0005] This object is achieved by a method according to claim 1 for suppressing reflections on the display surfaces of mobile devices when capturing their display contents in a vehicle, as well as by a correspondingly configured control unit and a vehicle equipped therewith according to the independent claims. Further embodiments are specified in the dependent claims. All further features and effects mentioned in the claims and the following description for the method also apply to the control unit and the vehicle, and vice versa.

[0006] According to a first aspect, a method is provided for suppressing reflections on the display surfaces of mobile devices (such as smartphones, laptops, etc.) when capturing their display contents in a vehicle. The vehicle can be, in particular, a motor vehicle, but alternatively also any other land, air, or water vehicle.

[0007] In the method presented here, the vehicle is equipped with at least one interior camera designed and configured to optically capture display content of a mobile device used in the vehicle. Capturing and processing such display content can be useful for various applications in the vehicle that are not described in detail herein.

[0008] The procedure includes the following steps: The display content of a mobile device used in the vehicle is captured by at least one of these interior cameras using at least one predetermined polarization filter. Knowledge of the polarization properties of the filter used is used to analyze the captured display content for the presence of corresponding polarization properties in the display light of this device, on the one hand, and in the presence of interfering light that is mixed with the display light via reflection from its display surface, on the other. The polarization properties of the display light and / or the interfering light obtained / extracted through this analysis are used for digital post-processing of this display content or subsequently captured display content.In this digital post-processing, at least part of the stray light in individual image pixels of the respective display content is subtracted in order to increase the proportion of display light therein and to reduce the proportion of stray light or, ideally, even to remove it completely.

[0009] One idea behind this method, or rather its working principle, is based on analyzing the different polarization properties of the interfering light, which can come, for example, from the sky or artificial light sources inside and outside the vehicle, and the display light that conveys the actual display content. The use of at least one polarization filter in the vehicle's interior camera(s) can help extract information about the various light sources within a captured image, which enables these light sources to be filtered during digital post-processing of the image. For example, during this post-processing of the captured display content, it is possible to filter out reflections coming from the sky and increase the proportion of display light generated by the mobile device itself. The display light can, but does not have to, be polarized in the present method.Nor is any connection between the vehicle and the mobile device required that could transmit any information about the polarization properties of its display light or about its display contents directly to the vehicle.

[0010] In this method, the stray light can originate from one or more light sources with different intensities and different polarization properties. Especially light sources outside the vehicle, such as the sun, headlights, or streetlights, can cause disturbing reflections on the display surface and even overshadow the display light due to their significantly higher light intensity compared to the display light.

[0011] The aforementioned analysis of the captured display content may, for example, include a pixel-by-pixel comparison. This comparison can be performed both within a single image capture and between multiple images captured with different polarization filters and otherwise under identical conditions.

[0012] For example, if the use of a predetermined linear polarizer in the indoor camera results in a uniformly illuminated display surface, while a crossed polarizer produces clear display content against a black background, then in this simple case it can be concluded that the display light is linearly polarized (e.g., p-polarized), while a key interfering light source (e.g., atmospheric scattered light) has a linear polarization perpendicular to this (e.g., s-polarized), so that this interfering light can be completely subtracted during digital post-processing of the image. In reality, the composition of the light from different light sources is usually more complex and requires correspondingly more complex analysis steps.

[0013] The aforementioned post-processing of the display contents for their further processing or use in the vehicle can be implemented at least partially in the respective camera, alternatively or additionally in an outsourced control or processing unit that is connected to the camera wirelessly or wired.

[0014] According to one embodiment, at least one of the polarization filters mentioned is a static polarization filter designed to filter out (in the sense of transmitting or, conversely, blocking) light of a predetermined polarization typical of light coming from the sky. In this embodiment, the analysis checks the individual image pixels of the captured display content for the presence and proportion of light with this polarization, and from this, a proportion of this light to be subtracted (which can in particular, but not necessarily, be the same for all pixels) is determined. Thus, when using the method presented here with only one static polarization filter, reflections of light coming from the sky can be greatly reduced.

[0015] According to a further embodiment, the one or more interior cameras are configured for simultaneous image capture with and without a predetermined static polarization filter. In this embodiment, a display content of the mobile device is thus captured simultaneously with and without this polarization filter. The two images thus obtained are compared pixel by pixel to extract the polarization properties of the display light, on the one hand, and the stray light, on the other.

[0016] For example, two cameras and a static filter can be used. This allows either two sensors (e.g., a CCD / CMOS chip), each with a separate optical system (e.g., a lens or lens combination), or a sensor with two lenses (focusing the same image on different parts of the sensor), with a fixed / static polarization filter added to only one of these optical systems. This makes it possible to assign the two simultaneously acquired images to each other pixel by pixel (and optionally also to calibrate any light intensity differences due to the other optical system), thereby extracting information about a polarized noise or display light source.Similar to a movable filter with only two positions (on and off), this would allow some information about the stray light sources to be extracted for each pixel of the captured display content and thus subtracted / ignored. Unlike a movable polarizing filter, which allows the use of a single camera but requires both images to be captured consecutively, this embodiment captures both images simultaneously, which can significantly increase the precision of their comparative analysis.

[0017] According to a further embodiment, at least one of the interior cameras is designed to capture multiple images directly one after the other, each with a different predetermined polarization filter. This can be achieved in particular by a rotatable or rotating filter (for example, in the form of a disk) that positions a different polarization filter (for example, with a different polarization angle of a linear polarization) in front of the camera depending on its rotational position.

[0018] In this embodiment, a display content is captured multiple times in succession, each time using a different polarization filter. The resulting images are compared pixel by pixel to extract the polarization properties of the display light and the stray light. In this way, for example, the amounts of light coming from the sky, the display, and the environment can be separated: Ambient light is usually diffuse / non-polarized, whereas the display light and the atmospheric scattered light coming from the sky are polarized, most likely in different directions. Performing an analysis across multiple individual images with different polarization angles allows for precise determination of the polarization angles of the light from different light sources.This can enable the distribution of different light sources into individual / separate images, so that any reflections on the screen (display), including diffusely scattered stray light, can be eliminated during subsequent post-processing of the display content.

[0019] As already mentioned, the display light can, in particular, have a predetermined polarization that differs from a typical polarization of the stray light coming from the sky. In this case, two or more different polarization filters can be used in the at least one interior camera, for example, which make it possible to separate the portion of the display light from the portion of the light coming from the sky and from the portion of the remaining, non-polarized ambient light in images of the display content captured simultaneously or directly one after the other using these different polarization filters.

[0020] As already mentioned, the respective display content of the mobile device can be passed on to a vehicle-mounted control unit after the aforementioned post-processing in order to be further used by it for specific applications in the vehicle.

[0021] According to a further aspect, a control unit is provided which is designed and configured for use (for example, by installation or integration) in the above-mentioned vehicle or in the at least one interior camera and for automatically executing the above-mentioned method. For this purpose, a corresponding computer program (software) can be loaded into a processor of the control unit, for example, and run during operation of the vehicle or the camera.

[0022] According to a further aspect, the above vehicle is provided, which comprises at least the following: - an interior space for one or more vehicle occupants; - at least one interior camera designed and configured to capture display contents of a mobile device used in the vehicle and comprising at least one predetermined polarization filter; and - the above control unit.

[0023] The above aspects of the invention and their embodiments and specific configurations are explained in more detail below with reference to examples shown in the accompanying drawings. The drawings are to be understood as purely schematic illustrations of the basic functional principle and are not necessarily to scale. They show: Fig. 1 shows a schematic longitudinal sectional view of a vehicle which is designed and equipped to carry out the method according to an embodiment of the invention; and Fig. 2 diagrams of a polarization angle dependent light intensity of the total of a display surface in the vehicle of the Fig. 1 outgoing light as well as the components of the display light, the light from the sky and the remaining diffuse ambient light.

[0024] All of the various embodiments, alternatives and specific design features of the method, the associated control unit and the vehicle according to the above aspects of the invention mentioned above in the description and in the following claims can be used in the Fig. 1 and Fig. 2, in particular alternatively or in addition to the features shown therein. They will therefore not be repeated again below. The same applies accordingly to the definitions and effects already given above with regard to individual features that are described in Fig. 1-2 are shown.

[0025] Fig. Figure 1 shows a highly simplified schematic longitudinal section of a vehicle 1 according to an embodiment of the invention. In this example, the vehicle 1 is a motor vehicle, the individual components of which are shown in Fig. 1 are only symbolically indicated.

[0026] The vehicle 1 has an interior 2 for one or more vehicle occupants (not shown) and is equipped with at least one interior camera 3, which is designed and configured for optically capturing display contents of a mobile device 4 used in the vehicle 1, for example a smartphone. The at least one interior camera 3 can comprise one or more individual cameras, which are Fig. 1 are not shown individually and may use one or more predetermined polarization filters 5 in their operation.

[0027] Furthermore, the vehicle 1 comprises a control unit 11, which is designed and configured for automated execution of a method according to an embodiment of the invention and for corresponding communication with the at least one interior camera 3. The method comprises the following steps: A display content presented on a display surface 6 of the mobile device 4 is captured by the at least one interior camera 3 using the at least one polarization filter 5. The polarization properties of the filter used in each case are known in advance and are used to analyze the captured display content for the presence of corresponding polarization properties in the display light and in the stray light reflected from the display surface 6. The polarization properties of the display light and / or the stray light thus extracted are used for digital post-processing of this display content or subsequently captured display content by subtracting at least a portion of the stray light in individual image pixels of the respective display content in order to increase the proportion of display light therein and reduce the proportion of stray light, and ideally, to remove it completely.

[0028] Fig. 2 shows, in a possible example, individual diagrams of the polarization angle-dependent light intensity of the total light 10 emitted by the display surface 6 in the vehicle 1 of the Fig. 1 and is captured by at least one interior camera 3, as well as the individual components of the display light 7, the light 8 originating from the sky, and the remaining diffuse ambient light 9 contained therein. As can be seen from these diagrams, the display light 7 in this example has a polarization whose polarization angle dependence differs significantly from that of the sky light 8. The remaining ambient light 9 is not polarized or diffusely polarized and therefore shows a constant intensity for each polarization angle. In this example, the above-mentioned stray light is therefore composed of the sky light 8 and the remaining ambient light 9.

[0029] By referring to Fig.1, the amounts of light coming from the display, the sky, and the rest of the environment can be separated from one another in this example because these light sources have different polarization properties. Performing an analysis across multiple individual images with different polarization angles allows for a precise determination of the respective polarization angles of the light 7, 8, and 9 from these different light sources. Through digital post-processing based on this, it is possible in this example to eliminate any reflections from the display surface 6 of the mobile device 4, including diffusely scattered light 9, in the display content of the mobile device 4 captured by the at least one interior camera 3. List of reference symbols 1 vehicle 2 Interior 3 at least one interior camera 4 mobile device 5 at least one polarization filter 6 display area 7 Display light 8 Light coming from heaven 9 remaining ambient light 10 total light passing from the display surface to the camera 11 Control unit

Claims

[1] Method for suppressing reflections on display surfaces (6) of mobile devices (4) when capturing their display contents in a vehicle (1), comprising the steps: - capturing a display content of a mobile device (4) used in the vehicle (1) by at least one interior camera (3) of the vehicle (1) using at least one predetermined polarization filter (5); - analyzing the captured display content for the presence of corresponding polarization properties in the display light (7) of this device (4), on the one hand, and in the case of stray light (8, 9) that is mixed with the display light (7) via reflection on its display surface (6), on the other hand; and - Using polarization properties of the display light (7) and / or the stray light (8, 9) extracted thereby for digital post-processing of this or subsequently captured display contents, in which at least a portion of the stray light (8, 9) is subtracted in individual image pixels in order to increase the proportion of the display light (7) therein and to reduce or almost completely remove the proportion of the stray light (8, 9). [2] The method according to claim 1, wherein - at least one of said polarization filters (5) is a static polarization filter designed to filter out the light of a predetermined polarization typical of the light (8) coming from the sky; - during the analysis, the individual image pixels of the captured display content are checked for the presence and proportion of light (8) with this polarization and a proportion of this light (8) to be subtracted is determined therefrom. [3] Method according to claim 1 or 2, wherein - the at least one interior camera (3) is designed for simultaneous image recording with and without a predetermined static polarization filter (5); - a display content of the mobile device (4) is captured simultaneously with and without this polarization filter (5); and - two images thus obtained are compared pixel by pixel in the said analysis of the polarization properties of the display light (7) and the stray light (8, 9). [4] Method according to one of the preceding claims, wherein - at least one of the interior cameras (3) is designed to record several images directly one after the other, each with a different predetermined polarization filter; - a display content of the mobile device (4) is captured with each of these polarization filters (5) directly one after the other; and - several images thus obtained are compared pixel by pixel with each other in the said analysis of the polarization properties of the display light (7) and the stray light (8, 9). [5] Method according to one of the preceding claims, wherein - the display light (7) has a predetermined polarization which differs from a typical polarization of the light (8) coming from the sky; and - in the at least one interior camera (3) two or more different polarization filters (5) are used, which make it possible to separate the portion of the display light (7) from the portion of the light coming from the sky (8) and from the portion of the remaining, non-polarized ambient light (9) in images of the display content captured simultaneously or directly one after the other with these different polarization filters (5). [6] Method according to one of the preceding claims, wherein - the respective display content of the mobile device (4) is forwarded to a vehicle-mounted control unit for a predetermined further use after the aforementioned digital post-processing. [7] Control unit (11) which is designed and arranged to automatically carry out the method according to one of the preceding claims. [8] Vehicle (1), in particular a motor vehicle, comprising: - an interior space (2) for one or more vehicle occupants; - at least one interior camera (3) which is designed and configured to capture display contents of a mobile device (4) used in the vehicle (1) and comprises at least one predetermined polarization filter (5); and - a control unit (11) according to claim 7.

Images